• Peeking at RubyGems Package Signing

    I last wrote about NuGet signing for packages. This has been a hot topic for some folks in the approach that is being taken. However, signing packages was something I didn’t have a whole lot of data on. I didn’t have a good feel for how package communities adopt signing, and decided to get a little more information.

    I turned to the RubyGems community. Gems support signing, also with X509 certificates like the NuGet proposal. Support has been there for a while, so the community there has been plenty of time for adoption. This is on top of a high profile hack on RubyGems, giving plenty of motivation for developers to consider signing their packages.

    Problem is, there isn’t a whole lot of information about it that I could find, so I decided to create it. I decided to look at the top 200 gems and see where they stood on signing.

    The Gems

    The top 200 list is based off of RubyGems own statistics. One problem: their list by popularity only gives up to 100 gems. Fortunately, RubyGems doesn’t do such a hot job on validating their query strings. If I change the page=10 URL query string, supposedly the last page, to page=11, it is quite happy to give me gems 101-110. So first problem solved.

    Many of these gems are supporting gems. That is, not gems that people typically include in their projects directly, but rather included by as a dependency of another gem.

    Getting the latest version of each gem is easy enough with gem fetch. After building our list of gems, we just cache them to disk for inspection later.

    Extracting Certificates

    Certificates can be extracted from gems using gem spec <gempath> cert_chain. This will dump the certificate chain as a YAML document. We can use a little bit of ruby to get the certificates out of the YAML document and as files on disk.

    The Results

    I will be the first to admit that 200 gems is not a huge sample. However, they represent the most popular gems and the ones I would typically expect to be signed.

    Of the 200 gems specified, 17 were signed. That’s approximately 12% of gems. Initially I didn’t know what to think of that number. Is it good? Is it bad? If you had asked me to guess, I would have thought only three or four of them would have been signed. I don’t think 17 is good, either. It’s just not as bad as I would have expected it to be.

    The next matter is, what is the quality of the signatures? Are they valid? Are they self signed? What digest algorithms and key sizes are used?

    Of the 17 signed gems, two of them weren’t really signed at all. They contained placeholders for the certificate to go. Indeed, performing gem install badgem -P HighSecurity resulted in Gem itself thinking the signature was invalid. So we are down to 15 signed gems.

    Some other interesting figures:

    • 15/15 of them were self signed.
    • 2/15 of them used SHA2 signature algorithms. The rest used SHA1.
    • 4/15 were expired.
    • 8/15 used RSA-2048; 1/15 used RSA-3072; 6/15 used RSA-4096.

    Data

    I set up a GitHub repository for the scripts used to create this data. It is available at vcsjones/rubygem-signing-research. Everything that you need to extract the certificates from Gems is there.

    The gemlist.txt contains the list of Gems examined. The fetch.sh script will download all of the Gems in this file.

    extract_certs.sh will extract all of the certificates to examine how you see fit.

    Thoughts

    It doesn’t seem like signing has really taken off with RubyGems. Part of the issue is that RubyGems simply doesn’t validate the signature by default. This is due to the default validation option in Gem being NoSecurity at the time of writing. Every single Gem that is signed would fail to install with the MediumSecurity trust policy:

    gem install gemname -P MediumTrust
    

    This will fail for one reason or another, usually because the certificate doesn’t chain back to a trusted root certificate.

    I’m not sure if this is indicative of how adoption will go for NuGet. I’m curious to see where NuGet is three years from now on signing.

  • NuGet Package Signing

    Recently the NuGet team announced they were going to start supporting package signing.

    The NuGet team announced that their solution would be based on x509, or PKI certificates from a traditional Certificate Authority. They haven’t announced much beyond that, but it’s likely to be just a plain Code Signing certificate with the Code Signing EKU. Certificates and PKI is not a perfect solution. Particularly, one of the problems around code signing certificates is the accessibility of them. Certificate Authorities typically charge for certificates and require identification.

    This presents a problem to a few groups of people. Young people who are just getting in to software development may be excited to publish a NuGet package. However getting a code signing certificate may be out of their reach, for example for a 15 year old. I’m not clear how a CA would handle processing a certificate for a minor who may not have a ID. The same goes for individuals of lesser privileged countries. The median monthly income of Belarus, a country very dear to me, is $827. A few hundred dollars for a code signing certificate is not nothing to sneeze at. There are many groups of people that will struggle with obtaining a certificate.

    Not signing might be okay, with a few exceptions. The first being that the NuGet team described that there would be a visual indicator for signed packages.

    Visual Studio Signed Package from https://blog.nuget.org/20170417/Package-identity-and-trust.html

    This indicator is necessary for part of the NuGet team’s desire to indicate a level of trustworthiness. However, as a package consumer, the indicator will likely draw preference. This puts packages that are able to sign in a position of preference over unsigned packages. This also hurts the community as a whole; it’s simply better for everyone if as many packages as possible were signed.

    Given that, the natural conclusion may be that x509 and PKI are not the correct solution. There are other options that will work, such as PGP and Web of Trust (WOT). Some members are asking the NuGet team to reconsider x509 and PKI. There are other issues with x509 and PKI, but the accessibility of code signing certificates seems to be the central point of the community’s concerns.

    I am sympathetic to these concerns which I have also expressed myself previously. However despite that, I would like to now explain why I think the NuGet team made the right decision, and why the other options are less likely to be workable solutions.

    PKI

    x509 Code Signing certificates use Public Key Infrastructure, or PKI for short. The hardest part of signing anything with a key is not a technical problem. It is “Should I trust this key?”. Anyone in the world can make a certificate with a Common Name of “Kevin Jones” and sign something with it. How would you, the consumer of NuGet package signed by CN=Kevin Jones, know that the certificate belongs to Kevin Jones?

    The PKI solution for that is to have the certificate for CN=Kevin Jones to be signed by someone you already trust, or in this case a Certificate Authority. The CA, since they are vouching for your certificate’s validity, will vet the application for the certificate. Even when I applied for a free Code Signing certificate (disclosure, Digicert gives free certs to MVPs, which I am grateful for), they still performed their verification procedures which involved a notarized document for my identification. CAs are motivated to do this correctly every time because if they prove to be untrustworthy, the CA is no longer trusted anymore. The CA’s own certificate will be removed or blacklisted from the root store, which operating systems maintain themselves, usually.

    While this has problems and is not foolproof, it has been a system that has worked for quite a long time. x509 certificates are well understood and also serve as the same technology as HTTPS. There is significant buy in from individuals and businesses alike that are interested in the further advancement of x509. Such advancements might be improved cryptographic primitives, such as SHA256 a few years ago, to new things such as ed25519.

    A certificate which is not signed by a CA, but rather signs itself, is said to a self-signed certificate. These certificates are not trusted unless they are explicitly trusted by the operating system for every computer which will use it.

    A final option is an internal CA, or enterprise CA. This is a Certificate Authority that the operating system does not trust by default, but has been trusted through some kind of enterprise configuration (such as Group Policy or a master image). Enterprises choose to run their own private CA for many reasons.

    Any of these options can be used to sign a NuGet package with x509. I’m not clear if the Microsoft NuGet repository will accept self signed packages or enterprise signed packages. However an enterprise will be able to consume a private NuGet feed that is signed by their enterprise CA.

    This model allows for some nice scenarios, such trusting packages that are signed by a particular x509 certificate. This might be useful for an organization that wants to prevent NuGet packages from being installed that have not been vetted by the corporation yet, or preventing non-Microsoft packages from being installed.

    Finally, x509 has not great, but at least reasonably well understood and decently documented tools. Let’s face it: NuGet and .NET Core are cross platform, but likely skew towards the Windows and Microsoft ecosystem at the moment. Windows, macOS, and Linux are all set up at this point to handle x509 certificates both from a platform perspective and from a tooling perspective.

    PKI is vulnerable to a few problems. One that is of great concern is the “central-ness” of a handful of Certificate Authorities. The collapse of a CA would be very problematic, and has happened, and more than once.

    PGP

    Let’s contrast with PGP. PGP abandons the idea of a Certificate Authority and PKI in general in favor for something called a Web of Trust. When a PGP key is generated with a tool like GPG, they aren’t signed by a known-trustworthy authority like a CA. In that respect, they very much start off like self-signed certificates in PKI. They aren’t trusted until the certificate has been endorsed by one, or multiple, people. These are sometimes done at “key signing parties” where already trusted members of the web will verify the living identity to those with new PGP keys. This scheme is flexible in that it doesn’t rely on a handful of corporations.

    Most importantly to many people, it is free. Anyone can participate in this with out monetary requirements or identifications. However getting your PGP key trusted by the Web of Trust can be challenging and due to its flexibility, may not be be immediately actionable.

    It’s likely that if NuGet did opt to go with PGP, a Web of Trust may not be used at all, but rather to tie the public key of the PGP key to the account in NuGet. GitHub actually does something similar with verified commits.

    Github GPG

    This, however, has an important distinction from an x509 code signing certificate: the key does not validate that Kevin Jones the person performed the signature. It means that whoever is in control of the vcsjones Github account performed the signature. I could have just as easily created a GitHub account called “Marky Mark” and created a GPG key with the email markymark@example.com.

    That may be suitable enough for some people and organizations. Microsoft may be able to state that, “our public key is ABC123” and organizations can explicitly trust ABC123. That would work until there was a re-keying event. Re-keying is a natural and encouraged process. Then organizations would need to find the new key to trust.

    This is harder for individuals. Do I put my public key on my website? Does anyone know if vcsjones.com is really operated by someone named Kevin Jones? What if I don’t have HTTPS on my site - would you trust the key that you found there?

    Adopting in to the “web of trust” tries to work around that problems of key distribution. However the website evil32.com puts it succinctly:

    Aren’t you suppose to use the Web of Trust to verify the authenticity of keys?

    Absolutely! The web of trust is a great mechanism by which to verify keys but it’s complicated. As a result, it is often not used. There are examples of GPG being used without the Web of Trust all over the web.

    The Web of Trust is also not without its problems. An interesting aspect of this is since it requires other users to vouch for your key, you are now disclosing your social relationships, possibly because you are friends with the other people used to vouch for the key.

    It also has a very large single point of failure. Anyone that is part of the strong set is essentially a CA compared to x509 - a single individual compromised in the strong set could arguably be said to compromise the entire web.

    For those reasons, we don’t see the WOT used very often. We don’t see it used in Linux package managers, for example.

    Linux, such as Debian’s Aptitude, use their own set of known keys. By default, a distribution ships with a set of known and trusted keys, almost like a certificate store. You can add keys yourself using apt-key add, which many software projects ask you to do! This is not unlike trusting a self signed certificate. You have to be really sure what key you are adding, and that you obtained it from a trustworthy location.

    PGP doesn’t add much of an advantage to x509 in that respect. You can manually trust an x509 just as much as you can manually trust a PGP key.

    It does however mean that the distribution now takes on the responsibilities of a CA - they need to decide which keys they trust, and the package source needs to vet all of the packages included for the signature to have any meaning.

    Since PGP has no authority, revoking requires access to the existing private key. If you did something silly like put the private key on a laptop and lose the laptop, and you didn’t have the private key backed up anywhere, guess what? You can’t revoke it without the original private key or a revoke certificate. So now you are responsible for two keys: your own private key and the certificate that can be used to revoke the key. I have seen very little guidance in the way of creating revoke certificates. This isn’t quite as terrible as it sounds, as many would argue that revocation is broken in x509 as well for different reasons.

    Tooling

    On a more personal matter, I find the tooling around GnuPG to be in rough shape, particularly on Windows. It’s doable on macOS and Linux, and I even have such a case working with a key in hardware.

    GPG / PGP has historically struggled with advancements in cryptography and migrating to modern schemes. GPG/PGP is actually quite good at introducing support for new algorithms. For example, the GitHub example above is an ed25519/cv25519 key pair. However migrating to such new algorithms is has been a slow process. PGP keys have no hard-set max validity, so RSA-1024 keys are still quite common. There is little key hygiene going on and people often pick expiration dates of years or decades (why not, most people see expiration as a pain to deal with).

    Enterprise

    We mustn’t forget the enterprise, who are probably the most interested in how to consume signed packages. Frankly, package signing would serve little purpose if there was no one interested in the verify step - and we can thank the enterprise for that. Though I lack anything concrete, I am willing to bet that enterprises are able to handle x509 better than PGP.

    Wrap Up

    I don’t want to slam PGP or GnuPG as bad tools - I think they have their time and place. I just don’t think NuGet is the right place. Most people that have interest in PGP have only used it sparingly, or are hard-core fanatics that can often miss the forest for the trees when it comes to usable cryptography.

    We do get some value from PGP if we are willing to accept that signatures are not tied to a human being, but rather a NuGet.org account. That means signing is tied to NuGet.org and couldn’t easily be used with a private NuGet server or alternative non-Microsoft server.

    To state my opinion plainly, I don’t think PGP works unless Microsoft is willing to take on the responsibility to vet keys, we adopt in to the web of trust, or we accept that signing does not provide identity of the signer. None of these solutions are good in my opinion.

  • Single Cable USB-C

    I had a pretty simple dream. I have a 4K monitor, a touch pad, keyboard, and a few computers between Lena and I. The dream was to have a single cable to connect a computer to this set up.

    I generally love the Mac keyboard and track pads. The second version of these can also work with vanilla USB instead of bluetooth, which is great for a docking setup. No need to do any re-pairing of the devices or shuffling things around on my desk.

    The single cable desire came from that fact that one of the computers is a MacBook. Not a pro, the 12” MacBook with a single USB-C port. Single cable, including charging, was a necessity.

    I thought this would be easy. Turns out, not so much. I went through a few different docks or “dongles” trying to find the one that made the right trade offs.

    Why not Thunderbolt?

    Two reasons. The first being that the MacBooks don’t do Thunderbolt, only USB-C. They use identical ports, and Thunderbolt 3 ports can also double as a USB-C port, but not the reverse. Thunderbolt 3 docks, while expensive, can typically do everything over a single cable, and they do it well. I’ve had experience with that going back to the 2011 Thunderbolt Display that Apple introduced.

    Generally now I am hesitant to adopt in to a Thunderbolt set up. While at the time of purchasing the Thunderbolt display I had all-thunderbolt devices, that isn’t true today, and they didn’t work with the Thunderbolt display.

    Because of that, I don’t want to invest heavily in a set up that might not work for me in the future. Instead, I wanted a regular monitor that had a variety of different inputs. DisplayPort, HDMI, mDP, etc and also a little USB hub.

    The monitor

    I settled on the advice of many. The Dell P2715Q. It’s a good 4K monitor for a software developer. It looks great, has bezels but I don’t really care about that, and has all of the inputs I wanted. Not much of an issue there.

    Docks

    This is where I started learning that despite some people’s belief that USB-C was going to obsolete Thunderbolt 3, they were not correct. USB-C and USB 3.1 Gen 2 seem to have a ways to go to catching up to Thunderbolt.

    Here was my wish list from a hub or dock, in the order that I consider them.

    1. Pass-through power
    2. 4K Resolution for Display
    3. 60 Hz refresh rate for Display
    4. USB Hub
    5. Ethernet

    There is no device on the market at the time or writing that hits all of these. Some, like the Arc Hub, are really close, but misses one mark which I’ll talk about later.

    Pass Through Power

    The first, pass through power, is a requirement for the 12” MacBook. It literally has only one port, including the one for charging it. Pass through power allows charging the device and using the port for USB at the same time. It sounds simple, but it’s not. There are a few problems here.

    The first is how much power is can pass-through. It’s not as simple as passing power through. Some hubs or docks can deliver 30 watts or so, others can go up to 60 watts, etc. Depending on the charging needs of your computer, it’ll either charge your computer very slowly, or not at all, depending on how much power it can deliver.

    The second that I’ve heard is that some hubs work fine when there is no pass through power connected, but then certain things start acting strange depending on the computer and power supply. Apple hardware doesn’t vary much in that regard, so I hoped this wouldn’t be an issue with a Mac.

    The Display

    I’m going to discuss the next two together because they go together, and can often fight each other. 4K monitors can do a few different modes. They can do 4K@60Hz, 4K@30Hz, or 1080p@60Hz. The type of connector used and the settings on the computer determine which is the best that you’re going to get.

    4K@60Hz requires a modern connector. HDMI 2.0 or DisplayPort 1.2. If your computer’s graphics can do 4K and you have an output port of one of those two kinds, then you get do 4K@60Hz. If not, then you will have to choose between 2K@60Hz or 4K@30Hz.

    The resolution of 4K is nice, but 60Hz refresh rate should be a bare minimum. Having seen 30 Hz playing around with settings, it’s quite choppy in its appearance for things like dragging or animations.

    Finding a single USB-C dock that could drive even one display at 4K@60Hz was a challenge. To make this a little more complicated, I was slightly mislead by USB-C a little bit.

    Let me explain. I have a USB-C to HDMI 2.0 cable. It works great, plug one end in to USB-C, plug the other in to the monitor, and instant 4K@60Hz. It shouldn’t be much of a stretch to add inline power and USB hub, right? That’s where I went astray. USB-C includes alternate modes, where the it has a physical port of USB-C, but isn’t actually USB-C. This cable was using USB-C’s alternate mode. It was not doing anything USB related, it was just acting like an HDMI 2.0 port with a USB-C interface.

    After doing some reading on this matter, I believe I came to the conclusion that a USB-C port cannot offer USB-C data and HDMI 2.0 at the same time - only HDMI 1.4b. So the 4K works, but with a 30 Hz refresh rate. To get the 4K@60Hz, I needed to find a docking solution that had DisplayPort, where the USB-C spec did use DisplayPort 1.2.

    This was quite the treasure hunt.

    USB Hub

    The monitor itself has a USB Hub, I just need somewhere to connect it to on the hub. This is used for a keyboard and a track pad. That’s really it. If I really needed a USB port with raw speed, I would use one of the other ports on the computer - it just wouldn’t be part of the docking setup.

    Ethernet

    This was a nice-to-have. WiFi 802.11ac Wave 2 is generally good enough for me, I can still make use of my near-gigabit internet connection, and the minor latency penalty isn’t trouble. The APs are rock solid in my house. Though, if a dock had an Ethernet port, I wouldn’t pass up the opportunity to have it if it came down to putting the money in.

    Shopping

    I initially used the Apple HDMI dongle. It has power pass-through, 1 USB port, and HDMI. Everything worked great except the 4K@60Hz, which I eventually figured out I would never get working. Still, I like this little dongle for travelling.

    It would take me one more attempt at purchasing an HDMI hub and dongle before I caught on that HDMI and USB-C just don’t do the 4K with the right refresh rates. This was complicated by the fact that some manufacturers don’t label the refresh rate. One reviewer of a product said the HDMI did 4K@60Hz, but I have to believe that reviewer was mistaken. Lesson learned: only buy a hub where the specs are completely spelled out, and reviewers aren’t disputing them.

    I was initially going to pull the trigger on the Arc Hub, something that The Verge gave very glowing remarks to. I came within moments of purchasing one, except reading the Compatibility section carefully…

    It is also a known issue that 4K@60hz is not being supported via MDP using a MacBook Pro despite having the capability to do so. Our engineers are thoroughly looking into the issue and will update with any new info.

    That’s rather disappointing! I hope they fix the issue, and if they do, I will certainly consider revisiting purchasing one of them. I am very pleased that they were up-front about the compatibility issue in the first place, so I am not going to discount them. Their technical specifications are impeccably documented.

    I then took a look at the “HyperDrive USB-C Hub with Mini DisplayPort”. This hit a few good points. It’s specifically documented as supporting 4K@60Hz, pass through power, and 2 USB 2.0 ports. USB 3.0 would be nice, but, I’ll take it.

    The only other thing was it really isn’t a single “cable” solution. It was just a little dongle with no cable at all. This was less pleasant because three thick cables were connected to it, and wrestling it in to place was annoying. It also meant that I would have three cables across my desk. This problem ended up being easy to fix with a USB-C extension cable.

    Overall, I’m a tad annoyed by the experience of this. USB-C is confusing and difficult to find the right components, whereas Thunderbolt is in it’s 3rd iteration and it seems that many of its problems have been addressed since using the Thunderbolt Display.

  • Subject Interface Packages - Part 2

    In part 1 of Subject Interface Packages, we left off being able to sign a custom file type, but not verify it. Here we are going to implement the two functions to perform verification.

    This is in some ways, the reverse of what we did in the previous part. Instead of injecting the signature data in to the file, we need to extract it, and instead of creating a digest, we need to create and compare the digest.

    Get Signed Data Message

    The last part we did was “putting” the signature data, now we need to do the reverse. Given a file handle, get the signed data message.

    BOOL WINAPI PngCryptSIPGetSignedDataMsg(
        SIP_SUBJECTINFO *pSubjectInfo,
        DWORD* pdwEncodingType,
        DWORD dwIndex,
    	DWORD *pcbSignedDataMsg,
        BYTE *pbSignedDataMsg
    )
    

    The purpose of this function is that upon successful completion, pbSignedDataMsg will point to the signed data message that we embedded in the file with CryptSIPPutSignedDataMsg.

    Other things that will need to be set is the pcbSignedDataMsg which is the size of the signed data message, and pdwEncodingType which is the encoding type.

    We can knock out pdwEncodingType easily because we can set it to an either/or as we see in many of the other CMS APIs:

    //TODO: validation
    *pdwEncodingType = X509_ASN_ENCODING | PKCS_7_ASN_ENCODING;
    

    The authenticode process will call this function twice. Once with pbSignedDataMsg pointing to NULL, and it is expected that pcbSignedDataMsg will be set with the size of the buffer that Win32 should allocate. The second call to the function will have pbSignedDataMsg pointing to a buffer of memory that is at least as big as the the indicated size from the first call.

    A pseudo-code implementation would look something like this:

    BOOL WINAPI PngCryptSIPGetSignedDataMsg(
        SIP_SUBJECTINFO *pSubjectInfo,
        DWORD* pdwEncodingType,
        DWORD dwIndex,
    	DWORD *pcbSignedDataMsg,
        BYTE *pbSignedDataMsg
    ) {
        //TODO: validation
        *pdwEncodingType = X509_ASN_ENCODING | PKCS_7_ASN_ENCODING;
        if (NULL == pbSignedDataMsg) {
            DWORD size;
            if (GetSignedDataMsgSize(pSubjectInfo->hFile, &size)) {
                *pcbSignedDataMsg = size;
                return TRUE;
            }
            return FALSE;
        }
        return GetSignedDataMsg(pSubjectInfo->hFile, pcbSignedDataMsg, pbSignedDataMsg));
    }
    

    Where GetSignedDataMsg will fill pbSignedDataMsg with the data message.

    You don’t have to do any verification with this. Internally, Win32 will use CryptVerifyMessageSignature to verify the message and integrity of the signature itself.

    If you are having trouble at this step, it’s worth pointing out that you can call CryptVerifyMessageSignature yourself at this point to verify that you’re extracting the signature from the file correctly. You should also be able to run this through an ASN.1 decoder and see properly decoded output.

    It should also be byte-for-byte identical to the “put” operation in part 1, so you can compare at these two steps.

    Verify Indirect Data

    The last step is to verify the hash that was signed.

    BOOL WINAPI PngCryptSIPVerifyIndirectData(
        SIP_SUBJECTINFO *pSubjectInfo,
        SIP_INDIRECT_DATA *pIndirectData)
    

    The first parameter gives us information about the file being verified. In this step you will need to re-hash the file, just the same way that was done in the very beginning. You then need to compare this hash with pIndirectData->Digest.pbData.

    If the hashes match, you should return TRUE and use SetLastError to ERROR_SUCCESS to indicate the hash is correct. If the hashes are not equal, you should return FALSE and use TRUST_E_SUBJECT_NOT_TRUSTED with SetLastError to indicate that there was no unexpected error, just that the signatures do not match.

    pIndirectData->Digest will contain the digest algorithm. Valid that it is correct, and that the parameters are what you expect. In cases for digests used for authenticode digests, the parameters will either be a literal NULL or more likely, {0, 5} as an ASN.1 NULL.

    Final Thoughts

    This is a rough beginning on writing a SIP. As mentioned in the first post, a GitHub project for a PNG SIP exists and can perform these basic operations. As a reminder, this code exists for demonstration purposes, and not for any real-world use.

    There is still plenty to do in later parts. As of now, we cannot:

    • Remove a signature
    • Timestamp
    • Dual Sign
    • Seal

    I hope to get to these soon for this project. More curiously, a slightly-related subject has me to believe this can be done in Rust. I don’t think it’s a stretch either, or an abuse. Rust seems well suited for the task without going down to C or C++.

    The last part, sealing, is a ways off, because the details of sealing signatures is not public yet, and are known only due to some careful inspection.

  • Subject Interface Packages - Part 1

    Authenticode continues to be an area of interest for me, and the surprisingly little documentation around some of its features remains an issue for others. Authenticode is interesting because it can sign a variety of formats. Executables and libraries (PE files), MSI installers, CAB files, and more recently APPX packages, which are used to distribute things in the Windows Store.

    The Authenticode process has seemingly simple process of creating a canonical digest of the file, signing it, and embedding the signature in the file. This simple task requires Authenticode to actually understand the file format. Take an executable as an example. Authenticode must first create a digest of this file. However, it cannot do so by simply hashing the whole file. It must be intelligent to ignore certain parts of the file, including where the signature itself will go. When it comes time to validate the signature, the Authenticode process will first validate that the signature on the embedded hash is correct, then re-hash the file and compare the hashes. If during the re-hash process it were to include the embedded signature, then the hashes would not be equal.

    Authenticode must further be smart enough to understand how to embed the signature in a file that doesn’t corrupt the file itself. PE files have specific regions for placing digital signatures called the certificate section. Finally, Authenticode must be able to do this for every file type that it can sign.

    Now if I were a good software architect, I would design the Windows components of Authenticode to create an abstraction over understanding file types. Perhaps something like the bridge pattern in software design. Fortunately for us, we do have something like the bridge pattern between the signing and understanding a file’s format: Subject Interface Packages.

    Subject Interface Packages, or “SIP” (referred to as such going forward), are a way to extend the types of files that Authenticode can understand how to sign and verify. Windows uses SIPs internally for its own file formats, but it’s actually something anyone can use.

    If you can develop a SIP, then you can teach Authenticode new file formats, including giving signtool the knowledge to sign a file, and Explorer the ability to show the “Digital Signatures” tab on a file.

    To help make this a bit more digestible, I included a fully-functioning SIP for signing PNG files over on GitHub.

    SIP Basics

    Under the covers, Windows’s signing operations uses SignerSignEx* to Authenticode sign files. The most recent versions of Windows use SignerSignEx3, while older ones use SignerSignEx2.

    A typical signing operation, loosely, has a few key phases. First, a digest of the file needs to be made. Second, the digest needs to be signed. In Authenticode, this is done with CMS style messages that result in a big chunk of PKCS#7 data. Third, and finally, the CMS message needs to be embedded in the file, somehow. The embedded signature should not affect the behavior of the file.

    A SIP allows you to extend the first and third steps, but not the second. A SIP does not allow you to modify the entire Authenticode process, though there are some ways (to be discussed another day) to allow modifying the actual signing step. This means a SIP does not allow you to to do something other than Authenticode or support any possible digital signing scheme, like XmlDSig. It’s still Authenticode-style CMS signature and SignerSignEx does step two for you, including the certificate selection, private key locating, etc.

    Developing a SIP has a few requirements of the file format itself, mainly that the file format needs to support some way of embedding the signature without breaking the file itself.

    Using PNG as an example, I can’t just put the signature at the end of the file. An image viewer would see those bytes, and not know what to do with them, and assume the file is corrupt. Fortunately for us, the designers of PNG thought the format should provide some extensibility, so we can continue. However for other formats that have no concept of metadata, embedding the Authenticode signature may not be possible. You can still used detached signatures at this point, but that’s for another time.

    SIP Structure

    A SIP is little more than a native DLL. There needs to be a way to register and un-register it, and it needs to implement the bare minimum of SIP functionality by exporting a few functions.

    During registration phase, a SIP needs to declare what it can, and cannot do. There are at least five things the SIP must do. For our PNG SIP on GitHub, we implement the bare requirements, but I’ll continue to develop it and write new posts as I make more progress.

    A SIP needs to perform these five things.

    1. It needs to identify if it is capable of handling a particular file.
    2. It needs to support digesting the file.
    3. It needs to support embedding the signature.
    4. It needs to support extracting the signature.
    5. It needs to support verifying the digest.

    Once we can do those five things, we can round trip a signature.

    I decided to write mine in C, but as long as your abide by the Win32 ABI and calling conventions, you can write a SIP in any language you would like. I am keen to try doing this in Rust myself.

    Optionally, the registration functionality may or may not be part of the library itself. You could write an external program that knows how to register the SIP, or the library could implement Dll(Un)RegisterServer and then use regsvr32 to do the registration.

    It’s also worth pointing out that you will want a 32-bit and a 64-bit version of your SIP, and you will need to register it twice, once as 64-bit, and another as 32-bit.

    The final aspect of SIP will be a unique GUID for it. This GUID should be the same for all platforms.

    Before we can start writing code, we need to get a project in to a compilable state. You’ll need a variety of headers, but you’ll also want to make sure you link against the following libraries for our project:

    • Crypt32.lib
    • BCrypt.lib
    • NCrypt.lib

    The main feature headers are Mssip.h and wincrypt.h. Those two you may want to consider including in your pre-compiled header for this project.

    Registering

    A SIP is registered with the function CryptSIPAddProvider, which takes a single structure describing what your SIP can do, as well as its GUID.

    For my PNG library, I decided the simplest approach is to hard-code the path and register it with DllRegisterServer to easily use regsvr32.

    Let’s pick a GUID first:

    // {DA005D72-4E32-4D5E-94C5-41AECBA650FA}
    DEFINE_GUID(GUID_PNG_SIP,
    	0xda005d72, 0x4e32, 0x4d5e, 0x94, 0xc5,
    	0x41, 0xae, 0xcb, 0xa6, 0x50, 0xfa);
    

    You will want to generate a new GUID for your own project. You can use the “guidgen” program in the Windows SDK, it even has a display format that is friendly for DEFINE_GUID.

    A very simple implementation of this might looks like this now:

    STDAPI DllRegisterServer()
    {
    	SIP_ADD_NEWPROVIDER provider = { 0 };
    	GUID subjectGuid = GUID_PNG_SIP;
    	provider.cbStruct = sizeof(SIP_ADD_NEWPROVIDER);
    	provider.pgSubject = &subjectGuid;
    #ifdef _WIN64
    	provider.pwszDLLFileName = L"C:\\Windows\\System32\\pngsip.dll";
    #else
    	provider.pwszDLLFileName = L"C:\\Windows\\SysWOW64\\pngsip.dll";
    #endif
    	provider.pwszGetFuncName = L"PngCryptSIPGetSignedDataMsg";
    	provider.pwszPutFuncName = L"PngCryptSIPPutSignedDataMsg";
    	provider.pwszCreateFuncName = L"PngCryptSIPCreateIndirectData";
    	provider.pwszVerifyFuncName = L"PngCryptSIPVerifyIndirectData";
    	provider.pwszIsFunctionNameFmt2 = L"PngIsFileSupportedName";
    	if (CryptSIPAddProvider(&provider))
    	{
    		return S_OK;
    	}
    	else
    	{
    		return HRESULT_FROM_WIN32(GetLastError());
    	}
    }
    

    Lets go through this line by line of DllRegisterServer. The first two lines create our struct and initializes it to zeros so everything is a clean “NULL”. There are a number of Win32 patterns here that should be familiar to Win32 developers. Particularly, setting the size of the struct as the first field. We then set our GUID, but we copy it locally first so we can take a pointer to it.

    pwszDLLFileName accepts the full path to the library. Note that this path must be how 64-bit Windows sees it. System32 is normally a directory that WOW64 does file system redirection. However, the path should be presented as if there is no file system redirection being performed. This can be meddlesome if you are trying to determine that current path of the library to dynamically determine it when WOW64 is in play from whatever is performing the registration.

    I instead just hard coded the path as I don’t really expect the SIP to be installed elsewhere. Note that this is a bit of a naive approach at the moment because it does not allow installation on a 32-bit Windows.

    The rest of the fields on the struct are names of exports for the functionality of the SIP. Their names don’t matter, but I would make them unique and not collide with other function names in Win32.

    The function definitions for these are loosely defined in SIP_ADD_NEWPROVIDER, but we can stub them out for now and just do return FALSE. All together, our SIP should look something like this, so far:

    STDAPI DllRegisterServer()
    {
    	SIP_ADD_NEWPROVIDER provider = { 0 };
    	GUID subjectGuid = GUID_PNG_SIP;
    	provider.cbStruct = sizeof(SIP_ADD_NEWPROVIDER);
    	provider.pgSubject = &subjectGuid;
    #ifdef _WIN64
    	provider.pwszDLLFileName = L"C:\\Windows\\System32\\pngsip.dll";
    #else
    	provider.pwszDLLFileName = L"C:\\Windows\\SysWOW64\\pngsip.dll";
    #endif
    	provider.pwszGetFuncName = L"PngCryptSIPGetSignedDataMsg";
    	provider.pwszPutFuncName = L"PngCryptSIPPutSignedDataMsg";
    	provider.pwszCreateFuncName = L"PngCryptSIPCreateIndirectData";
    	provider.pwszVerifyFuncName = L"PngCryptSIPVerifyIndirectData";
    	provider.pwszIsFunctionNameFmt2 = L"PngIsFileSupportedName";
    	if (CryptSIPAddProvider(&provider))
    	{
    		return S_OK;
    	}
    	else
    	{
    		return HRESULT_FROM_WIN32(GetLastError());
    	}
    }
    
    
    BOOL WINAPI PngIsFileSupportedName(WCHAR *pwszFileName, GUID *pgSubject)
    {
    	return FALSE;
    }
    
    BOOL WINAPI PngCryptSIPGetSignedDataMsg(SIP_SUBJECTINFO *pSubjectInfo,
    	DWORD* pdwEncodingType, DWORD dwIndex, DWORD *pcbSignedDataMsg,
    	BYTE *pbSignedDataMsg)
    {
    	return FALSE;
    }
    
    BOOL WINAPI PngCryptSIPPutSignedDataMsg(SIP_SUBJECTINFO *pSubjectInfo,
    	DWORD dwEncodingType, DWORD *pdwIndex,
    	DWORD cbSignedDataMsg, BYTE *pbSignedDataMsg)
    {
    	return FALSE;
    }
    
    BOOL WINAPI PngCryptSIPCreateIndirectData(SIP_SUBJECTINFO *pSubjectInfo,
    	DWORD *pcbIndirectData, SIP_INDIRECT_DATA *pIndirectData)
    {
    	return FALSE;
    }
    
    BOOL WINAPI PngCryptSIPVerifyIndirectData(SIP_SUBJECTINFO *pSubjectInfo,
    	SIP_INDIRECT_DATA *pIndirectData)
    {
    	return FALSE;
    }
    

    At this point we can verify we are making some progress. We should be able to compile this, put it in System32, and register it with regsvr32. It’s a small step, but we should have a SIP registered that always fails. Now would be a good time to examine what registration is actually doing.

    At the heart of it, all the registration is doing is adding a few keys in the registry under

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\OID\EncodingType 0
    

    Under that key are sub-keys for all of the possible things a SIP can do. We aren’t doing everything, and some of them remain entirely undocumented and to an extent, unused. However, for the five cores parts of a SIP we are supporting, we should see our SIP’s GUID under those registry keys. Namely,

    • CryptSIPDllCreateIndirectData
    • CryptSIPDllIsMyFileType2
    • CryptSIPDllPutSignedDataMsg
    • CryptSIPDllVerifyIndirectData
    • CryptSIPDllGetSignedDataMsg

    If we see our GUID under those keys, have a successfully registered SIP. You will see an identical pattern for 32-bit SIPs except under the WOW6432Node registry key. You don’t have to register both a 32-bit and a 64-bit SIP during development. If you’d like, you can just as well develop a 64-bit only SIP. However, 32-bit uses of your SIP will not work, so you need to make sure you are testing with 64-bit tools, like signtool.

    You’ll also want to make sure your library is exporting all of these functions correctly. Dependency Walker, ancient as it is, does this well. You’ll want to not only make sure they are exported, but also are unmangled.

    Dependency Walker

    File Support

    There are two ways a SIP can decide if a file format is supported. It can do so by just the name of the file, or it can do so by actually examining the contents of the file. For performance, I would recommend implementing just the file name check. You can still bail out of the actual signing process with an error if you determine that you can’t actually sign the file. In our bare-bones implementation above, we exported the function PngIsFileSupportedName and put that in our registration. We need to export a function with that name and return a Win32 BOOL indicating if it is supported.

    A simple approach is to just check the extension of the file by creating a substring and using _wcsicmp. An implementation might look something like this:

    const WCHAR* ext = L".png";
    size_t len = wcslen(pwszFileName);
    if (len < wcslen(ext))
    {
    	return FALSE;
    }
    size_t offset = len - wcslen(ext);
    assert(offset >= 0);
    const WCHAR* substring = &pwszFileName[offset];
    int result = _wcsicmp(substring, ext);
    if (result == 0)
    {
    	*pgSubject = GUID_PNG_SIP;
    	return TRUE;
    }
    return FALSE;
    

    The name of the file that you are given is wide characters, so make sure you use the appropriate functions. You must also specify the GUID of the SIP as an out parameter for pgSubject. In our case we use our own SIP, however you can use this if you have multiple SIPs - you can actually have just one library that determines which SIP GUID to use, or delegate the work to another SIP.

    If this returns TRUE, it moves on to the actual process of creating a digest.

    Create Indirect Data

    The actual structure in the PKCS#7 signature is a structure called indirect data. Among the things in here is the digest of the file. Our next step is we need to create a digest of the file in a function with a signature like this:

    BOOL WINAPI PngCryptSIPCreateIndirectData(
    	SIP_SUBJECTINFO *pSubjectInfo,
    	DWORD *pcbIndirectData,
    	SIP_INDIRECT_DATA *pIndirectData
    	)
    

    This function will actually be called twice, the first time with pIndirectData as NULL, and the second with an address. This pattern may be familiar to Win32 developers. The first call it is expecting you to set the pcbIndirectData parameter with the size of the amount of memory needed for pIndirectData. Internally, the Authenticode system will then allocate the amount of memory you told it to, then call the function again with the address of the memory in pIndirectData. During the second call, it’s up to you to set this as a pointer to a SIP_INDIRECT_DATA structure.

    For the first call, it isn’t as simple as just using sizeof(SIP_INDIRECT_DATA) and going with that. SIP_INDIRECT_DATA has a field called Digest, which is another struct. This is itself a CRYPT_HASH_BLOB, which has two fields. The first is DWORD cbData, and the second is BYTE *pbData.

    This pbData is a pointer to the hash that we need to create. Problem is, a pointer to what? Who owns the memory? We cannot stack allocate it because we need to return it from a function. We can allocate it on the heap, but the SIP doesn’t have a cleanup phase, or a way to tell us “I’m done with that memory”.

    Instead what we need to do is set pcbIndirectData to enough memory for everything that we need to return. The authenticode system owns this memory, and will free it when its done with it. We need to make sure this still points to a SIP_INDIRECT_DATA structure, but we can put anything past that. To make this easier and avoid pointer acrobatics, I defined a new structure that looks like this:

    #define MAX_HASH_SIZE 64
    #define MAX_OID_SIZE 128
    typedef struct INTERNAL_SIP_INDIRECT_DATA_
    {
    	SIP_INDIRECT_DATA indirectData;
    	BYTE digest[MAX_HASH_SIZE];
    	CHAR oid[MAX_OID_SIZE];
    } INTERNAL_SIP_INDIRECT_DATA;
    

    It starts with a SIP_INDIRECT_DATA structure so it still looks like one. After that, I make two more fields. The first is a digest field, which will create 64 BYTEs after the indirectData for us to place our hash. There is no hash algorithm that produces a digest bigger than 64 bytes currently, so 64 seems sufficient. After that we have a CHAR array in the struct which will contain the OID of the hash algorithm. I used a generate 128 CHARs for this, even though most commonly the longest it can be us 22 CHARs.

    We can then set the pbData that introduced this problem to the address of the digest field in our new structure. We will do the same with the OID.

    Roughly, that gives us something like this (all validation is omitted here for brevity - be careful!):

    BOOL WINAPI PngCryptSIPCreateIndirectData(
    	SIP_SUBJECTINFO *pSubjectInfo,
    	DWORD *pcbIndirectData,
    	SIP_INDIRECT_DATA *pIndirectData
    	) {
    	if (NULL == pIndirectData) {
    		*pcbIndirectData = sizeof(INTERNAL_SIP_INDIRECT_DATA);
    		return TRUE;
    	}
    	//TODO: validations
    	INTERNAL_SIP_INDIRECT_DATA* pInternalIndirectData = (INTERNAL_SIP_INDIRECT_DATA*)pIndirectData;
    	memset(pInternalIndirectData, 0, sizeof(INTERNAL_SIP_INDIRECT_DATA));
    	DWORD digestSize;
    	MagicGetOurHash(&digestSize, &pInternalIndirectData->digest[0]);
    	pInternalIndirectData->indirectData.Digest.cbData = digestSize;
    	pInternalIndirectData->indirectData.Digest.pbData = &pInternalIndirectData->digest[0];
    	//TODO: set all other members of the struct including the OID.
    	*pIndirectData = pInternalIndirectData;
    	return TRUE;
    }
    

    For a real implementation of this member, see the GitHub project. This pattern allows us to forgo worrying about the memory cleanup. When authenticode is done with the memory, it frees the whole block that we asked it to allocate.

    When CryptSIPCreateIndirectData is called for the second time, Authenticode expects that, at a minimum, the pIndirectData’s Digest member is correctly filled out. That includes the digest itself, and the OID of the algorithm.

    pSubjectInfo gives us the information we need to compute the digest. It itself includes a DigestAlgorithm structure indicating what digest algorithm it’s asking for. This includes the OID of the algorithm.

    To translate the OID in to a more useful type which allows you to get a CSP or CNG algorithm identifier, you can use CryptFindOIDInfo:

    PCCRYPT_OID_INFO info = CryptFindOIDInfo(
    	CRYPT_OID_INFO_OID_KEY,
    	pSubjectInfo->DigestAlgorithm.pszObjId, //validation omitted here
    	CRYPT_HASH_ALG_OID_GROUP_ID);
    

    One trick with this call us you need to set CRYPT_OID_INFO_HAS_EXTRA_FIELDS before <windows.h> is included. For me that meant putting it in a pre-compiled header near WIN32_LEAN_AND_MEAN. Setting this define puts members on the PCCRYPT_OID_INFO struct that are helpful for use with CNG, like pwszCNGAlgid, which can be used with BCryptOpenAlgorithmProvider.

    Finally, the pSubjectInfo->hFile parameter will give a handle to the file that is being signed.

    It’s worth discussing some best practices at this point about how to digest a file.

    Don’t over-interpret the file

    Let’s use a PE file as an example here. Part of the PE file that is digest is the .text section, or program code. The digesting process does not care what-so-ever about the program code itself. For example, a NOP assembly instruction doesn’t alter the behavior of the program directly, so one could argue that the signing process should attempt to read the x86 assembly and skip over NOP instructions.

    But of course that isn’t done. It isn’t the signing process’s job to understand x86, and the theoretical example above could be used in an attack.

    The PE signing does skip some parts of the PE beyond the signature itself, such as skipping the checksum, because the checksum has to be updated after the signature has been applied.

    Conversely, some canonicalization might be expected. For example, the way a PE’s sections are arranged in the executable do not matter. The signing process sorts the sections (.text, .rdata, etc) so that the order of the sections do not matter.

    Don’t execute input

    Another part to make clear is the signing process shouldn’t use the signing input as executable code. That would leave the file being signed in control of how it is being signed, and could likely convince a SIP to do something unexpected.

    Include as much as possible

    My recommendation here is to sign everything except the signature itself. The PE format makes an exception for the checksum, which cannot be included, but otherwise, that’s it. The PE format used to support omitting the signing of embedded debug information, but that is no longer that case.

    You might be tempted to skip over benign contents of the file. For example, with PNG we might be tempted to skip embedded metadata that aren’t part of the actual image itself. I would argue that all data is relevant, and omitting metadata attached to the image is harmful to the signing process.

    Be wary of the file size

    Canonicalization and being smart about the file format might make reading the whole file in to memory tempting. However, keep in mind what some file sizes might be. You may want to open a memory-mapped view of the file if you cannot read it in a stream.

    Put Indirect Data

    We can calculate a digest of the file now, but now we need to be help Authenticode understand how to embed the signature in the file.

    The final thing we need to actually sign a file is to implement CryptSIPPutSignedDataMsg. This function is pretty straight-forward. Given a SIP_SUBJECTINFO with a handle to the file being signed, embed the pbSignedDataMsg parameter whose length is cbSignedDataMsg.

    As mentioned earlier, we cannot just write this data at the end of the file, otherwise it will corrupt the file. We need to embed it in such a way that the file will continue to function.

    For PNG, this is actually a straight forward process. It is composed of chunks, and each chunk has a 4 character identifier. The case of the identifiers letters indicate a few certain flags. This lead me to create a new chunk type called dsIG, which indicates that the chunk is not critical and that it shouldn’t be copied. When we digest the PNG file above when we’re creating the indirect data, we skip over dsIG chunks.

    The exact details of this are in the GitHub project. I may write another post later about the specifics of PNG files if there is enough interest. However, I would take a look at the specification first if you’re interested. The format is very easy to understand.

    At this point, we can sign a file, and signtool should report successfully signing the file type. The verify step will fail though because we have not implemented CryptSIPGetSignedDataMsg and CryptSIPVerifyIndirectData.

    I’ll save those two for Part 2, though they are available on GitHub right now.

    Update: Part 2 is available.