Open source cryptography takes a step forward with the release of OpenPGPjs 4.0

protonmail-openpgpjs-v4-0-release-notes

We have just released a major new version of OpenPGPjs, the world’s most popular open-source JavaScript encryption library.

The goal of our OpenPGPjs project is to make public-key cryptography not only available to users, but also to the global developer community. We believe the widespread availability of open-source and secure cryptography libraries is a prerequisite of the privacy revolution.

OpenPGPjs version 4.0 introduces streaming cryptography. This makes it possible for users of the library to encrypt, decrypt, sign, and verify files while they are simultaneously being downloaded or uploaded, meaning that large files no longer need to be stored in memory in order to perform these operations. The associated performance and storage benefits will allow us to speed up the encryption and decryption of email attachments and help us develop new products like ProtonDrive.

ProtonMail is committed to open source, and all the code for OpenPGPjs can be found on Github. You can also download the source code as a .zip file or as a tar.gz file.

The support of the developer community is essential for the continued development of OpenPGPjs, and we welcome pull requests and comments. Here’s what’s new with OpenPGPjs 4.0:

Streaming Encryption

There are two different types of streaming implemented in this release.

The first can only be used in implementations that support the latest draft of authenticated encryption with associated data (AEAD), which was first added in OpenPGPjs v3.0.9. AEAD is particularly well-suited to streaming, because it allows messages to be divided into chunks, each of which has its own integrity tag that can be used to authenticate it as it is downloaded. In contrast to non-AEAD messages, this allows the client to trust each chunk as it is received, rather than waiting until the end of the data packet to check the integrity tag.

For compatibility with older messages and with other OpenPGP clients that do not support this draft, we have also implemented streaming for non-AEAD OpenPGP messages that use the unauthenticated cipher feedback mode (CFB) encryption rather than the authenticated modes supported in the AEAD implementation (EAX, OCB, and GCM).

Configuration Details

Streaming AEAD

The configuration setting openpgp.config.aead_chunk_size_byte controls the one-octet chunk size defined in the AEAD data packet, and defaults to 12. The size of each message chunk will be 2 ^ (openpgp.config.aead_chunk_size_byte + 6) bytes, and thus defaults to 256KB. This can be adjusted to receive data on the stream more or less frequently.

Streaming CFB

Because CFB-encrypted OpenPGP messages only have a single integrity tag, at the end of the message, it is not possible to authenticate data during stream decryption until the end of the message. Therefore, due to the security issues associated with using unauthenticated data, we do not output any data to the stream by default during decryption. To override this default and allow unauthenticated data to be streamed, set openpgp.config.allow_unauthenticated_stream to true.

Web Streams Implementation 

Browser support and Polyfills

This feature relies on the Web Streams API, which Chrome, Safari, Firefox, and Edge currently have partially implemented, with Firefox’s implementation behind feature flags. Chrome is the only browser that implements TransformStreams, which are required in our implementation, so we include a polyfill for all other browsers. Please note that in those browsers, the global ReadableStream property gets overwritten with the polyfill version if it exists. Thus, if you need to use the native ReadableStream, you may need to store a reference to it before loading OpenPGPjs, or use the web-streams-adapter library to convert back and forth between them.In order to avoid separate implementations for web streams, node streams, and non-streamed data, all data is temporarily converted to a web stream internally and then converted back when returning. However, in the future, we might look into re-implementing some of the convenience functions of the web-stream-tools (see below) to not need streams, so that you don’t need to include the streams polyfill when not using streams.

New Web-Stream-Tools library

The OpenPGP spec requires us to manipulate and transform streams in complex ways: a stream of armored encrypted data needs to be parsed, then base64-decoded, then decrypted, then perhaps decompressed, then perhaps UTF8-decoded. Internally, we’re chaining together TransformStreams to achieve this. However, the TransformStream API is not quite sufficient for us: there is no way to control the amount of data that comes in at once, for example. And even if there was, we don’t always know in advance how many bytes we need: often, that depends on a field earlier in the data. To avoid having complex buffering code at every step of the way, we created a library to make this and other aspects of reading and transforming streams easier: web-stream-tools. Contributions to make handling streams even easier are welcome!

High-Level API Changes

  • openpgp.message.fromText(), fromBinary(), readArmored() and
    read() now accept ReadableStreams as well as Node streams.
  • The high-level encrypt, decrypt, sign and verify functions now
    have an streaming parameter, to control whether the return value
    contains a stream. It can take the values “web”, “node”, or false.
    It defaults to the type of stream you passed in, if any.
  • When streaming, the signatures returned by verify and decrypt have
    a verified: Promise<Boolean> property instead of valid: Boolean. The
    signature property is also a Promise in that case.
  • openpgp.{message,key,signature,cleartext}.readArmored() and
    openpgp.{message,key,signature}.read() are now asynchronous.

For example, instead of writing let publicKey = openpgp.key.readArmored(publicKeyArmored).keys[0]; write, in an async function: let publicKey = (await openpgp.key.readArmored(publicKeyArmored)).keys[0];

  • openpgp.encrypt() and openpgp.sign() now take a message
    parameter instead of data, dataType and filename. Use
    openpgp.message.fromText(), openpgp.message.fromBinary() or
    openpgp.cleartext.fromText() to create a message. Note that if you
    previously used the date parameter, you should now additionally pass
    it to fromText/fromBinary as well.
  • After calling let keyring = new openpgp.Keyring(), you now have to
    call await keyring.load() to read the keys from LocalStorage.

Separate bundle for old browsers

OpenPGPjs has been increasing in size due in part to the many new features (ECC in 3.0, Streaming in 4.0), but also because we supported a wide range of browsers. That meant we had to transpile ES6 to ES5 and include quite a lot of polyfills, both of which increase the library size.

We’ve now eliminated some polyfills from the default openpgp.min.js bundle, so it will now work with only recent versions of Chrome, Firefox, Safari, and Edge. If you need support for Internet Explorer 11 and old versions of Safari, you can use the new compat/openpgp.min.js bundle.

You could even load one or the other depending on which browser the user is using. However, if you’re using the web worker, keep in mind that you also need to pass { path: ‘compat/openpgp.worker.min.js’ } to initWorker whenever you load compat/openpgp.min.js.

Development 

  • You can now do grunt browsertest –dev to debug using a build with a source map and original module names intact. You can also do grunt browsertest –compat to test a build that’s compatible with IE11 and older versions of Safari. Both the –dev and –compat parameters also work for grunt build and other tasks that depend on it.
  • There’s a new npm run build command to build both compat and non-compat bundles.
  • The browserify bundles are now cached and built incrementally, speeding up development after the first build.

Other Updates

  • Armor and packet parsing are now stricter: previously, missing —–END PGP PUBLIC KEY BLOCK—– or truncated packets wouldn’t throw errors, but do now.
  • SHA1 and SHA256 hashing now use asmcrypto, making them faster.
  • Encryption and decryption of text now properly supports Unicode surrogate code points in JavaScript strings.

Future Roadmap 

  • Improve the performance of public-key operations by using asm.js big numbers.
  • Re-implement ECC cryptography using asm.js to improve performance and make the primitives constant-time.
  • Continue to find ways to make openpgp.js smaller, for example by converting asm.js to WebAssembly

About the Author

Admin

We are scientists, engineers, and developers drawn together by a shared vision of protecting civil liberties online. Ensuring online privacy and security are core values for the ProtonMail team, and we strive daily to protect your rights online.

 

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

3 comments on “Open source cryptography takes a step forward with the release of OpenPGPjs 4.0

  • I believe the Snowden leaks had U.S. surveillance privately complaining that the open source community not only created difficult software to crack, but also freely disseminated that software – so this OpenPGPjs news is a good thing.

    For once, governments are not in control. Someday they might actually have to show citizens they can make life better for them if they want the top positions in society. Right now, they can get us fighting each other, and killing each other, and control the message so that this is thought of as a good thing.

    Governments will soon have to provide actual value. If they don’t, we’ll freely discuss it with each other on the internet, and elsewhere, and remove them.

    Reply
  • Terry- that’s putting a lot of faith in the general public to assume: 1) they’re intelligent enough to see and understand what is truly happening in that regard 2) they are not too far gone, already caught in the snares of diversion (social media/fixation on mindless entertainment, self-absorbency/narcissism), consumerism/materialism, the endless subtle propaganda and systemic control/suppression, etc. 3) they do not feel powerless to change the way it all has become– and therefore submit to a sense of “learned helplessness” or apathy as a defense.

    You are right, that is how it has always been… the masses are mistreated and abused, played like puppets by those in power– and then… those masses reach a breaking point; there is an uprising and subsequent coup. A paradigm shift and a social reset results. The people take their lives and their freedom back… and they move on. Yes, it certainly reeks of Marxism.

    Each time it’s a little different. The culture, the terms and the circumstances have varied throughout history. Yes, they are fundamentally the same play, we can’t ignore that. Technology seems to be the X-factor to me. It develops, further and further and further, forever introducing new entropy into the equation.

    Technology, in most cases, is most accessible to the wealthy and powerful (typically mutually inclusive traits), with some trickle-down effect as new replaces the old. Think about medicine… cell phones, the internet. It’s the cycle we know and love: the proletariat get the table scraps of the bourgeoisie.

    The average person, I would say, is becoming dumber with each new generation– especially in the United States. Not by nature, but by nurture.

    It seems an uphill battle.

    So I have to wonder– is it enough, today, for the few enlightened and motivated to make a difference? I appreciate and respect those that are trying to bring about the means to revolution– but I wonder: is it ultimately in vain? What can be said to those who can’t hear? All the counter-technology in the world doesn’t matter if society isn’t interested in learning about it or don’t feel driven to use it.

    These kinds of technologies– and even the very ideas that should drive people to use them– still feel very esoteric to me. Not in a personal sense, but in a general one. What I mean is that the average person doesn’t see a real need or use for them. They’re often considered superfluous– even outright excessive or paranoid to some. Ostensibly, they’re tin foil hats. Inconvenient niceties.

    Then again, they used to say that about condoms/rubbers too, didn’t they? Haha…

    It’s all a product of ignorance; and an intended one, at that. Governments love to keep their masses “barefoot and pregnant” because it’s so much easier for them that way. Whether through fear, poverty, ignorance, oppression or outright slavery– control is control. The means are inconsequential.

    I look around and it’s “bread and circuses” every day, as the rats are being led through the streets…

    So with all we see going on at present, why are most still just not getting it? Is it nativity? Maybe denial? Are we going to just wait past the point of inevitability? When will it be too late?

    With this in mind (since that’s the direction we seem to be headed) I have to myself: is “too late” a real idea, or will there always be a sleeping guard somewhere?

    All I know is that Orwell was right all along. He was a true visionary; a prophet. We’re beginning to see that exact dystopian narrative play out right before our very eyes.

    People– it’s 1984 and we’ve seen the blueprints for the machine whose gears will surely crush us.

    So what do we do? What can we do?

    Maybe we should all wake up and chose our fate rather than accept the one that is otherwise coming to us. Projects/movements like this are a huge step in the right direction.

    Trust me, the rest of the world will thank you later.

    Reply