Mastodon

Using ghcr: a credential managment side quest

GitHub has included it’s own OCI image registry for a long time but it’s only recently that I ended up needing it.

While authenticating docker printed out a warning…

WARNING! Your credentials are stored unencrypted in '/home/mike/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/

Things like this bug me, so I wanted to loop back and figure this out.

Getting a token

Using GitHub’s container registry requires an auth token, so we can start our adventure by going to github.com/settings/token and clicking the “Generate New Token” button.

Since GitHub made the somewhat strange decision to associate “packages” with your account (by default) rather than a repo, just choosing a classic (non-repo scoped) token seems like a good fit.

Securing your token

Getting the token solves the authentication problem, but instantly creates a credential management problem.

To solve that problem Docker offers the docker-credential-helper. It has a few flavours, that each delegate management of the credentials to some other service.

Secure against what?

As a developer, my threat model somewhat unavoidably includes running untrusted code, usually in the form of npm postinstall hooks; Shai Halud 2.0 was a recent example of how postinstall scripts are used to harvest credentials.

With this in mind I want my credentials encrypted, and I want to to be prompted when they are getting decrypted.

The pass command line tool does both of those things, so I’ll start by installing that.

Installing pass

On Manjaro or Archlinux you can install pass with pacman:

sudo pacman -S pass

Pass uses gpg but it’s a long time since I messed with that, so I’ll generate a new key with gpg --full-generate-key . Take the email you used to initialize your key (let’s assume it’s me@example.com) and run pass init:

$ pass init me@example.com
mkdir: created directory '/home/mike/.password-store/'
Password store initialized for me@example.com

Now that we have somewhere to store our passwords, let’s get the credential helper installed. On Manjaro or Archlinux you can install that helper from the AUR with Yay:

$ yay -S docker-credential-pass

After that we need to change our docker config to tell docker to delegate credential handling to pass. Then we’ll restart the docker service for good measure.

$ jq '. += {"credsStore": "pass"}' ~/.docker/config.json | sponge ~/.docker/config.json
$ sudo systemctl restart docker.service

Logging into GHCR

If I use pass ls now, I can see there is nothing in my credential store. Lets login to ghcr.io and see if docker stores the password in pass.

$ # use read to define a variable by prompting you for your token
$ read -rs SECRET
$ echo $SECRET | docker login --username myuser ghcr.io --password-stdin
Login Succeeded
$ pass ls
Password Store
└── docker-credential-helpers
    └── Z2hjci5pbw==
        └── myuser

Importantly, cat ~/.password-store/docker-credential-helpers/Z2hjci5pbw\=\=/myuser.gpg shows they are encrypted, so exfiltrating them is useless. And if a malicious script attempts to decrypt them by running echo "ghcr.io" | docker-credential-pass get or directly using pass with pass docker-credential-helpers/Z2hjci5pbw==/myuser I’ll be prompted, so my credentials can’t be silently stolen anymore.

Now I’m ready to docker build -t ghcr.io/username/project/image:latest . and push/pull as normal.

The road not taken: secret service

One other option I explored and eventually rejected was docker-credential-secretservice, which delegates to the linux desktop secret service.

On Manjaro or Archlinux you can install that helper from the AUR with Yay:

$ yay -S docker-credential-secretservice

You’ll need to update your ~/.docker/config.json to specify that you want to use the secret service (docs).

$ jq '. += {"credsStore": "secretservice"}' ~/.docker/config.json | sponge ~/.docker/config.json

Afterwards, simply use docker login and it’ll prompt you for your password, and store it in your operating systems secret store.

$ docker login --username myuser ghcr.io
Password:
Login Succeeded

Whats the problem with that?

I had high hopes for this initially but after some investigation I realized it’s not a good fit for the developer threat model.

Using the secret service means the password is stored with our other credentials in secret service and accessible via the Passwords and Keys app (the GUI frontend for the Gnome Keyring service).

While we’ve avoided leaving unencrypted credentials lying around on disk, we’ve ended up running into the limitations of the Gnome Keyring security model; The user session is the trust boundary, and (all!) programs running within that boundary can access the credentials once the keyring is unlocked.

For a normal desktop user that might be a plausible security assumption, but as a developer, running untrusted code inside that session trust boundary is uncomfortably common.

Since docker-credential-secretservice stores it’s credentials on the default keyring and that keyring is unlocked on login and stays unlocked for the duration of your session, this is pretty underwhelming security-wise.

This means that any malicious npm postinstall script can list my credentials and silently access my token(s) without me ever knowing.

$ docker-credential-secretservice list
{"Registry credentials for ghcr.io":"myuser"}
$ echo "ghcr.io" | docker-credential-secretservice get
{"ServerURL":"ghcr.io","Username":"myuser","Secret":"ghp_abc123YourActualPlaintextTokenHere"}

Safer development practices

Doing development inherently involves running some amount of untrusted code. While pass looks pretty great for mitigating some of the consequences, it’d be nice to address the root problem too.

For those looking to push secure development further putting pass together with something like Deno which doesn’t execute postinstall hooks by default, makes things safer still.

Modernise security to modernise government

With the resignation of the CIO of the Government of Canada, the person placed at the top of the Canadian public service to fix the existing approach to IT, there is lots of discussion about what’s broken and how to fix it.

Across these discussions, one thing stands out to me: IT security always seems to get a pass in discussions of fixing/modernising IT.

This post is an attempt to fix that.

As the article about the CIO points out, “All policies and programs today depend on technology”. IT Security’s Security Assessment and Authorization (SA&A) process applies to all IT systems therefore landing on the critical path of “all policies and programs”. This one process adds a 6-24 month delay to every initiative and somehow escapes any notice or criticism at all.

If you imagine some policy research, maybe a public consultation and then implementation work, plus 6-24 months caught in the SA&A process, it should be clear that a single term in office may not be enough be able to craft and launch certain initiatives let alone see benefits from them while in office. Hopefully all political parties can agree fixing this is in their best interests.

As a pure audit process, the SA&A is divorced from the technical work of securing systems (strangely done by operations groups or developers, rather than by security groups) leaving lots of room to reshape (or eliminate) this process without threatening actual security work. Improvements in this process are probably the single most impactful change that can be made in government.

It’s also key to accelerating all other modernisation initiatives.

Everyone in Ottawa is well aware that within each department lies one or more likely political-career-ending ticking legacy IT timebombs. Whether this is the the failure of the system itself, or of the initiative launched to fix it, or even just the political fallout from fixed capacity systems failing to handle a surge in demand, every department has these and the only question is who will be in office when it happens.

Though you’d never guess, inside the government it is actually known how to build systems that can be modernised incrementally, changed quickly, rarely have user visible downtime and can expand to handle the waves of traffic without falling over.

The architecture that allows this (known as microservices) was made mandatory by TBS in the 2017 Directive on Service and Digital. The Directives successor (the Enterprise Architecture Framework) doesn’t use the term directly but requires that developers “design systems as highly modular and loosely coupled services” and several other hallmarks of microservices architecture that allow for building resilient digital services.

I think TBS was correct in it’s assessment that this architecture is key to many modernisation initiatives and avoiding legacy system replacements just as inflexible as their predecessors. Treasury Board themselves describes the difference between current practice as their target architecture as a “major shift” but the number of departments willing/able to make that shift hovers close to zero.

AWS uses the same microservices architecture to deliver their digital services and promotes it, along with the infrastructure and team structures needed to support it, under the banner “Modern Applications“. Substantially similar advice is given by Google and others and these best practices have been worked into TBS’s policy since 2017.

While much of TBS IT policy has been refreshed, all core IT security guidance and patterns are built around pre-cloud ideas (ITSG 22 from 2007, ITSG 38 from 2009) and process (ITSG 33 from 2012).

While TBS might want departments to adopt microservices (created circa 2010 around the “death” of SOA), it’s the 1990s-era 3-tier architecture (what ITSG-38 still calls the “preferred architecture”) that the network and security infrastructure is set up to support, rather than the fancy compute clusters and cloud functions needed for the microservices architecture; an application architecture that exists to fix the very visible problems governments (and others) have with availability and scalability.

Unfortunately, these legacy design patterns mean that security teams spend months or years on non-nonsensical routing and subnets and placing fixed-capacity Virtual Machine based security appliances between the internet and services running in the cloud before a cloud account can be used. Beyond the staggering delay, the result is an anti-pattern where the network layer cancels out the scalability and availability benefits of the application architecture above and the cloud architecture below.

Fixed capacity firewalls can’t handle waves of traffic like AWS Lambda or Google Cloud Run, and high-availability microservices architecture doesn’t matter when security brings down the firewall in front to patch it.

Similar conflicts exist with TBS’s requirements for the Dev(Sec)Ops and “multidisciplinary teams” practices needed for this architecture. These practices are likely not feasible given almost every action currently requires manual security approvals, or the current interpretations and enforcement of ITSG-33’s AC-5 “separation of duties” and the naive insistence that distant monitoring/SoC teams operating with zero context can somehow parse meaning out of the complex communication patterns of microservices applications.

While TBS has updated it’s policy to require agile development practices, ITSG-33, the foundation of all government security process is explicitly waterfall, and while adapting it to agile is theoretically possible, it’s developers that get exposure to agile methods, rather than the well intentioned auditors and former network admins that populate most security groups.

Beyond the shift to agile, the division of labour (the split between dev/sec/ops) systemically undermines ITSG-33’s approach: the technical work of securing systems builds security engineering expertise in operations groups, while the spreadsheet-based audit work of security teams chases away technical talent and steadily erodes any remaining technical capacity.

This dynamic reliably creates a skills imbalance that undermines the audit/watchdog role ITSG-33 imagines for security teams; The 2019 State of DevOps report noted a similar effect with CoEs: “This disconnect between theory and hands-on practice will eventually threaten their expertise”. Luck and smart hiring can help, but security teams are forever swimming against this current. When you mix security and compliance, what survives is compliance.

The failure case is the familiar security-theater of the audit process: non-technical auditors “verifying” the work of developers and operations via screenshots without being able to read or run code and executives placing their faith in this paperwork rather than the judgement of the dev/ops teams doing the engineering work of securing systems.

Typically contractor driven, each one costs over $70,000 and at least 6 months for a paper audit of a simplistic one-piece monolithic application. Nobody is sure how to apply this process to the microservices architecture TBS is pushing where each application is made of dozens, or even thousands of separate applications.

This process is almost single-handedly responsible for the lack of progress on modernisation: Since all departments have hundreds of applications, assuming an optimistic audit timeline of 6 months each, even the most milquetoast modernization effort implies a SA&A/audit process bottleneck measured in decades.

Surrounded by hollowed-out waterfall security processes and pre-cloud security architecture that no-one seems equipped to change, Treasury Board’s vision of the government delivering modern, reliable digital services flounders.

The idea here is that in many cases modernising security is a precondition to successfully modernising anything else. For those that overlook security, the assumptions embedded in their tools, processes and architectures will subtly but steadily undermine their efforts. Treasury Board is filled with smart policy analysts learning this the hard way.

Security is the base of the modernisation pyramid… start there to fix things.

GraphQL and security

Imagine you have a web application that allows people view widgets by name. Somewhere deep in your codebase, a programmer has thoughtfully updated the existing ES5 SQL injection to this stylish new ES6 SQL injection:

`select * from widgets where name = '${name}';`

Injection attacks just like this one persist in-spite of the fact that a search for “sql injection tutorial” returns around 3,740,000 results on Google. They have made the top of the OWASP top 10 in 2010 and 2013 and probably will again in 2016 and likely for the foreseeable future. Motherboard even calls it “the hack that will never go away“.

The standard answer to this sort of problem is input sanitization. It’s standard enough that it shows up in jokes.

exploits_of_a_mom
xkcd’s famous “Exploits of a Mom

But when faced with such consistent failure, it’s reasonable to ask if there isn’t something systemic going on.

There is a sub-field of security research known as Language Theoretic Security (Langsec) that is asking exactly that question.

Exploitation is unexpected computation caused reliably or probabilistically by some crafted inputs. — Meredith Patterson

Dropping the students table in the comic is exactly the kind of “unexpected computation” they are talking about. To combat it, Langsec encourages programmers to consider their inputs as a language and the sum of the adhoc checks on those inputs as a parser for that language.

They advocate a clean separation of concerns between the recognition and processing of inputs, and bringing a recognizer of appropriate strength to bear on the input you are parsing (ie: not validating HTML/XML with a regex)

Where this starts intersecting with GraphQL is in what this actually implies:

This design and programming paradigm begins with a description of valid inputs to a program as a formal language (such as a grammar or a DFA). The purpose of such a disciplined specification is to cleanly separate the input-handling code and processing code.

A LangSec-compliant design properly transforms input-handling code into a recognizer for the input language; this recognizer rejects non-conforming inputs and transforms conforming inputs to structured data (such as an object or a tree structure, ready for type or value-based pattern matching).

The processing code can then access the structured data (but not the raw inputs or parsers’ temporary data artifacts) under a set of assumptions regarding the accepted inputs that are enforced by the recognizer.

If all that starts sounding kind of familiar, well it did to me too.

GraphQL

GraphQL allows you to define a formal language via it’s type system. As queries arrive, they are lexed, parsed, matched against the user defined types and formed into an Abstract Syntax Tree (AST). The contents of that AST are then made available to processing code via resolve functions.

The promise of Langsec is “software free from broad and currently dominant classes of bugs and vulnerabilities related to incorrect parsing and interpretation of messages between software components”, and GraphQL seems poised to put this within reach of every developer.

Turning back to the example we started with, how could we use GraphQL to protect against that SQL injection? Lets get this going in a test.

require("babel-polyfill")

import expect from 'expect'
import {
  graphql,
  GraphQLSchema,
  GraphQLScalarType,
  GraphQLObjectType,
  GraphQLString,
  GraphQLNonNull
} from 'graphql'
import { GraphQLError } from 'graphql/error';
import { Kind } from 'graphql/language';

describe('SQL injection', () => {

  it('returns a SQL injected string', async () => {

    let schema = new GraphQLSchema({
      query: new GraphQLObjectType({
        name: 'Query',
        fields: () => ({
          widgets: {
            type: GraphQLString,
            args: {
              name: {
                description: 'The name of the widget',
                type: new GraphQLNonNull(GraphQLString)
              }
            },
            resolve: (source, {name}) => `select * from widgets where name = '${name}';`
          }
        })
      })
    })

    let query = `
      query widgetByName($name: String!) {
        widgets(name: $name)
      }
    `

    //args: schema, query, rootValue, contextValue, variables
    let result = await graphql(schema, query, null, null, {name: "foo'; drop table widgets; --"})
    expect(result.data.widgets).toEqual("select * from widgets where name = 'foo'; drop table widgets; --'")
  })

})

GraphQL brings lots of benefits (no need for API versioning, all data in a single round trip, etc…), and while those are compelling, simply passing strings into our backend systems misses an opportunity to do something different.

Let’s create a custom type that is more specific than just a “string”; an AlphabeticString.

  it('is fixed with custom types', async () => {

    let AlphabeticString = new GraphQLScalarType({
      name: 'AlphabeticString',
      description: 'represents a string with no special characters.',
      serialize: String,
      parseValue: (value) => {
        if(value.match(/^([A-Za-z]|\s)+$/)) {
          return value
        }
        return null
      },
      parseLiteral: (ast) => {
        if(value.match(/^([A-Za-z]|\s)+$/)) {
          return ast.value
        }
        return null
      }
    });

    let schema = new GraphQLSchema({
      query: new GraphQLObjectType({
        name: 'Query',
        fields: () => ({
          widgets: {
            type: GraphQLString,
            args: {
              name: {
                description: 'The name of the widget',
                type: new GraphQLNonNull(AlphabeticString)
              }
            },
            resolve: (source, {name}) => {
              return `select * from widgets where name = '${name}';`
            }
          }
        })
      })
    })

    let query = `
    query widgetByName($name: AlphabeticString!) {
        widgets(name: $name)
      }
    `

    //args: schema, query, rootValue, contextValue, variables
    let result = await graphql(schema, query, null, null, {name: "foo'; drop table widgets; --"})
    expect(result.errors).toExist()
    expect(result.errors[0].message).toInclude("got invalid value")
  })

This test now passes; the SQL string is rejected during parsing before ever reaching the resolve function.

On making custom types

There are already libraries out there that provide custom types for things like URLs, datetime’s and other things, but you will definitely want to be able to make your own.

To start defining your own types you will need to understand the role
the various functions play:

    let MyType = new GraphQLScalarType({
      name: 'MyType',
      description: 'this will show up in the documentation!',
      serialize: (value) => { //... }
      parseValue: (value) => { //... },
      parseLiteral: (ast) => { //... }
    });

The serialize function is the easiest to understand: GraphQL responses are serialized to JSON, and if this type needs special treatment before being included in a response, this is the place to do it.

Understanding parseValue and parseLiteral requires a quick look at two different queries:

//name is a string literal, so parseLiteral is called 
query {
  widgets(name: "Soft squishy widget")
}

//name is a value so parseValue is called
query widgetByName($name: AlphabeticString!) {
  widgets(name: $name)
}

In the first query, name is a string literal, so your types parseLiteral function will be called. The second query, the name is supplied as the value of a variable so parseValue is called.

Your type could end up being used in either of those scenarios so it’s important to do your validation in both of those functions.

The standard GraphQL types (GraphQLInt, GraphQLFloat, etc.) also implement those same functions.

Putting the whole thing together, the process then looks something like this:

A query arrives, gets tokenized, the parser builds the AST, calling parseValue and parseLiteral as needed. When the AST is complete, it recurses down the AST calling resolve, using parseValue and parseLiteral again on whatever is returned before calling serialize on each to create the response.

Where to go from here

Langsec is a deep topic, with implications wider than just what is discussed here. While GraphQL is certainly not “Langsec in a box”, it not only seems to be making the design patterns that follow from Langsec’s insights a reality, it has a has a shot at making them mainstream. I’d love to see the Langsec lens turned on GraphQL and see how it can guide the evolution of the spec and the practices around it.

I would encourage you to dig into the ideas of Langsec, and the best place to start is here:

That stonewalling thing

There is a meme in the current crypto “debate” that makes me cringe whenever I read it: the idea of “stonewalling”. It’s come up in the Apple vs FBI case as the ForbesLA Times, Jacobin magazine and others all described Apple as “stonewalling” the FBI.

Wired’s recent Whatsapp story mentioned that “WhatsApp is, in practice, stonewalling the federal government” and while Foreign Policy magazine avoided the word, they captured the essence when they described Whatsapp as a “a service willing to adopt technological solutions to prevent compliance with many types of court orders”.

All of these articles make it sound like Apple/Whatsapp has the data, but it unwilling to give it to the government.

�
  !��s�����|Ǧ�2}|q�h�J�,�^��=&/
                                    _,e�r%����/D@�1f��"�
                                                                ]�?c�,��y�l?��3�lF�'���ǘ��IA��O�Y�i�����ё�R��`�[�]�H���P�1'��������S����~tF\�������^��f@��<P�g�	!X���6eh�U�rN���=d@܉eQe���B�lk����\ҠcE��
�$�d&���_xor�s�-���l,v���44�E����n�[���1YL�o�ޜ�g�m�����Tx�f	܁�����å+e�LR�E1���ޅx
                                                                                              �a*�Զ\l�ϫ)4&���or�-�4���C���q��|-2[͘7 ��
��0�ǹ����+�5b!�wV����������3\n�꨻�R�,Ĝ�

\F����P�IJ<Ը$�`Q/���D�w��̣���v"|��z�g/I��@!�(�z������]ɹ3}+f1�
                                                                  ju��vw�y~#7�w��K������M\g�.uW�i
                                                                                                    TYc���I@�s�;�/��
                                                                                                                        �����s�c�ݮ���C�
                                                                                                                                         �6~�e

Blobs of encrypted text like the one above are useless for anyone put the holder of the decryption key. Where the company holds the decryption key and refused to give it up, it seems reasonable to call that “stonewalling”.

Without the decryption key, you may be in possession of such a blob but you can’t meaningfully be described as “having” the data within it. Calls of “stonewalling” in cases like that are either grandstanding or reveal an opinion-disqualifying level of ignorance.

These accusations of stonewalling obscure what I think is the real appeal of encryption and tools such as Tor: it’s not that these technologies prevent compliance, it’s that companies can prevent the collection certain types of data in the first place.

The authors of a recent paper called “Cryptopolitik and the Darknet” did exactly that when they crawled the darknet for data:

“In order to avoid illegal material, such as media files of child pornography or publications by terrorist organisations, only textual content was harvested automatically. Any other material was either filtered out or immediately discarded.”

Nobody would think to accuse them of stonewalling or adopting “technological solutions to prevent compliance” for finding a way to do their darknet crawl without accumulating a bunch of data that is going to bring with it complicated law enforcement dealings.

When Whatapp wants to “avoid illegal material” while still remaining in the messaging business, they do it with end-to-end encryption.

 

Why end-to-end? In end-to-end encryption, the end users hold the decryption keys. Companies who decide to keep the keys themselves become a target of every spy agency on the planet and run the risk of becoming the next Gemalto.

That technologies and architectural choices exist which allow you to filter the data you are exposed to, and therefore your level of legal liability/obligation feels new. Or maybe what’s new is companies willingness to actually implement them.

No-one is interested in obstructing investigations, just managing risk and legal exposure. “If you collect it, they will come” is becoming a common phrase among programmers and security people, and for companies who don’t want to end up holding data related to a crime, painting a giant target on themselves, dedicating resources to servicing government requests, or having awkward public relations moments, end-to-end encryption starts to look like good risk management. Doubly so when you are dealing with multiple governments.

In that context, governments pushing back against end-to-end seem to indicate an existing idea that companies are somehow obligated to collect data on behalf of the government and that using encryption to limit your collection is not OK. This is visible in the issue of the government conscripting companies to do it’s work raised by the FBI’s recent use of the 1789 All-Writs Act law to try to force Apple to build software to hack it’s own phone.

With many years of enthusiastic support from companies like AT&T it’s easy to see where that idea might have come from. As the American government oscillates between attacking tech companies and asking them to do it’s job, and authoritarian governments and international customers look on, it’s not hard to see why many tech companies are far less enthusiastic about facilitating government aims. So far “stonewalling” seems to be a deliberately provocative framing for the “we’ve stopped collecting that data, leave us out of this” reality that end-to-end encryption creates.

Seeing that kind of inflammatory rhetoric from the FBI or congress is one thing, but it’s widespread use by journalists is very disconcerting.

As cries of “stonewalling” turn to accusations of tech companies thinking they are “above the law” and now draft anti-encryption legislation, it’s probably good to remember that blob of encrypted text. It’s not that these companies are getting in the way of the FBI getting data, they are trying to get themselves out of the way by removing their own access to it.

Of all people, former NSA director Michael Hayden recently observed “America is simply more secure with unbreakable end-to-end encryption”. I never thought I would be hoping more people would listen to him.

gpg and signing your own Arch Linux packages

One of the first things that I wanted to install on my system after switching to Arch was ArangoDB. Sadly it wasn’t in the official repos. I’ve had mixed success installing things from the AUR and the Arango packages there didn’t improve my ratio.

Using those packages as a starting point, I did a little tinkering and got it all working the way I like. I’ve been following all the work being done on reproducible builds, and why that is needed and it seems that with all that going on, anyone dabbling with making packages should at the very least be signing them. With that as my baseline, I figured I might as well start with mine .

Of course package signing involves learning about gpg/pgp whose “ease of use” is legendary.

Before we get to package signing, a little about gpg.

gpg --list-keys
gpg -k
gpg --list-public-keys

All of these commands list the contents of ~/.gnupg/pubring.gpg.

gpg --list-secret-keys
gpg -K

Both list all keys from ~/.gnupg/secring.gpg.

The pacman package manager also has its own gpg databases which you can explore with:

gpg --homedir /etc/pacman.d/gnupg --list-keys

So the task at hand is getting my public key into the list of public keys that pacman trusts. To do that we will need to do more than just list keys we need to reference them individually. gpg has a few ways to do that by passing an argument to one of our list keys commands above. I’ll do a quick search through the list of keys that pacman trusts:

mike@longshot:~/projects/arangodb_pkg☺ gpg --homedir /etc/pacman.d/gnupg -k pierre
pub   rsa3072/6AC6A4C2 2011-11-18 [SC]
uid         [  full  ] Pierre Schmitz (Arch Linux Master Key) &lt;pierre@master-key.archlinux.org&gt;
sub   rsa1024/86872C2F 2011-11-18 [E]
sub   rsa3072/1B516B59 2011-11-18 [A]

pub   rsa2048/9741E8AC 2011-04-10 [SC]
uid         [  full  ] Pierre Schmitz &lt;pierre@archlinux.de&gt;
sub   rsa2048/54211796 2011-04-10 [E]

mike@longshot:~/projects/arangodb_pkg☺  gpg --homedir /etc/pacman.d/gnupg --list-keys stephane
pub   rsa2048/AB441196 2011-10-30 [SC]
uid         [ unknown] Stéphane Gaudreault &lt;stephane@archlinux.org&gt;
sub   rsa2048/FDC576A9 2011-10-30 [E]

If you look at the output there you can see what is called an openpgp short key id. We can use those to refer to individual keys but we can also use long id’s and fingerprints:

gpg --homedir /etc/pacman.d/gnupg -k --keyid-format long stephane
pub   rsa2048/EA6836E1AB441196 2011-10-30 [SC]
uid                 [ unknown] Stéphane Gaudreault &lt;stephane@archlinux.org&gt;
sub   rsa2048/4ABE673EFDC576A9 2011-10-30 [E]


gpg --homedir /etc/pacman.d/gnupg -k --fingerprint stephane
pub   rsa2048/AB441196 2011-10-30 [SC]
      Key fingerprint = 0B20 CA19 31F5 DA3A 70D0  F8D2 EA68 36E1 AB44 1196
uid         [ unknown] Stéphane Gaudreault &lt;stephane@archlinux.org&gt;
sub   rsa2048/FDC576A9 2011-10-30 [E]

So we can identify Stephane’s specific key using either the short id, the long id or the fingerprint:

gpg --homedir /etc/pacman.d/gnupg -k AB441196
gpg --homedir /etc/pacman.d/gnupg -k EA6836E1AB441196
gpg --homedir /etc/pacman.d/gnupg -k 0B20CA1931F5DA3A70D0F8D2EA6836E1AB441196

Armed with a way to identify the key I want pacman to trust I need to do the transfer. Though not initially obvious, gpg can push and pull keys from designated key servers. The file at ~/.gnupg/gpg.conf tells me that my keyserver is keys.gnupg.net, while pacman’s file at /etc/pacman.d/gnupg/gpg.conf says it is using pool.sks-keyservers.net

Using my key’s long id I’ll push it to my default keyserver and tell pacman to pull it and then sign it.

#send my key
gpg --send-key F77636AC51B71B99
#tell pacman to pull that key from my keyserver
sudo pacman-key --keyserver keys.gnupg.net -r F77636AC51B71B99
#sign the key it recieved and start trusting it
sudo pacman-key --lsign-key F77636AC51B71B99

With all that done, I should be able to sign my package with

makepkg --sign --key F77636AC51B71B99

We can also shorten that by setting the default-key option in ~/.gnupg/gpg.conf.

# If you have more than 1 secret key in your keyring, you may want to
# uncomment the following option and set your preferred keyid.

default-key F77636AC51B71B99

With my default key set I’m able to make and install with this:

mike@longshot:~/projects/arangodb_pkg☺ makepkg --sign
mike@longshot:~/projects/arangodb_pkg☺ sudo pacman -U arangodb-2.8.1-1-x86_64.pkg.tar.xz
loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) arangodb-2.8.1-1

Total Installed Size:  146.92 MiB
Net Upgrade Size:        0.02 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring
[##################################################] 100%
(1/1) checking package integrity
[##################################################] 100%

The ease with which I can make my own packages is a very appealing part of Arch Linux for me. Signing them was the next logical step and I’m looking forward to exploring some related topics like running my own repo, digging deeper into GPG, the Web of Trust, and reproducible builds. It’s all fun stuff, if you can only find the time.

Private communications with Signal

The internet is a global network. You would think that the fact that every message sent over it passes though many legal jurisdictions would make the need for encryption obvious and uncontroversial. Sadly that is not the case.

The need for something more than legal safeguards becomes obvious when you see that a request from a Toronto home to toronto.com (whose server is in Toronto!) leaving Canadian legal jurisdiction on a path through both New York and Chicago before finally reaching it’s Toronto destination.

boomerang_route
An example of a boomerang route from the ixmaps.ca project. About 25% of traffic with both a start and end point in Canada is routed this way.

Applications that deliver technical safeguards, like end-to-end encryption, offer that “something more” that protects my data beyond the border. One of these a applications is Signal, a project of Open Whisper Systems.

In an offline world, privacy was the default, a product of things you didn’t say or do, and probably also a byproduct of how hard it was to move information around. As things like chatting with friends and family or reading a newspaper all moved online, those activities suddenly involved sending data in plain text over public infrastructure. Privacy become something that existed only for those that found a way to avoid the default of sending plain text. Privacy became a product of action rather than inaction.

Signal and its predecessor projects Textsecure and Redphone are part of an effort to make privacy the default again by rolling high end encryption into apps polished for mainstream adoption.

Screenshot_20151103-142606

Signal does two main things: sending text messages and making calls. What Signal actually does for secure communications is very different from what it does for insecure ones and is worth understanding.

Text messages

When sending a text message to someone who does not have Signal, the application sends a standard SMS message. The details of what constitutes an SMS message were hashed out in 1988 long before security was a thing and consequently, a related specification notes “SMS messages are transported without any provisions for privacy or integrity”, but importantly they are transported over the telephone network.

When sending secure text messages, Signal uses your mobile data to send the message using the Textsecure protocol v2.

The distinction between those two is worth making since coverage for data vs telephone can vary as can the costs, say if you are travelling and turn off mobile data.

The first time you send or receive a secure message with someone, behind the scenes you exchange cryptographic keys. Everything Signal does is focused on ensuring secure communication with the holder of that key. If the key for one of your contacts changes, it should be considered an event worth a quick phone call. This can happen innocently enough, say if they uninstall and then reinstall the app, but since all the other security measures a built on that, its worth asking about.

After the first text message has been sent or received, from then on Signal uses those keys to generate new keys for each individual message (Described in detail here.). This ensures that even if one message were to be decrypted, every other message is still safe.

Calling

Calling follows a similar pattern; for insecure calls Signal simply launches your phones standard phone app, while encrypted calls it handles itself. And like the secure text messages, this also uses your mobile data rather than routing through the phone network.

Secure calls are placed using the ZRTP protocol, the details of which are hidden from the user with one important exception.
On screen when you make a secure call you will notice two words displayed. These words are part of the ZRTP protocol and were generated based on the key that both parties used to encrypt the call.

zrtp_call

Both parties should see the same two words. If you say one and ask your contact to read the other, and they don’t match up, they keys you have agreed upon are not the same. If the keys are not the same it suggests someone has tampered with connection information inflight and inserted themselves into your conversation.

Verifying keys

Part of whole key exchange process that allows users to identify each other involves pulling your contacts public key from a central key directory server. The use of a central server means that I now have to trust that server not to maliciously give me a key for someone else. Open Whisper Systems Trevor Perrin addressed the problem of trusting unauthenticated keys in a talk at the NSEC security conference. It’s just a few minutes but its an interesting insight into the balancing act involved in bringing private communications to the masses:

For the interested or the paranoid, Signal lets you verify a contacts key by touching your contacts name/number at the top of your conversation. This brings up the details for that contact which includes a “Verify identity” option.verifyWith that, and your own identity details, found under Settings (three vertical dots on the top right of the main screen) > “My Identity Key”, you are able to either read a key fingerprint or if you have a QR/Barcode scanner you can use that to verify your contacts key.

scan_optionsverified

Open Source

Establishing that there is no secret behaviour or hidden flaws somewhere in the code is critical in this world where we put a significant amount of trust in computers (and where the definition of computer is expanding to encompass everything from voting machines to Volkswagens).

Signal establishes trust by developing the code in the open the code so that it can be reviewed (like this review of Signal’s predecessor Redphone by Cryptographer Matthew Green). Former Google security researcher Morgan Marquise-Boire has endorsed Signal as has Edward Snowden.

But even if you believe the experts that Signal works as advertised, its common for “free” apps to seem significantly less “free” once you realize what they do to turn a profit. With that in mind, another component of trust is understanding the business model behind the products you use.

When asked about the business model on a popular tech forum, Open Whisper Systems founder Moxie Marlinspike explained “in general, Open Whisper Systems is a project rather than a company, and the project’s objective is not financial profit.”

The project is funded by a combination of grants and donations from the Freedom of the Press Foundation and The Shuttlesworth Foundation among others. It is worked on by a core group of contributors led by and supporting cast of volunteers.

Final thoughts

Signal does a great job of making encrypting your communications a non-event. Encrypted as they travel the network, our messages are now secure against tampering and interception, no matter whose servers/routers they pass through. The result: privacy.

The fact that applying security measures result in privacy should tell you that the oft quoted choice between “security vs privacy” is a false one. As pointed out by Timothy Mitchener-Nissen, assuming a balance between these things only results in sacrificing increments of privacy in pursuit of the unachievable idea of total security. The ultimate result is reducing privacy to zero. Signal is just one way to grab back one of those increments of privacy.

All of that said my interest in privacy technologies and encryption is an interest for me as a technologist. If you are counting on these technologies like Signal to protect you from anyone serious (like a nation-state) the information above is barely a beginning. I would suggest reading this best practices for Tor and the grugq’s article on signals, intelligence. Actually anything/everything by the grugq.