Offline JSON Pretty Printing

Today when you’re dealing with Web APIs, you often find yourself in the situation of handling JSON, either in the input for these APIs or in the output, or both. Some browsers have the means to pretty print the JSON from their dev tools. But you don’t always have that opportunity. That’s why there are tools to pretty print JSON. I’ve found quite a few of them on the web, but all the ones I’ve found have one terrible flaw: they actually send the JSON you’re trying to pretty print to the server (*shudder*). I don’t want my JSON data (sensitive or not) to be sent to some random servers!

All your JSON are belong to us!

Now as I wrote, I don’t particularly like the fact that my JSON data is sent over the wire for pretty printing. It may not be super secret or anything, but in these days, you cannot be careful enough. Besides, it’s completely unnecessary to do it. All you need is already in your browser! So I quickly built my own JSON pretty printer (and syntax highlighter). You can find it right here.

Offline JSON Pretty Printing to the Rescue

Actually, the design is very simple. All my JSON pretty printer is doing, is to take your JSON input and try to parse it as JSON in the browser.

JSON.parse(yourJsonInput)

If that fails, I’m showing the parsing error and it’s done. If it succeeds, I get back a JavaScript object/array/value, which then I’m inspecting. For objects, I’m using basic tree navigation to go through all the properties and nested objects/arrays/values for pretty printing. That’s it, really simple. No need to transmit the data anywhere — it stays right in your browser!

So like it, hate it, use it or don’t: cymbeline.ch JSON Pretty Printer

Gzip Encoding an HTTP POST Request Body

I was wondering how difficult it was to Gzip-compress the body of an HTTP POST request (or any HTTP request with a body, that is), for large request bodies. While the .Net HttpClient has supported compression of response bodies for a while, it appears that to this day there is no out-of-the-box support for encoding the body of a request. Setting aside for now that the server may not natively support Gzip-compressed request bodies, let’s look at what we need to do to support this on the client side.

Enter HttpMessageHandler

The HttpMessageHandler abstract base class and its derived classes are used by the HttpClient class to asynchronously send HTTP requests and receive the response from the server. But since we don’t actually want to send the message ourselves – just massage the body and headers a little bit before sending – we’ll derive a new class GzipCompressingHandler from DelegatingHandler so we can delegate sending (and receiving) to another handler and just focus on the transformation of the content. So here’s what that looks like.

public sealed class GzipCompressingHandler : DelegatingHandler
{
    public GzipCompressingHandler(HttpMessageHandler innerHandler)
    {
        if (null == innerHandler)
        {
            throw new ArgumentNullException("innerHandler");
        }

        InnerHandler = innerHandler;
    }

    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        HttpContent content = request.Content;

        if (request.Method == HttpMethod.Post)
        {
            // Wrap the original HttpContent in our custom GzipContent class.
            // If you want to compress only certain content, make the decision here!
            request.Content = new GzipContent(request.Content);
        }

        return base.SendAsync(request, cancellationToken);
    }
}

As you can see, all we’re doing is just wrapping the original HttpContent in our GzipContent class. So let’s get right to that.

Gzip-compressed HttpContent: GzipContent

We’re almost there, all we need to do is actually compressing the content and modify the request headers to indicate the new content encoding.

internal sealed class GzipContent : HttpContent
{
    private readonly HttpContent content;

    public GzipContent(HttpContent content)
    {
        this.content = content;

        // Keep the original content's headers ...
        foreach (KeyValuePair<string, IEnumerable<string>> header in content.Headers)
        {
            Headers.TryAddWithoutValidation(header.Key, header.Value);
        }

        // ... and let the server know we've Gzip-compressed the body of this request.
        Headers.ContentEncoding.Add("gzip");
    }

    protected override async Task SerializeToStreamAsync(Stream stream, TransportContext context)
    {
        // Open a GZipStream that writes to the specified output stream.
        using (GZipStream gzip = new GZipStream(stream, CompressionMode.Compress, true))
        {
            // Copy all the input content to the GZip stream.
            await content.CopyToAsync(gzip);
        }
    }

    protected override bool TryComputeLength(out long length)
    {
        length = -1;
        return false;
    }
}

Easy, right? Of course you could add other supported compression algorithms, using more or less the same code (or even adding some abstraction for different compression algorithms), but this is basically all that’s required.

Summary

Using the HttpMessageHandler and its associated classes makes it extremely easy to apply transformations to all (or a well-defined subset) of HTTP requests you’re sending. In this case, we’re applying Gzip-compression to the bodies of all outgoing POST requests, but the logic to decide when to compress can be as customized as you want; you could even apply Gzip-compression only if the requested URI ends with “.gzip” or for certain content types.

Dynamic AES Key Exchange Through RSA Encryption

I wanted to prototype encrypted communication channel between a client and a server. Now of course there are HTTPS and other TLS channels that work quite well, but what I have in mind is supposed to be used to transfer rather sensitive data. So how can I establish a secure channel through an HTTP/HTTPS channel?

  1. Have the server generate an RSA key pair and send the public key to the client.
  2. Have the client generate an AES key, encrypt it with the received public key, and send the encrypted key to the server.
  3. Let the server decrypt the AES key.
  4. Both the client and the server are now in possession of the same AES key and can therefore communicate securely.

Of course, the generated AES key should only be used for the communication with the one client which sent it, so some sort of secure key management on the server (also regarding the RSA key pair) is vital. Also, the AES key could periodically be updated (i.e. a new key generated). At the very least, every message sent back and forth encrypted with AES will have to use a separate IV — but naturally that IV could be part of the transmitted message. So let’s get a very basic REST API-based implementation going.

Generate RSA key-pair on the Server

[...]

public sealed class SessionKey
{
    public Guid Id;
    public byte[] SymmetricKey;
    public RSAParameters PublicKey;
    public RSAParameters PrivateKey;
}

[...]

private Dictionary<Guid, SessionKey> sessionKeys;

[...]

public RSAParameters Generate(Guid sessionId)
{
    // NOTE: Make the key size configurable.
    using (RSACryptoServiceProvider rsa = new RSACryptoServiceProvider(2048))
    {
        SessionKey s = new SessionKey()
        {
            Id = sessionId,
            PublicKey = rsa.ExportParameters(false /* no private key info */),
            PrivateKey = rsa.ExportParameters(true /* with private key info */),
            SymmetricKey = null, // To be generated by the client.
        };

        sessionKeys.Add(id, s);

        return s.PublicKey;
    }
}

[...]

This key generation can then be used to generate a new RSA key pair whenever a new client connects and requests secure communication. Of course, make sure you send the public key back to the client, and not the private key — else there’s no point in encrypting in the first place.

Generate an AES key on the Client

[...]

// Get the Public Key from the Server
RSAParameters publicKey = GetFromServer(...);

// Holds the current session's key.
byte[] MySessionKey;

// Send encrypted session key to Server.
SendToServer(GenerateAndEncryptSessionKey(publicKey));

[...]

private byte[] GenerateAndEncryptSessionKey(RSAParameters publicKey)
{
    using (Aes aes = Aes.Create())
    {
        aes.KeySize = aes.LegalKeySizes[0].MaxSize;
        // Setting the KeySize generates a new key, but if you're paranoid, you can call aes.GenerateKey() again.

        MySessionKey = aes.Key;
    }

    using (RSACryptoServiceProvider rsa = new RSACryptoServiceProvider())
    {
        rsa.ImportParameters(publicKey);

        return rsa.Encrypt(MySessionKey, true /* use OAEP padding */);
    }
}

[...]

As you can see, we just take the public key we got from the server to set up the RSA provider and then encrypt the generated AES key using that public key. Once the client sends the encrypted key to the server, they both share the same secret and can securely communicate with each other.

Decrypt AES Key on the Server

[...]

public void SetSymmetricKey(Guid id, byte[] encryptedKey)
{
    SessionKey session = sessionKeys[id];

    using (RSACryptoServiceProvider rsa = new RSACryptoServiceProvider())
    {
        rsa.ImportParameters(session.PrivateKey);

        session.SymmetricKey = rsa.Decrypt(encryptedKey, true /* use use OAEP padding */);
    }
}

[...]

Since we already have the private key for this session, we can just use it to decrypt the AES key we got from the client. Again, making sure that the stored symmetric key is safe, is key to security.

Encrypt / Decrypt

Encrypting and decrypting can now be done the same way on both sides (since we’re using a symmetric-key algorithm). So here’s what that looks like.

[...]

public byte[] EncryptData(byte[] key, string data)
{
    using (Aes aes = Aes.Create())
    {
        byte[] result;

        aes.Key = key;
        aes.GenerateIV();

        using (ICryptoTransform encryptor = aes.CreateEncryptor())
        {
            using (MemoryStream ms = new MemoryStream())
            {
                using (CryptoStream cs = new CryptoStream(ms, encryptor, CryptoStreamMode.Write))
                {
                    using (StreamWriter writer = new StreamWriter(cs))
                    {
                        writer.Write(data);
                    }
                }

                byte[] encrypted = ms.ToArray();
                result = new byte[aes.BlockSize / 8 + encrypted.Length];

                // Result is built as: IV (plain text) + Encrypted(data)
                Array.Copy(aes.IV, result, aes.BlockSize / 8);
                Array.Copy(encrypted, 0, result, aes.BlockSize / 8, encrypted.Length);

                return result;
            }
        }
    }
}

public string Decrypt(byte[] key, byte[] data)
{
    using (Aes aes = Aes.Create())
    {
        aes.Key = key;

        // Extract the IV from the data first.
        byte[] iv = new byte[aes.BlockSize / 8];
        Array.Copy(data, iv, iv.Length);
        aes.IV = iv;

        // The remainder of the data is the encrypted data we care about.
        byte[] encryptedData = new byte[data.Length - iv.Length];
        Array.Copy(data, iv.Length, encryptedData, 0, encryptedData.Length);

        using (ICryptoTransform decryptor = aes.CreateDecryptor())
        {
            using (MemoryStream ms = new MemoryStream(encryptedData))
            {
                using (CryptoStream cs = new CryptoStream(ms, decryptor, CryptoStreamMode.Read))
                {
                    using (StreamReader reader = new StreamReader(cs))
                    {
                        return reader.ReadToEnd();
                    }
                }
            }
        }
    }
}

[...]

As you can see, each time we encrypt something we generate a new IV, which we send at the beginning of the data to the other side. The other side then extracts the IV first and uses it to initialize AES.

REST APIs?

Using all this through REST APIs is trivial: All you really need to make sure is that the client sends the session GUID (or whatever you use to identify a session) with every encrypted message, either through the URL, parameters or headers. Of course it is vital to guarantee that a client cannot get access to another client’s session (e.g. to provide a new session key), but through ordinary (secure) authentication that should easily be doable.

Next Steps

As far as encryption is concerned, this should already do the trick. You may want to add signatures to the encrypted messages too, to make sure that the encrypted blocks have not been tampered with. In addition, the AES key exchange could be repeated periodically (maybe even after every exchanged message).