Sending thousands of agreements without giving up privacy

CSV files have somehow survived every attempt by the software industry to kill them, so naturally that is where we started when we built bulk sending for Inklok.

The feature sounds ordinary. Upload a spreadsheet, map its columns to an agreement template, and send hundreds or thousands of agreements. Plenty of software does this. The interesting part is what happens after the upload.

A conventional server can take every row, combine it with a template, send the result, and keep whatever plaintext it needs along the way. Inklok has a less convenient requirement. The protected information people add to completed agreements should remain private from Inklok.

One agreement was the easy case

Privacy is fairly intuitive when one person prepares one agreement in a browser. The browser can encrypt protected values before they leave the device. Inklok stores ciphertext. The people who need to read the protected data receive the cryptographic access required to do so.

Now replace one agreement with thousands of rows in a CSV file. Each row might contain a different recipient and different values. Some recipients may already have Inklok accounts. Others may be guests who have never heard of us. The sender expects the batch to keep moving if they close a laptop. Email delivery can slow down. A vendor can have an outage. Workers can retry. A recipient can open an invitation days later.

Suddenly, "just loop over the rows" is not much of an architecture.

The tempting shortcut

We could have made bulk sending considerably easier by moving the sensitive work to the server and giving our backend whatever keys it needed to decrypt protected agreement data. That is a perfectly normal design for many applications.

It would also have weakened the privacy property that makes Inklok different. A feature intended to make the product more useful would have changed the trust model underneath it. We did not think that was a reasonable trade.

So we treated batch processing as an orchestration problem rather than permission to make the server omniscient.

A batch is orchestration, not one giant request

Large batches are asynchronous. The upload establishes the work to be done, then durable queues and workers move individual agreements through processing. That matters for boring reasons that become very exciting at 2 a.m.: retries, failures, timeouts, rate limits, and vendor outages.

It also means a batch can report useful progress instead of making a browser tab responsible for the survival of thousands of sends. Individual rows can succeed or fail independently. Work can be retried without resending everything. Delivery can pause without throwing away the batch.

We designed the database and queueing paths so scaling a batch does not concentrate all of its activity onto one database partition or one global counter. A system that works beautifully for a small test and melts when the batch gets serious is just a smaller batch system with good marketing.

Guests make privacy harder

Existing users can already have durable encryption identities. Guests are more interesting. When the sender prepares an agreement, the future signer may not have an Inklok account or a public key yet. We still need to prepare protected data for that person without giving Inklok a permanent decryption key.

Our signing flow handles that with an agreement specific invitation key and a short lived bootstrap secret. The sender can encrypt the necessary agreement key material for the future guest. When the guest arrives, the invitation is exchanged into the guest signing session. The guest creates a durable encryption identity and the required access is moved to that identity.

The temporary bootstrap material exists to get an invited person into the encrypted conversation. It is not a recovery key that lets Inklok come back later and read a completed agreement.

That distinction matters more in bulk sending because one batch can create thousands of future guest relationships at once.

Email is part of the distributed system

Sending an agreement also means sending an email, and email is an external system with its own failure modes. Treating an email API call as an incidental line at the end of a worker is an easy way to discover those failure modes in production.

Inklok hands delivery work to a durable email queue. Delivery can be retried and deduplicated independently from agreement creation. If the email provider is unavailable, queued work can wait instead of forcing the entire batch to fail.

We also do not want one bad batch to damage delivery for everyone else. Large sends are released progressively. Early delivery results act as a health check before later portions of a batch are allowed through. Elevated bounce or complaint signals can hold the remaining work rather than enthusiastically sending thousands more messages into a problem.

The browser still matters, but it cannot be the worker

We spent time exploring how much of batch processing should happen in the browser. There is an appealing purity to that approach. The browser already sits on the trusted side of the privacy boundary, so why not make it do everything?

Because browsers close. Phones sleep. Networks disappear. Browser background processes have lifecycle rules. External integrations do not necessarily have a human browser sitting around waiting to finish thousands of cryptographic operations.

The useful boundary is not "the server may never participate in processing." The useful boundary is that server participation should not quietly create permanent access to protected completed agreement data.

That lets us use durable server infrastructure for the work servers are good at while keeping the privacy promise tied to the data that actually needs protection.

Zero knowledge is a constraint, not a screen

None of this should require a customer to understand key wrapping, invitation keypairs, queue shards, or worker leases. The customer experience is intentionally boring: upload a CSV or JSON file, map the fields, review the batch, and send it.

That is the point. Zero knowledge is useful when it changes what the service is capable of seeing without making the person using the service become a cryptographer.

There are still visible parts of an agreement workflow that Inklok needs in order to operate the service, including routing and other operational metadata. Our security documentation describes that boundary in detail. The claim is deliberately narrower and more useful than pretending a working SaaS application knows absolutely nothing about anything.

What we learned from sending at scale

Bulk sending reinforced something we believed from the beginning: privacy works best when it is part of the architecture, not a feature attached to it later. Once thousands of agreements start moving through the system, every shortcut gets amplified. Encryption, reliability, delivery, recovery, and scale all have to work within the same security model. You cannot bolt privacy onto the side after the rest of the system has already decided who gets to see the data.

Encryption affects how guests are invited. Guest invitations affect how work is prepared. Preparation affects how batches are queued. Queues affect retries. Retries affect email delivery. Email delivery affects reputation controls. Reputation controls affect how quickly a large batch is allowed to progress.

Once you decide the service should not be able to read protected completed agreement data, that decision follows you into some surprisingly ordinary places.

Including, apparently, CSV uploads.