FilesCLOUD-SPECIFICPROTOCOL-SPECIFIC

Presigned URLs

Your backend issues a signed, expiring permission slip; the client uploads directly to storage; your process never sees a byte — which is the benefit and the cost.

What actually happensHow to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

How can a client write to your private storage bucket without holding your credentials, and what do you give up by never seeing the bytes?

The requirement

Video uploads, up to a few gigabytes, from browsers and mobile apps. They must land in private storage. The backend must not be a bottleneck and must not fall over when fifty people upload at once.

The obvious build

Keep the current upload endpoint and give the instances more memory. Or, if the bucket must be reachable from the browser, make it public and let clients write to it directly.

Why it breaks

More memory raises the ceiling and does not remove it. Fifty concurrent multi-gigabyte uploads exceeds any instance size you are willing to pay for (File Uploads Through the Backend).

How it breaks in production
  • More memory raises the ceiling and does not remove it. Fifty concurrent multi-gigabyte uploads exceeds any instance size you are willing to pay for (File Uploads Through the Backend).
  • A publicly writable bucket is not a solution, it is an open file host: anyone can write anything, overwrite anything, and use your storage bill to serve their own content.
  • Shipping storage credentials to the client is the same thing with extra steps — anything in a browser bundle or a mobile binary is extractable, and those credentials usually grant far more than one upload.
  • Routing through the backend also means the upload cannot resume: a dropped connection at 90% starts again at zero.
Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

  • A presigned URL is a normal storage URL with the request's essential parameters — bucket, key, method, expiry, and often the content type and size constraints — folded into a signature computed with your credentials. The storage service recomputes the signature from the request it receives and rejects anything that does not match.
  • The signature *is* the authorization. Nothing else is checked: no session, no user identity, no relationship to your application. Whoever holds the URL has exactly the permission it encodes, until it expires (Authorization in Backends).
  • That permission is scoped by whatever you signed. A URL signed for PUT on one specific key allows writing that one object and nothing else. A URL signed loosely — a key prefix, no content-type constraint, no size limit — allows correspondingly more.
  • The flow is three steps: the client asks your API for permission, your API authorizes the *user* and returns a signed URL, and the client uploads directly to storage. Your API sees step one and step three never comes back to it — which is why a fourth step is needed to learn that the upload happened.
  • The completion signal is the design problem. Either the client tells you (a call to your API after the upload succeeds, which an attacker or a crashed browser may never make) or storage tells you (an event emitted by the storage service on object creation, which is reliable but asynchronous and provider-specific) (What Happens After the Bytes Land).
  • The same mechanism works for downloads: a signed GET URL grants time-limited read access to a private object without proxying the bytes (Serving Files).

The permission slip flow

Four steps, and the important one is the last. Steps one and two are an ordinary authorized API call. Step three does not involve you at all. Step four is how your application ever finds out — and it is the step teams forget when they draw this diagram with three arrows.

  • Step 1–2 — the only authorization that happens. Get it right here or not at all.
  • Step 3 — the signature encodes exactly what you permitted, and nothing else is checked.
  • Step 4 — bytes flow between two parties you do not sit between.
  • Step 5–6 — the completion path. Without it, "uploaded" is a state you never observe.
1. may I upload?2. record intent3. signed URL, short expiry4. PUT bytes — you are not here5. object created6. match to intent, then processClientYour API authorize user, generate key, signObject storage verifies signature onlyIntent row status: pendingObject-created eventWorker verify, scan, process
UserLLMAgentToolDataDecisionHumanGuardrail

Sign narrowly

CLOUD-SPECIFICBinding content type and a content-length range is expressible in an S3 POST policy and in a GCS signed policy document; a plain S3 SigV4 PUT URL can bind the content type but not a size range, so on that path the size limit has to come from a bucket policy or from checking the object after the fact. The API shown here is illustrative, not any one SDK's surface — check what your provider can actually constrain before assuming a limit is enforced.

Everything a signature does not constrain is a permission you granted by omission. The difference between a safe presigned upload and an open bucket is entirely in the parameters, and the loose version looks perfectly reasonable in a code review.

Two signatures, very different permissions
1// LOOSE — every omission is a grant
2const url = await storage.signPut({
3  key: `uploads/${req.body.filename}`,  // client picks the key
4  expiresIn: 60 * 60 * 24,              // valid for a day
5})
6// permits: writing anywhere the client names, any type, any size, all day
7 
8// NARROW — the signature is the whole permission, so state all of it
9const attachment = await db.attachments.insert({
10  tenantId: ctx.tenantId,               // from the principal, never the body
11  ticketId,                             // authorized above, as an object check
12  key: `t/${ctx.tenantId}/tickets/${ticketId}/${randomUUID()}`,
13  status: 'pending',
14  expectedType: req.body.contentType,
15})
16 
17const url = await storage.signPut({
18  key: attachment.key,                  // one exact object
19  expiresIn: 300,                       // five minutes
20  contentType: attachment.expectedType, // bound: a different type fails the signature
21  contentLengthRange: [1, 50 * 1024 * 1024],
22})

The loose version is not a weaker version of the same thing — it is a different feature. It grants write access across a shared prefix for 24 hours to whoever holds the URL, which includes anyone who finds it in a log. Note also that the narrow version derives the tenant from the principal: the key is where tenant isolation is enforced for files (Tenant Isolation).

What moves, and where it has to go

The honest way to evaluate this change is as a relocation, not a removal. Every responsibility the upload endpoint had still exists; it now happens at a different time, in a different process, with different failure modes. Teams that adopt presigned uploads without moving these end up with a fast upload path and no validation at all.

ResponsibilityThrough the backendDirect with a signed URL
AuthorizationIn the upload handlerIn the signing request — the only place left
Size limitEnforced while reading the bodyBound in the signature if the provider allows; otherwise checked after the object exists
Type checkingSniffed from the bytes before storingDeclared at signing; verified after the fact by reading the stored object
Virus scanningBefore the object existsAfter it exists, before it is downloadable — a state machine, not a check (What Happens After the Bytes Land)
Knowing it happenedThe handler returned 200A storage event or a verified client callback
IdempotencyOne request, one objectUnique server-generated key makes the write idempotent by construction
CleanupNothing to clean if the request failedAbandoned intents and orphaned objects both need sweeping
Cost profileIngress + egress + compute per byteOne small API call per upload; storage handles the bytes
Failure surfaceYour logsProvider errors surfaced to the client, invisible to you (What a Backend Should Actually Log)

How to build it

Most important first.

  • Authorize the *request for the URL* exactly as you would authorize the upload itself: this user, this ticket, this tenant, this quota. That call is where all of your authorization now lives (Object-Level Authorization).
  • Generate the key server-side and sign for that exact key. Never sign a prefix or accept a client-supplied key — a client that picks its own key can overwrite another tenant's object (Tenant Isolation).
  • Constrain what the signature permits: exact key, method, short expiry, and — where the provider supports it — content type and a maximum size. Every constraint you omit is a permission you granted.
  • Keep expiry short. Minutes, not hours. The URL is a bearer credential and it will end up in logs, browser history and screenshots (Short-Lived Credentials).
  • Record an intent row before signing — pending, with the key, the owner, the tenant and the expected type — so a later completion event can be matched to something you authorized (What Happens After the Bytes Land).
  • Prefer a storage-emitted event over a client callback for completion, and treat the client callback as a latency optimization rather than the source of truth. If you must trust a client callback, verify the object exists and its size and type match before marking anything complete.
  • Use the provider's multipart or resumable upload flow for large files, signing each part, so a dropped connection resumes instead of restarting.
  • Enforce quotas at signing time. It is the only moment you control, because after that the bytes go somewhere you are not.

What can go wrong

Failure modes
  • The client obtains the URL and never uploads: an intent row stuck in pending forever, needing expiry and cleanup.
  • The upload succeeds and the completion call is lost: the object exists and your application does not know, which is only recoverable if you reconcile against storage.
  • Clock skew between your signer and the storage service causing valid URLs to be rejected as expired, or expired ones accepted.
  • CORS not configured on the bucket, so browser uploads fail with an opaque network error that looks like a client bug (CORS Without the Myths).
  • A signature that permits more than intended — a prefix instead of a key, no content-type binding, no size cap — discovered only when someone abuses it.
  • Storage events arriving out of order, more than once, or for objects your application never authorized (At-Least-Once Delivery).
  • A file uploaded and immediately served to other users before any scanning has run (What Happens After the Bytes Land).
  • Presigned URLs generated in bulk and cached by the client, extending the effective permission window far past what you intended.
What can race
  • The upload completes after the URL expires from your application's point of view but before the storage service considers it expired — an object arrives for an intent you already cleaned up.
  • A completion event and a client callback both arrive, or the event arrives twice, producing duplicate processing unless the handler is idempotent (Webhook Idempotency).
  • Two clients issued URLs for the same key would overwrite each other; server-generated unique keys make this impossible, which is another reason not to accept a client key.
Security
  • If the signed URL permits a key prefix rather than one exact key, an attacker gets write access across that prefix: they can overwrite other users' — or other tenants' — objects with content of their choosing, which turns an upload feature into a content-injection primitive.
  • If the client chooses the key, the attacker gets the same thing more directly. Server-generated keys are the control.
  • If expiry is long, an attacker who obtains the URL from a log, a referrer header, a shared screenshot or browser history gets a valid write (or read) credential for as long as it lasts. Nothing else is checked.
  • If the signature does not bind content type and size, an attacker gets to upload anything of any size into your bucket — a storage-cost attack, and a malware host if the object is later served.
  • If completion is trusted from the client without verification, an attacker gets to declare arbitrary objects complete: mark a file they never uploaded as ready, or attach an object from a different intent to their own record.
  • If storage credentials are shipped to the client instead of signatures, an attacker gets everything those credentials can do, forever — the failure mode presigning exists to prevent (Roles vs Static Keys).
  • If nothing scans the object before it is downloadable by others, your product distributes whatever was uploaded. Going direct does not remove that requirement; it moves it after the fact (File Upload Security).
Misreads
  • "Presigned URLs are more secure." They are more *scoped* than shipping credentials and strictly less inspected than proxying the bytes. Which is safer depends on what you needed to check (Choosing an Upload Path).
  • "The URL is secret, so it is safe." It is a bearer token in a URL — the least protected place a credential can live. Short expiry is the mitigation, not secrecy.
  • "The client tells us when it is done, so we know." The client is untrusted and also unreliable. Verify against storage, or use a storage event (The Trust Boundary).
  • "We can validate on the URL request." You can validate the user, the quota and the declared intent. You cannot validate bytes that do not exist yet.
  • "This removes the need for a size limit." Bind the size in the signature where the provider supports it; otherwise the limit is whatever the client feels like uploading.

Operating it

How you see it in production
  • Count signatures issued versus uploads completed. A persistent gap is either broken clients, abandoned uploads or someone harvesting URLs (The Metrics a Backend Must Emit).
  • Track the age distribution of pending intent rows. A growing tail means completion signals are being lost.
  • Reconcile storage listings against your database on a schedule: objects with no intent row, and intent rows with no object. Neither shows up in request logs.
  • Log the key, the principal and the expiry for every signature issued — never the signed URL itself, which is a credential (Secrets in Logs).
  • Alert on signature requests per principal. It is the rate limit that matters, since the uploads themselves never touch your service (Rate Limiting).
What changes at 10x and 100x
  • This is the change that makes upload volume independent of your compute. Your service handles one small JSON request per upload regardless of file size, so a gigabyte and a kilobyte cost you the same (Making an Existing Service Stateless).
  • At 100x, the constraint moves to the storage service's own request rate and your event-processing pipeline, neither of which is your instance count.
  • Nothing changes for a 50 KB avatar. Introducing signing, an intent row, an event pipeline and a reconciliation job for small files is complexity with no corresponding benefit (Choosing an Upload Path).
  • Egress cost becomes the dominant number long before compute does, and it is charged per byte leaving the provider — which is why serving strategy and storage location matter more than upload strategy at volume (Egress: Moving Data Costs Money, Not Just Storing It).
What this costs
  • Your backend never sees the bytes. Every check that needed the bytes — virus scanning, content validation, real size enforcement, format verification — moves to after the object exists, or does not happen.
  • The upload becomes asynchronous from your application's point of view. "Uploaded" is now a state your system learns about later, which means an intent lifecycle, a completion signal, reconciliation and cleanup — several moving parts replacing one handler.
  • A signed URL is a bearer credential, with all the handling problems that implies: it leaks through logs and referrers, it cannot be revoked before expiry on most providers, and its power is exactly what you signed.
  • Debugging is harder. A failed upload happens between two systems you do not control, and the error the user sees comes from the storage provider in the provider's vocabulary.
  • Provider coupling increases: signing, constraints, multipart flow and completion events are all provider-specific surface in your code (Object Storage).

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

  • CLOUD-SPECIFICEvery major object store offers signed URLs, and the details differ in ways that reach your code. S3 signs with SigV4 and supports both a simple PUT URL and a POST policy that can bind content-length range and content type; GCS has V4 signed URLs and separately signed policy documents, with different canonicalization; Azure Blob uses shared access signatures, which can be delegated from a user identity and revoked by rotating a stored access policy — a revocation story the others do not have in the same form. Maximum expiry, what can be constrained in the signature, and whether presigned URLs work with customer-managed encryption keys all differ. Treat "issue a signed URL" as portable and every parameter as not.
  • PROTOCOL-SPECIFICBrowser uploads to a different origin are subject to CORS on the bucket, including a preflight for a PUT with a content type; native mobile clients are not. The same signed URL that works from a mobile app can fail from a browser purely on CORS configuration.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

API Designfile-upload-apisasync-job-pattern
Securityshort-lived-credentialsfile-upload-securitycors
Cloudobject-storageroles-vs-static-keysegress-costs
Domains that do not exist yet
  • — Distributed Systems — an upload that your application only learns about through an event is a two-phase workflow with an unreliable participant, and inherits every reconciliation problem that implies.