A mirror beside your edge. Six steps to a record.
The capture node sits beside your serving path, not in it. Nothing about how you answer traffic changes, nothing slows down, and removing the node is exactly as cheap as adding it. It watches, and it signs.
The entire integration project.
# your edge, before: requests are served and forgotten # after: one mirror directive beside the serving path location / { mirror /_lentra; ... } location = /_lentra { internal; proxy_pass http://127.0.0.1:7402; # capture node, your key }
changes to the path that serves your visitors
target time to a first signed record
key: yours, generated and held by you
Target figures for the integration path, not measured results.
Six steps between a request and a record.
Mirror
A copy of the request and the response arrives at the node. The visitor is never touched, delayed or answered differently; the capture interferes with nothing.
Classification
Is this machine traffic worth recording? Verified-bot lists, claimed agents, your own rules. What is not recorded is dropped, not stored.
Attribution evidence
The claim and the network behind it: claimed agent, peer address, ASN, reverse DNS with forward confirmation. Recorded, not judged: judgement is the court's job.
Terms snapshot
The robots.txt and machine-readable license state as served at that exact moment, pinned by digest. Not what the policy says today, but what it said then, to that client.
Record and signature
Six field groups, a content digest, a signature with your key and, where the weight is needed, a qualified-timestamp countersignature.
Ledger
The record joins the hash chain. Export a single visit or a year of them as portable exhibits; the chain travels with them.
The same node, in front of a crawler fleet, proves the discipline.
Point the mirror at your own egress instead of your edge, and the ledger records what your crawlers honoured and what they refused. In ordinary infrastructure a refusal is noise in a log. In an evidence layer it is the exhibit.
- Forbidden by reservation
- The fleet provably never made the request. The refusal is signed and kept: the central exhibit of a good-faith crawl policy.
- Target said no
- A disallowed target stays untouched, and the ledger shows it visibly, rather than in a policy document that asks to be believed.
- Terms changed mid-crawl
- The pinned digest shows which version of the terms was in force for every fetch. No argument about what the policy said when.
- Asked to prove it
- An inquiry about your copyright policy is answered with records: exportable, chained, verifiable by the asker without your help.
A crawl policy is a promise. A signed crawl ledger is a record. Regulators, counterparties and courts are rapidly learning the difference.
Closed trust does not scale.
The checking party does not need us at all, and that is the same property that makes the format a candidate for a standard. Opposing counsel, a marketplace, a regulator, a competitor: any of them can verify these records without asking us. Openness here is not a gesture. It is the strategy.
You hand over the records
A file, an export, however you like. The reviewer needs no contract with us.
The reviewer installs the open verifier
A library and a CLI. Runs locally, offline. The code is open.
It checks the signatures and the chain
Against the issuer's published keys. Yours, not ours.
Answer: authentic or not
Mathematics instead of correspondence. Minutes instead of weeks.