What Truthring publishes, and what it will never change quietly

A transparency page is usually a table of subpoena counts and nothing else. The more useful version tells you how a company behaves when its own numbers turn against it, so most of this page is about that.

Four things we publish that a detector usually does not

Both error directions, separately

Every detector has two ways of being wrong. It can miss synthetic speech, and it can accuse a genuine recording of being fake. These are not interchangeable, and averaging them into one accuracy percentage hides the only fact that determines whether a result is safe to rely on. The rate that matters most is the second one, because a person wrongly flagged has no way of proving a negative. Truthring publishes the false positive rate next to the detection rate, broken out by recording condition, on the accuracy page and on individual reports.

There is a second asymmetry that follows from this and is worth restating in the same breath. A verdict of likely synthetic is the stronger of the two answers, because reaching it requires positive evidence in the signal. A verdict of likely human is weaker than it sounds: it can mean the recording is genuine, or it can mean a phone codec stripped out the evidence somewhere between the speaker and the file. We say which of those we are looking at, rather than letting a reassuring word do work it cannot support.

The conditions under which we fail

A published failure mode is more useful than a published success rate, because it tells you when not to use the tool at all. Short clips, several people talking over each other, music underneath speech, a voice note forwarded through two platforms, a language the model was not evaluated on: each of these degrades a result in a way that is predictable and therefore worth writing down in advance. Limitations is the page that exists to be quoted against us.

The engine version on every result

Detection models are retrained as new generators appear, which means a verdict is only interpretable if you know which model produced it. Every Truthring result and every report will name the engine version and the date it was analysed. A result from six months ago should be readable as a statement about that model, not silently reinterpreted as though it came from the current one.

A changelog that records the bad news

The method changelog exists to record every change to the engine and to the numbers it produces, including the ones that make the product look worse. If a retrain improves telephone audio and costs us something on clean studio recordings, both halves go in the entry. If a new generation system defeats the model until we catch up, the gap is dated at both ends. A changelog that only ever moves upward is not a record, it is an advertisement.

Government and legal requests

Lacewing Technologies is an Indian company and Truthring is operated from Navi Mumbai, so requests will arrive under Indian law and, potentially, through mutual assistance from elsewhere. The policy has four parts.

• We will require lawful process. A request from any authority, in India or abroad, will be answered only where it is valid, properly served and binding on us. An informal approach from an investigator is not process.
• We will produce the narrowest thing that satisfies it. Because of how retention is structured, what we hold about a given check is often much less than a requester expects. The audio has the shortest life of the four objects a check produces, and if it has expired we cannot produce it for anyone. Data retention sets that out.
• We intend to notify the person concerned. Where the law permits it, and before material is handed over, so that they have the opportunity to object. Whether this holds in every category of request under Indian procedure is [VERIFY: notification policy under legal review; exceptions not enumerated]
• We will count what we receive. Requests received, requests refused, requests complied with, and users notified, on a fixed schedule [VERIFY: reporting period and first publication date not set]

The count today is zero in every category, and that zero deserves an explanation rather than a proud presentation. There have been no requests because there is no live service, no submitted audio and no users. A transparency report of zeroes proves only that a product has not launched.

Corrections are published, never absorbed

The rule we are binding ourselves to is short. Anything that changes the meaning of a published page carries a dated note saying what changed. Fixing a spelling mistake does not qualify. Changing a percentage, a policy, a retention period, a claim about what the product does, or a statement about who has access to your audio does qualify, every time.

This matters more for a detection product than for most, because the whole proposition is that a Truthring result can be handed to a third party and checked. A company that revises its own record retrospectively has no standing to ask anyone to trust a document it issued last month. If you find a page that appears to have changed without a note, tell us at corrections@aivoicedetctor.com and the correction to the correction will be published too.

What we do not publish yet

An honest transparency page has to include its own gaps, or it is doing the same selective reporting it criticises.

• No request statistics. There have been no legal or government requests, because there are no users and no stored audio to request. The first report will be published once there is a service to report on.
• No third-party audit, and no certification. Nobody outside Lacewing Technologies has examined our infrastructure, our controls or our evaluation method. There is no SOC 2 report and no ISO 27001 certificate anywhere in the company. Independent verification is what would make the rest of this page worth something, and we do not have it.
• No measured accuracy figures. The engine has not been evaluated on a held-out set that we are prepared to stand behind in public. Until it has, the accuracy page carries placeholders rather than numbers [VERIFY: evaluation not complete]
• No full evaluation dataset. What we intend to release, and under which licence, is described on the dataset page and is not yet released.
• No takedown or abuse statistics. Same reason as the request count. Nothing to count.

Questions people put to us