Detecting VALL-E and its descendants

VALL-E is a research speech synthesis system described in published work by researchers at Microsoft. There is no shop for it and, as far as we are aware, no consumer product you can sign up for [VERIFY: verify]. Coverage on this page is therefore a slightly different sort of promise from the one we make about a commercial vendor, and it is worth explaining before you read a rate.

Research is not a category of harm. It is a schedule.

People sometimes read coverage of a research system as scaremongering — a detector claiming credit for guarding against something that does not exist in the world. The objection would be fair if the pipeline from a paper to a phone call were long or uncertain. It is neither.

What actually happens is dull and repeatable. A capability is demonstrated and written up. Other developers, working only from the description, produce their own versions and publish the code. Those versions get weights, then a command-line wrapper, then a graphical one, then a hosted demo that requires no installation at all. At the end of that chain sits someone who has never read a paper, using a capability the paper introduced.

Each step is public and each takes a knowable amount of time. A detection service that begins work at the last step is arriving after the harm has already reached people. We would rather hold a weak, honestly labelled signature early than a strong one that arrives late.

What a family signature can and cannot say

For a commercial vendor, a signature tracks a product: one publisher, one release schedule, one thing to retest when it changes. For a research lineage, the object is looser. Several teams implement the same described method and arrive at systems that share broad architectural traits while differing in specifics.

So our verdict here is deliberately coarser. A clip may be reported as consistent with this family, which is a genuine and useful statement, without being reported as the output of any particular build — because the traits we can read are shared by implementations we cannot separate. We would rather publish the coarser claim than dress it up.

That also means the two rates above should be read differently. The synthetic-or-not figure describes something stable: the difference between manufactured and captured audio does not depend on whose implementation produced it. The attribution figure describes something fragile, and it is the number most likely to fall when a new reimplementation appears.

Where this fails

• Implementations that diverge sharply. A version that departs substantially from the described method may not match the family signature at all, and will be reported as unknown generator if it is reported as synthetic.
• Demonstration audio that has been re-encoded. Research samples circulate as compressed files copied between sites. By the time one reaches us it may have been through several encodes, each removing evidence.
• Small sample sizes. Coverage of research families rests on fewer clips from fewer sources than coverage of a shipped product. Confidence intervals are correspondingly wider; the figures are on the benchmark page.
• Short clips. Below [VERIFY: n] seconds the family signature contributes little and the verdict rests on capture history.

How this shapes what we choose to cover

We add a system to our coverage when we can measure it, not when it becomes newsworthy. In practice that means we monitor published work, build test material as reimplementations appear, and publish a page like this one with the honest label — family-level rather than product-level — rather than waiting for a launch that may never come.

It also means we sometimes carry a signature that produces very few real-world matches. That is an acceptable cost. The alternative is a coverage list that describes the market rather than the threat, which is a comfortable thing to publish and not much use to anyone holding a suspicious file.

Questions