NaturalSpeech 3

Provider-documented facts. “Not collected” means we have not verified that field yet, not that the feature is unsupported.

Context / input limit

Not collected

No public service contract or verified value for this original research checkpoint.

Maximum input tokens

Not collected

No public service contract or verified value for this original research checkpoint.

Maximum output

Not collected

No public service contract or verified value for this original research checkpoint.

Parameters

500M factorized diffusion modelSource for Parameters (opens in a new tab)

Section 4.4 model-scale label; not a verified total including every codec and auxiliary component.

Reasoning

Not collected

No public service contract or verified value for this original research checkpoint.

Tool calling

Not collected

No public service contract or verified value for this original research checkpoint.

Structured output

Not collected

No public service contract or verified value for this original research checkpoint.

Streaming

Not collected

No public service contract or verified value for this original research checkpoint.

Batch processing

Not collected

No public service contract or verified value for this original research checkpoint.

Fine-tuning

Not collected

No public service contract or verified value for this original research checkpoint.

Context caching

Not collected

No public service contract or verified value for this original research checkpoint.

Knowledge cutoff

Not collected

No public service contract or verified value for this original research checkpoint.

Release date

Not collected

No public service contract or verified value for this original research checkpoint.

Input modalities

Text, AudioSource for Input modalities (opens in a new tab)

Output modalities

AudioSource for Output modalities (opens in a new tab)

Languages

EnglishSource for Languages (opens in a new tab)

Language of the documented experiment; not an exhaustive claim of supported production languages.

Research publication

2024-03-05Source for Research publication (opens in a new tab)

First research-family disclosure on arXiv; reviewed specifications may reflect a later paper revision. Not a commercial or separate variant release date.

Endpoint availability

Research paper and prerecorded samples. No public full-model download or interactive inference service found in reviewed original materials.Source for Endpoint availability (opens in a new tab)

Voice cloning

SupportedSource for Voice cloning (opens in a new tab)

Zero-shot research use of a reference voice, not an available commercial API.

Reference audio

Reference speech conditions the individual speech attributes; three-second prompts used in the LibriSpeech evaluation.Source for Reference audio (opens in a new tab)

An evaluation setting, not an advertised maximum reference duration.

Architecture

FACodec plus factorized discrete diffusion. Shared prosody/content/acoustic-detail Transformer has 12 layers, eight heads and 1,024 embedding dimensions; phoneme and duration paths use separate encoders.Source for Architecture (opens in a new tab)

Training dataset

LibriLight: 60,000 hours of English speech from approximately 7,000 speakers.Source for Training dataset (opens in a new tab)

Audio sample rates

16 kHz speech in the reported systemSource for Audio sample rates (opens in a new tab)

Voice controls

Attribute-specific prompts permit timbre, prosody and speech-duration manipulation.Source for Voice controls (opens in a new tab)

Audio codec design

FACodec factorizes content, prosody, acoustic detail and timbre. Public FACodec checkpoints cover the codec component, not the full diffusion synthesis system.Source for Audio codec design (opens in a new tab)

Training hardware

8 NVIDIA A100 80GB GPUs; one million steps for the documented main diffusion training setup.Source for Training hardware (opens in a new tab)

Research controls

Four iterations per diffusion process; paper reports 60 forward passes with classifier-free guidance. A one-step inference setting uses 15 forward passes.Source for Research controls (opens in a new tab)

Research configurations

500M / 1,000-hour LibriLight subset, 500M / 60,000-hour LibriLight main model, 500M / 200,000-hour internal corpus, 1B / 200,000-hour internal corpus (separate detail record)Source for Research configurations (opens in a new tab)

Separately trained data/model scaling experiments in Section 4.4; not commercial releases. The one-step latency variant changes inference iterations, not the training dataset.

Prices belong to an endpoint and billing tier. A provider listing does not guarantee access for every account or region.

Caching can reuse earlier input processing. Availability, lifetime and billing depend on the endpoint. A cache hit still generates a new response.

No benchmark observations have been added for this model. We do not turn a vendor description into a performance score.

Latency and throughput depend on the host, request size, reasoning settings and load. There is no measured speed or global usage figure in this record.

Why it is included

Classification is an evidence-based inference for the original research checkpoint: the authors provide papers and prerecorded samples, but no full original model weights or callable service in the reviewed project materials. The Microsoft Hugging Face account lookup also returns no matching model. This does not classify independent implementations or separately released codec components as closed.

Lifecycle dates

Sources reviewed