Skip to content
Model infoModelsSnapshot: 15 Oct 2026

Microsoft AI / NaturalSpeech 2

NaturalSpeech 2 (singing experiment)

Separately trained NaturalSpeech 2 singing configuration, mixing additional singing data with speech and conditioning on another song’s pitch and duration.

Speech & voice
restrictedClosed weightsText-to-speech
Official documentation (opens in a new tab)Reviewed 14 Oct 2026
15 of 27 specification fields collected
Audio sample rates16 kHzSee specification notes below
Languages1 listedFull language list in specifications
Input billingNot collectedMicrosoft Research (prerecorded samples)
Output billingNot collectedNo inferred price

Specifications

Provider-documented facts. “Not collected” means we have not verified that field yet, not that the feature is unsupported.

Context / input limit
Not collected

No public service contract or verified value for this original research checkpoint.

Maximum input tokens
Not collected

No public service contract or verified value for this original research checkpoint.

Maximum output
Not collected

No public service contract or verified value for this original research checkpoint.

Parameters
435M total in Table 11; described as 400M in the introductionSource for Parameters (opens in a new tab)

Published component counts: codec 27M, phoneme encoder 72M, duration predictor 34M, pitch predictor 50M, speech-prompt encoder 69M and diffusion model 183M.

Reasoning
Not collected

No public service contract or verified value for this original research checkpoint.

Tool calling
Not collected

No public service contract or verified value for this original research checkpoint.

Structured output
Not collected

No public service contract or verified value for this original research checkpoint.

Streaming
Not collected

No public service contract or verified value for this original research checkpoint.

Batch processing
Not collected

No public service contract or verified value for this original research checkpoint.

Fine-tuning
Not collected

No public service contract or verified value for this original research checkpoint.

Context caching
Not collected

No public service contract or verified value for this original research checkpoint.

Knowledge cutoff
Not collected

No public service contract or verified value for this original research checkpoint.

Release date
Not collected

No public service contract or verified value for this original research checkpoint.

Input modalities
Text, AudioSource for Input modalities (opens in a new tab)
Output modalities
AudioSource for Output modalities (opens in a new tab)
Languages
EnglishSource for Languages (opens in a new tab)

Language of the documented experiment; not an exhaustive claim of supported production languages.

Research publication
2023-04-18Source for Research publication (opens in a new tab)

First research-family disclosure on arXiv; reviewed specifications may reflect a later paper revision. Not a commercial or separate variant release date.

Endpoint availability
Research paper and prerecorded samples. No public full-model download or interactive inference service found in reviewed original materials.Source for Endpoint availability (opens in a new tab)
Voice cloning
SupportedSource for Voice cloning (opens in a new tab)

Zero-shot research use of a reference voice, not an available commercial API.

Reference audio
Speech or singing reference for timbre; ground-truth pitch and duration supplied by a separate singing recording.Source for Reference audio (opens in a new tab)
Architecture
Residual-quantized audio codec and a 40-layer WaveNet diffusion model, with Transformer phoneme and speech-prompt encoders and attention-conditioned pitch/duration predictors.Source for Architecture (opens in a new tab)
Training dataset
Approximately 30 hours of cleaned web singing/lyrics, upsampled and mixed with the speech dataset for the singing experiment.Source for Training dataset (opens in a new tab)
Audio sample rates
16 kHzSource for Audio sample rates (opens in a new tab)
Audio codec design
16 residual-vector-quantization blocks; codebook size 1,024, dimension 256 and hop size 200.Source for Audio codec design (opens in a new tab)
Research speech tokenizer rate
80 latent frames per second (16 kHz / hop size 200), not an API token billing rate.Source for Research speech tokenizer rate (opens in a new tab)
Denoising steps
1,000Source for Denoising steps (opens in a new tab)

Experimental inference setting, not a product limit.

Research controls
Ground-truth pitch/duration conditioning; singing experiment learning rate 0.00005 and 1,000 diffusion steps.Source for Research controls (opens in a new tab)

Access & pricing

Prices belong to an endpoint and billing tier. A provider listing does not guarantee access for every account or region.

Microsoft Research (prerecorded samples)

Provider source (opens in a new tab)
API model ID
Not collected

No public API model selector.

Access
Read the paper and listen to prerecorded examples. No user-callable inference endpoint established by the reviewed project.
Regions
Not applicable to prerecorded research samples.

Endpoint pricing has not been collected. Check the provider for applicable rates.

Data & enterprise terms

No hosted customer inference or data-retention contract verified.

Caching

Caching can reuse earlier input processing. Availability, lifetime and billing depend on the endpoint. A cache hit still generates a new response.

01 / INITIAL INPUTProcess & retain

The system processes eligible input and retains reusable state.

02 / LATER REQUESTCheck available state

Reuse depends on matching input, session state and the cache policy.

03 / CACHE HITReuse & generate

Reusing state can reduce repeated work. Any charges follow the billing rules below.

Microsoft Research (prerecorded samples)

Not verified

No public service cache contract verified.

How it works
Not collected
Minimum reusable input
Not collected
Lifetime & refresh
Not collected
How to enable it
Not collected
Usage fields
Not collected
Billing rules
Not collected
Limits & data boundaries
Not collected

No verified cache rate for this endpoint. An empty rate is not a zero charge.

Caching reviewed 14 Oct 2026

What the measurements say

No benchmark observations have been added for this model. We do not turn a vendor description into a performance score.

Latency and throughput depend on the host, request size, reasoning settings and load. There is no measured speed or global usage figure in this record.

The record behind the model

Why it is included

Classification is an evidence-based inference for the original research checkpoint: the authors provide papers and prerecorded samples, but no full original model weights or callable service in the reviewed project materials. The Microsoft Hugging Face account lookup also returns no matching model. This does not classify independent implementations or separately released codec components as closed.

Classification evidence (opens in a new tab)

Lifecycle dates

Research family first published on arXiv18 Apr 2023Source (opens in a new tab)

Sources reviewed