Skip to content
Model infoModelsSnapshot: 15 Oct 2026

Microsoft AI / VALL-E

VALL-E

Original Microsoft neural-codec speech model, trained on English audiobooks for zero-shot speech synthesis from short reference recordings.

Speech & voice
restrictedClosed weightsText-to-speech
Official documentation (opens in a new tab)Reviewed 14 Oct 2026
16 of 29 specification fields collected
Audio sample rates24 kHz waveform codecSee specification notes below
Languages1 listedFull language list in specifications
Input billingNot collectedMicrosoft Research (prerecorded samples)
Output billingNot collectedNo inferred price

Specifications

Provider-documented facts. “Not collected” means we have not verified that field yet, not that the feature is unsupported.

Context / input limit
Not collected

No public service contract or verified value for this original research checkpoint.

Maximum input tokens
Not collected

No public service contract or verified value for this original research checkpoint.

Maximum output
Not collected

No public service contract or verified value for this original research checkpoint.

Parameters
Not collected

No public service contract or verified value for this original research checkpoint.

Reasoning
Not collected

No public service contract or verified value for this original research checkpoint.

Tool calling
Not collected

No public service contract or verified value for this original research checkpoint.

Structured output
Not collected

No public service contract or verified value for this original research checkpoint.

Streaming
Not collected

No public service contract or verified value for this original research checkpoint.

Batch processing
Not collected

No public service contract or verified value for this original research checkpoint.

Fine-tuning
Not collected

No public service contract or verified value for this original research checkpoint.

Context caching
Not collected

No public service contract or verified value for this original research checkpoint.

Knowledge cutoff
Not collected

No public service contract or verified value for this original research checkpoint.

Release date
Not collected

No public service contract or verified value for this original research checkpoint.

Input modalities
Text, AudioSource for Input modalities (opens in a new tab)
Output modalities
AudioSource for Output modalities (opens in a new tab)
Languages
EnglishSource for Languages (opens in a new tab)

Language of the documented experiment; not an exhaustive claim of supported production languages.

Research publication
2023-01-05Source for Research publication (opens in a new tab)

First research-family disclosure on arXiv; reviewed specifications may reflect a later paper revision. Not a commercial or separate variant release date.

Endpoint availability
Research paper and prerecorded samples. No public full-model download or interactive inference service found in reviewed original materials.Source for Endpoint availability (opens in a new tab)
Voice cloning
SupportedSource for Voice cloning (opens in a new tab)

Zero-shot research use of a reference voice, not an available commercial API.

Reference audio
A three-second speaker recording is used in zero-shot evaluation.Source for Reference audio (opens in a new tab)

An experimental prompt choice, not an API input limit.

Architecture
AR and NAR Transformer models, each with 12 layers, 16 heads, 1,024-dimensional embeddings and 4,096-dimensional feed-forward layers.Source for Architecture (opens in a new tab)
Training dataset
LibriLight: 60,000 hours of English speech and approximately 7,000 speakers; transcripts produced with an ASR model trained on LibriSpeech.Source for Training dataset (opens in a new tab)
Training hardware
16 NVIDIA Tesla V100 32GB GPUs for 800,000 steps.Source for Training hardware (opens in a new tab)
Audio sample rates
24 kHz waveform codecSource for Audio sample rates (opens in a new tab)
Audio codec design
EnCodec residual vector quantization: eight codebooks of 1,024 entries; first-code AR generation followed by NAR prediction of the remaining codebooks.Source for Audio codec design (opens in a new tab)
Research speech tokenizer rate
75 codec frames per second at the documented 6 kbps setting.Source for Research speech tokenizer rate (opens in a new tab)
Tokenizer
Phoneme text promptsSource for Tokenizer (opens in a new tab)
Inference modes
Zero-shot cross-sentence synthesis, Speech continuationSource for Inference modes (opens in a new tab)
Voice controls
Research demonstrations preserve reference emotion and acoustic environment.Source for Voice controls (opens in a new tab)

Access & pricing

Prices belong to an endpoint and billing tier. A provider listing does not guarantee access for every account or region.

Microsoft Research (prerecorded samples)

Provider source (opens in a new tab)
API model ID
Not collected

No public API model selector.

Access
Read the paper and listen to prerecorded examples. No user-callable inference endpoint established by the reviewed project.
Regions
Not applicable to prerecorded research samples.

Endpoint pricing has not been collected. Check the provider for applicable rates.

Data & enterprise terms

No hosted customer inference or data-retention contract verified.

Caching

Caching can reuse earlier input processing. Availability, lifetime and billing depend on the endpoint. A cache hit still generates a new response.

01 / INITIAL INPUTProcess & retain

The system processes eligible input and retains reusable state.

02 / LATER REQUESTCheck available state

Reuse depends on matching input, session state and the cache policy.

03 / CACHE HITReuse & generate

Reusing state can reduce repeated work. Any charges follow the billing rules below.

Microsoft Research (prerecorded samples)

Not verified

No public service cache contract verified.

How it works
Not collected
Minimum reusable input
Not collected
Lifetime & refresh
Not collected
How to enable it
Not collected
Usage fields
Not collected
Billing rules
Not collected
Limits & data boundaries
Not collected

No verified cache rate for this endpoint. An empty rate is not a zero charge.

Caching reviewed 14 Oct 2026

What the measurements say

No benchmark observations have been added for this model. We do not turn a vendor description into a performance score.

Latency and throughput depend on the host, request size, reasoning settings and load. There is no measured speed or global usage figure in this record.

The record behind the model

Why it is included

Classification is an evidence-based inference for the original research checkpoint: the authors provide papers and prerecorded samples, but no full original model weights or callable service in the reviewed project materials. The Microsoft Hugging Face account lookup also returns no matching model. This does not classify independent implementations or separately released codec components as closed.

Classification evidence (opens in a new tab)

Lifecycle dates

Research family first published on arXiv5 Jan 2023Source (opens in a new tab)

Sources reviewed