Skip to content
Model infoModelsSnapshot: 15 Oct 2026

Hyper3D / Deemos / Media2Face / GNPFA

Media2Face

Research diffusion model that animates a 3D face from speech with text or image style guidance.

restrictedClosed weights3D processing
Official documentation (opens in a new tab)Reviewed 14 Oct 2026
9 of 24 specification fields collected
Context windowNot collectedAwaiting source review
Maximum outputNot collectedAwaiting source review
Input billingNot collectedMedia2Face research publication
Output billingNot collectedNo inferred price

Specifications

Provider-documented facts. “Not collected” means we have not verified that field yet, not that the feature is unsupported.

Context / input limit
Not collected
Maximum input tokens
Not collected
Maximum output
Not collected
Reasoning
Not collected
Tool calling
Not collected
Structured output
Not collected
Streaming
Not collected
Batch processing
Not collected
Fine-tuning
Not collected
Knowledge cutoff
Not collected
Release date
Not collected

Paper submission and revision dates are not weight-release dates.

Parameters
Not collected
Languages
Not collected
Context caching
Not collected
Architecture
Eight-layer transformer decoder, four attention heads, feature width 512; Wav2Vec2 audio features and CLIP style features condition diffusion in GNPFA expression space.Source for Architecture (opens in a new tab)
Input modalities
Speech audio, Text style prompt, Image style prompt, Optional keyframe expressionSource for Input modalities (opens in a new tab)
Output modalities
Expression latent sequence, Head-pose sequenceSource for Output modalities (opens in a new tab)
Training sequence length
200-frame windows at 30 fps.Source for Training sequence length (opens in a new tab)

Frame windows are not text-token context limits; overlapping windows handle longer audio subject to memory.

Training hardware
36 hours on NVIDIA RTX 3090 with AdamW; 500 training noise steps and cosine schedule.Source for Training hardware (opens in a new tab)
Published training data
M2F-D: 60.6 hours; includes MEAD, CREMA-D, RAVDESS, HDTF, Acappella, additional multilingual video and captured head movements. Two hours held out for testing.Source for Published training data (opens in a new tab)
Frame rates (fps)
30 fps animation; author reports over 300 fps offline processing and 30 fps real-time on RTX 3090.Source for Frame rates (fps) (opens in a new tab)

Hardware-specific reported throughput, not a hosted endpoint SLA.

Generation modes
DDIM sampling, keyframe inpainting and per-frame style guidance; overlapping-window denoising.Source for Generation modes (opens in a new tab)
Endpoint availability
Research edition; code and dataset are listed as upcoming by the official project.Source for Endpoint availability (opens in a new tab)
Usage metering
Not collected

No current hosted API price or callable checkpoint selector verified.

Access & pricing

Prices belong to an endpoint and billing tier. A provider listing does not guarantee access for every account or region.

Media2Face research publication

Provider source (opens in a new tab)
API model ID
Not collected

No public native selector for this research edition verified.

Access
Paper and project page only; no original weight download or current callable API verified.
Regions
Not established

Endpoint pricing has not been collected. Check the provider for applicable rates.

Data & enterprise terms

No commercial retention or training-use contract assigned to a research publication.

Caching

Caching can reuse earlier input processing. Availability, lifetime and billing depend on the endpoint. A cache hit still generates a new response.

01 / INITIAL INPUTProcess & retain

The system processes eligible input and retains reusable state.

02 / LATER REQUESTCheck available state

Reuse depends on matching input, session state and the cache policy.

03 / CACHE HITReuse & generate

Reusing state can reduce repeated work. Any charges follow the billing rules below.

Media2Face research publication

Not verified

No model-specific caching contract verified.

How it works
Not collected

No model-specific caching contract verified.

Minimum reusable input
Not collected

No model-specific caching contract verified.

Lifetime & refresh
Not collected

No model-specific caching contract verified.

How to enable it
Not collected

No model-specific caching contract verified.

Usage fields
Not collected

No model-specific caching contract verified.

Billing rules
Not collected

No model-specific caching contract verified.

Limits & data boundaries
Not collected

No model-specific caching contract verified.

No verified cache rate for this endpoint. An empty rate is not a zero charge.

Caching reviewed 14 Oct 2026

What the measurements say

Lip Vertex Error

Media2Face paper authors · author-reported
10.44mm

Paper Table 2, full model. Two-hour M2F-D holdout with similar composition to training data. Lower is better. Maximum L2 error over lip vertices per frame. Not an independently reproduced benchmark. Baselines without head pose omit BA; differing facial topology prevents vertex metrics for SadTalker.

Evaluation source (opens in a new tab)

Upper Face Dynamics Deviation

Media2Face paper authors · author-reported
12.21×10^-5 m

Paper Table 2, full model. Two-hour M2F-D holdout with similar composition to training data. Lower is better. Deviation of upper-face motion standard deviations. Not an independently reproduced benchmark. Baselines without head pose omit BA; differing facial topology prevents vertex metrics for SadTalker.

Evaluation source (opens in a new tab)

Beat Alignment

Media2Face paper authors · author-reported
0.254score

Paper Table 2, full model. Two-hour M2F-D holdout with similar composition to training data. Higher is better. Audio/head-motion beat synchronization. Not an independently reproduced benchmark. Baselines without head pose omit BA; differing facial topology prevents vertex metrics for SadTalker.

Evaluation source (opens in a new tab)

Latency and throughput depend on the host, request size, reasoning settings and load. There is no measured speed or global usage figure in this record.

The record behind the model

Why it is included

Qualified unreleased-weight research classification: the official project labels code and data upcoming, and its links lead back to the project. No original weight release was established in reviewed sources. This is not a claim of permanent proprietary licensing; publication alone does not release weights.

Classification evidence (opens in a new tab)

Lifecycle dates

Research paper first submitted28 Jan 2024Source (opens in a new tab)
Research paper revised30 Jan 2024Source (opens in a new tab)

Sources reviewed