BloombergGPT

Provider-documented facts. “Not collected” means we have not verified that field yet, not that the feature is unsupported.

Context / input limit

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Maximum input tokens

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Maximum output

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Parameters

50.6BSource for Parameters (opens in a new tab)

Release date

2023-03-30Source for Release date (opens in a new tab)

First research-paper publication, not product or weight availability.

Reasoning

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Tool calling

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Structured output

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Streaming

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Batch processing

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Fine-tuning

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Context caching

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Knowledge cutoff

Not collected

Exact model-specific contract not established in the reviewed sources. Unknown is not unsupported.

Input modalities

TextSource for Input modalities (opens in a new tab)

Output modalities

TextSource for Output modalities (opens in a new tab)

Architecture

Dense decoder-only transformer: 70 layers, 40 attention heads, 7,680 hidden dimensions; ALiBi positional biases.Source for Architecture (opens in a new tab)

Base model

Trained from scratch; BLOOM influenced the architecture, not a claimed inherited checkpoint.Source for Base model (opens in a new tab)

Training context length

2,048Source for Training context length (opens in a new tab)

Training sequence length. The report permits longer ALiBi inference sequences without publishing a service limit.

Pretraining tokens

569,000,000,000Source for Pretraining tokens (opens in a new tab)

Tokens consumed by the selected checkpoint; the full candidate corpus was larger.

Training dataset

About 709B available tokens: approximately 363B financial and 345B general-domain tokens (rounded figures).Source for Training dataset (opens in a new tab)

Published training method

Causal language modeling; selected step 139,200, while the run ended at step 146,000.Source for Published training method (opens in a new tab)

Tokenizer

131,072-token unigram vocabulary with multi-word tokens.Source for Tokenizer (opens in a new tab)

Training hardware

512 NVIDIA A100 40GB GPUs across 64 AWS p4d.24xlarge instances.Source for Training hardware (opens in a new tab)

Prices belong to an endpoint and billing tier. A provider listing does not guarantee access for every account or region.

Caching can reuse earlier input processing. Availability, lifetime and billing depend on the endpoint. A cache hit still generates a new response.

No benchmark observations have been added for this model. We do not turn a vendor description into a performance score.

Latency and throughput depend on the host, request size, reasoning settings and load. There is no measured speed or global usage figure in this record.

Why it is included

Section 8.2 explicitly states that the authors do not release the model because exposing its weights risks disclosing proprietary FinPile data. This is a private research model, not a verified public API.

Lifecycle dates

Sources reviewed