DeepInfra

DeepInfra was founded in 2023 and is headquartered in Palo Alto, California, USA. The company is led by Nikola Borisov as CEO and co-founder. The company provides a cloud AI inference platform that hosts open source machine learning models through serverless APIs, supporting multiple tasks such as text generation, speech processing, and document understanding.

The company completed an $8 million seed round in November 2023, led by A. Capital and Felicis. An $18 million Series A round of financing was completed in April 2025, led by Felicis and with participation from Georges Harik. The company's infrastructure is located in US data centers, is SOC 2 and ISO 27001 certified, and has a zero data retention policy.

The company supports custom models deployed onto its infrastructure and also offers on-demand GPU instance rentals, such as the DGX B200.

Core services and products

DeepInfra's platform provides inference services for over 100 open source models, supporting the following task categories:

text generation

text to speech

speech recognition

OCR and document processing

embedded model

Text to image (partially supported)

The platform runs in a US data center, supports long context windows, multiple quantization formats (such as fp4, fp8, bfloat16), and provides real-time performance indicators, including token generation speed, first token time and request processing volume.

Users can access hosted models through the API, or deploy custom models. The company also supports API access for the Gemini family of models.

Available model categories and examples

text generation model

deepseek-ai/DeepSeek-V3.2 (fp4 quantization, 160k context window, supports sparse attention mechanism)

deepseek-ai/DeepSeek-V3.1 (fp4 quantization, 160k context window, supports thinking mode and non-thinking mode)

deepseek-ai/DeepSeek-V3.1-Terminus (fp4 quantization, 160k context windows, supports structured tool calls and code proxies)

Qwen/Qwen3-Next (fp8 quantized, 256k context window)

Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo (fp4 quantization, 256k context windows, supports proxy encoding tasks)

nvidia/Nemotron-3-Nano-30B-A3B (bfloat16 quantization, 256k context window, mixed MoE and Mamba architecture)

MiniMaxAI/MiniMax-M2 (fp8 quantization, 256k context window, 10B activation parameters)

moonshotai/Kimi-K2-Instruct-0905 (fp4 quantization, 128k context window, 1T parameter MoE model)

openai/gpt-oss-120b (fp4 quantization, 128k context window, supports native tool calls)

openai/gpt-oss-20b (fp4 quantization, 128k context window, supports low-latency inference)

text-to-speech model

ResembleAI/chatterbox-turbo (350M parameters, supports language-like tags such as [cough], [laugh], suitable for low-latency voice agents)

speech recognition model

mistralai/Voxtral-Small-24B-2507 (bf16 quantification, 32k context window, supports transcription and translation)

mistralai/Voxtral-Mini-3B-2507 (bf16 quantification, 32k context window, supports transcription and translation)

OCR and document processing model

The following models are integrated in the text generation category for document understanding tasks:

deepseek-ai/DeepSeek-OCR (bfloat16 quantization, 8k context window, supports high compression ratio OCR)

allenai/olmOCR-2-7B-1025 (fp8 quantization, 16k context window, supports complex PDF structured conversion)

PaddlePaddle/PaddleOCR-VL-0.9B (bfloat16 quantization, 16k context window, supports text, table, formula recognition in 109 languages)

Model support feature summary

Category Examples Model Number Common Context Window Quantization Options Main Tasks

Text generation over 20 32k–256kfp4, fp8, bfloat16 inference, encoding, tool calls

Text-to-speech1 N/A N/A Low-latency speech generation

Speech recognition 2 32kbf16 transcription, translation, audio understanding

OCR/document processing 3 8k–16kfp8, bfloat16PDF parsing, table extraction, multi-language support