AIsaAIsaStudio 中文 Open AIsa Studio

LiquidAI releases LFM3-Omni-2B, an any-to-any omni-modal model for text, image, video and speech

Original title: LiquidAI/LFM3-Omni-2B (any-to-any)

LiquidAI released LFM3-Omni-2B, the first omni-modal model in its LFM3 family: a single end-to-end model that understands text, images, video and audio and answers in text and speech. The model has 2.17B total parameters, a 32K token context length, 24 kHz speech output, and processes video at 2 fps with 4 to 384 frames by default. Designed for on-device use, it replaces separate ASR, vision-language, LLM and TTS models and can jointly understand videos together with their soundtracks.

Why it matters

The README provides parameter count, input/output specs and its on-device positioning, helping readers assess whether it can replace multiple separate models.

Read the original ↗

The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.

Original article

Hugging Face 热门模型第 27 名:LiquidAI/LFM3-Omni-2B,162 赞,近期下载 714 次。 标签:transformers, safetensors, lfm3_omni, image-text-to-text, liquid, lfm3, lfm3-omni, omni, audio, vision, speech-to-speech, edge

README:

<img src="https://huggingface.co/LiquidAI/LFM3-1.7B/resolve/main/assets/liquid-ai-banner.webp" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: block; margin: 0;" />

API
Try
Docs
Blog

LFM3-Omni-2B

LFM3-Omni-2B is Liquid AI's first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video and audio, and answers in text and speech. It is built for on-device use: one model replaces a separate ASR model, vision-language model, LLM and TTS model.

  • Any input, text or speech output: ask about a photo by voice, transcribe or translate speech, chat out loud, or ask questions about a video and its soundtrack together.
  • Watches and listens together: for videos with sound, the model sees and hears them at the same time, so it can answer questions that need both.
  • Small enough for devices: at 2.2B parameters, it is designed to run efficiently on phones, laptops and other edge hardware.

Find more information about the LFM3 family in our release blog post.

Model details

The LFM3 family includes base, text, vision, audio, omni, and decision models.

Property Specification

Checkpoint Post-trained omni-modal model

Parameters 2.17B total

Architecture LFM3 backbone, LFM3 vision encoder, LFM3 causal audio encoder, LiquidCodec audio tokenizer.

Inputs Text, images, video with or without its soundtrack, and 16 kHz audio. Images and video are processed at native resolution; each 2×2 block of patches becomes one language token. Video is sampled at 2 fps, with 4 to 384 frames by default.

Outputs Text, 24 kHz speech, or interleaved text and speech

Context length 32K tokens (input + output)

Vocabulary <td style="border: 0; padding: 0.55em 0.7

The original belongs to Hugging Face 热门模型 and its authors and is reproduced here for reading. To have it removed, write to support@aisa.one.

Other coverage 5