Skip to content
K = 0.731
2026.10.17 · Sat

#多模态

#多模态

Models

New models and versions, open weights, and changes in capability and pricing, with their evaluations.

10/16Fri
  1. Hugging Face Blog87

    Meta releases Muse Spark Open, an open-weight multimodal mixture-of-experts model

    Meta’s Muse Spark Open is an open-weight mixture-of-experts model with approximately 816 billion parameters, around 42 billion active per token, and a context window of up to one million tokens. Trained for agentic coding, it accepts text, images, audio, and video and generates text. Hugging Face provides inference and deployment recipes, reports its own quantization tests, and demonstrates attention-layer LoRA fine-tuning with TRL while keeping the experts frozen.

    Why it matters: The deployment recipes clarify how sparse expert activation reduces per-token computation while retaining substantial memory requirements, helping developers assess hardware needs across checkpoint formats and serving frameworks.

  2. The Decoder81

    Nvidia releases Nemotron 3.5 Super VL for continuously running enterprise AI agents

    Nvidia released Nemotron 3.5 Super VL, an open multimodal model with 120 billion parameters and 12 billion active per token, for continuously running enterprise agents. Nvidia says it approaches Nemotron 3 Ultra’s accuracy on specified agent tasks with about one-fifth the parameters, and is releasing weights, datasets, and training recipes. The accompanying Nemotron 3.5 Bolt 74B-A9B 24GB targets local agents, while Bolt E4B 4GB is planned for later this month.

    Why it matters: The deployment details clarify how quantization, caching, and hardware requirements shape the practical tradeoffs of running continuous enterprise agents on company infrastructure.

10/15Thu
  1. NVIDIA Blog81

    NVIDIA Expands Nemotron Family With Super VL and Bolt Open Models

    NVIDIA released Nemotron 3.5 Super VL, an open 120-billion-parameter model for multimodal agents, and Nemotron 3.5 Bolt 74B-A9B 24GB for local agentic workloads; Bolt E4B 4GB is coming Oct. 23. Super VL targets tool use, business workflows and visual document understanding, with open weights, datasets and training recipes. Bolt 74B-A9B uses native 2-bit quantization to fit in 24GB of memory, while Bolt E4B targets physical and conversational agents.

    Why it matters: Open weights, datasets and training recipes give organizations a path to adapting agents to their own workflows while meeting local deployment and data residency requirements.

  2. Hugging Face Blog74

    Apertus 1.5 Release Notes Detail Multimodal Understanding, Thinking Mode, and Longer Context

    Project Apertus released Apertus 1.5 in 8B and 70B sizes, adding image and audio understanding, optional thinking, a 262,144-token context window, and improved instruction-following and tool use. The weights were released on Hugging Face on July 24, 2026 under Apache 2.0; the models generate only text, and spoken-language processing remains experimental. The team also publishes training data, recipes, intermediate checkpoints, and reproducible evaluation records.

    Why it matters: Published intermediate checkpoints, training mixtures, and evaluation settings provide a concrete basis for investigating how individual training stages change model behavior and for reproducing reported results.

  3. 量子位82

    Skywork AI releases open-source Matrix-Game 4.0 for real-time gaming on one RTX 5090

    Skywork AI released Matrix-Game 4.0, an open-source game world model that reportedly runs at 480p and 60 FPS on one RTX 5090, with input response under 0.15 seconds and peak VRAM usage of approximately 18.5GiB. A game engine maintains state and executes rules, while a video model generates visuals from structural frames and dynamic captions. Model weights and code are available through https://huggingface.co/Skywork/Matrix-Game-4.0 and https://github.com/SkyworkAI/Matrix-Game.

    Why it matters: The engine and video model division offers a transferable design approach for keeping game rules deterministic while using generative models to render interactive worlds.

10/14Wed
  1. Google DeepMind · X73

    Gemini 4 Argon taught a robot to play cornhole just by watching a video

    Gemini 4 Argon taught a robot to play cornhole just by watching a video. It mapped the physics, coded its own simulation, and taught @Apptronik's Apollo humanoid to sink 7 throws in a row.

    Why it matters: The demonstration connects video observation, physics modeling, and a model-written simulation in a concrete example of teaching a humanoid robot a physical task.

  2. Google DeepMind94

    Google DeepMind launches Gemini 4 Argon for complex reasoning and agentic tasks

    Google DeepMind has launched Gemini 4 Argon globally through its API and selected Google products, with access depending on subscription or service tier. Introductory pricing is $2 per million input tokens and $10 per million output tokens, with cached input tokens at 95% off and a 1M unified input-output token limit through the Gemini API. Google reports improvements across coding, professional workflows, scientific research, and robotics, alongside stronger agent safeguards and prompt injection resistance.

    Why it matters: Google’s reported gains in code submissions and reductions in rollbacks connect Argon’s engineering capabilities to delivery speed and code stability in internal workflows.

  3. Demis Hassabis · X83

    Gemini 4 Argon rolls out to Google Ultra and Pro subscribers

    Gemini 4 Argon is rolling out today to Google Ultra and Pro subscribers in @GeminiApp and Search, and to Ultra subscribers in @antigravity. Demis Hassabis praises its knowledge work and coding capabilities, sharing a wizard chess game they asked it to create as an example of its physics understanding and 3D modeling capabilities that reminds him of Battlechess on the Amiga. More information and full availability: https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-argon/

    Why it matters: The rollout details distinguish access across GeminiApp, Search, and antigravity, giving subscribers concrete context for where they can try Argon under their current plans.

10/13Tue
  1. 宝玉 · X82

    Meta launches Muse Wonder with developer access and conflicting consumer availability statements

    Meta released Muse Wonder and Muse Wonder Full on October 12, according to Baoyu’s account of the announcement. Developers can access Muse Wonder through Meta Model API or Muse Code, while Full is available to Muse Code subscribers; Meta’s blog and Zuckerberg’s X post differ on consumer availability. The models support a 1 million Token context window and multimodal input with text output, while Meta’s reported benchmarks show strengths in sustained agentic and coding tasks alongside weaker results elsewhere.

    Why it matters: The comparison distinguishes stronger results on sustained agentic and coding tasks from broader intelligence rankings, while identifying Meta’s internal benchmarks and limitations in how competing models were tested.

10/8Thu
  1. Hugging Face Blog62

    Liquid AI open-sources d1-3B and d1-omni-600M on-device decision models

    Liquid AI released two open-weight models in its d1 decision-model family: d1-3B and the experimental d1-omni-600M.

    Why it matters: Liquid AI open-sourced two decision models and reported on-device latency and results across seven benchmarks, providing evidence on the use of small models for structured decisions.

10/7Wed
  1. Google DeepMind71

    Google DeepMind releases EmbeddingGemma 2, a lightweight open multimodal embedding model

    Google DeepMind released EmbeddingGemma 2, a 740M-parameter open model based on the Gemma 4 architecture and licensed under Apache 2.0. It maps text, images, audio, and video into a shared embedding space.

    Why it matters: EmbeddingGemma 2 maps text, images, audio, and video into one embedding space and reports concrete on-device memory and vector truncation figures, helping assess local multimodal retrieval.

10/6Tue
  1. Mistral AI80

    Mistral launches public preview of Mistral Large 4, a 1-trillion-parameter multimodal model

    Mistral released a public preview of Mistral Large 4, a natively multimodal model with 1 trillion parameters and 49 billion active parameters. Its weights will be opened at the end of the month.

    Why it matters: Mistral outlined the open-weight release schedule for its 1-trillion-parameter multimodal model and its investment in European compute, allowing comparison with mainstream open models.

10/1Thu
  1. Google DeepMind88

    Google DeepMind releases frontier model Gemini 4 Argon

    Google DeepMind released frontier model Gemini 4 Argon, initially making it available through the Fairwind Program to trusted cybersecurity defenders before gradually expanding access. Pricing starts at $2 per million input tokens and $10 per million output tokens.

    Why it matters: Gemini 4 Argon raises the maximum output to 1M tokens and reports multiple benchmark results and internal deployment data, offering a view of its limits on long-running workflows.

9/30Wed
  1. 雷锋网78

    First hands-on tests of GPT-6 Astra: build Manhattan in a week, but long tasks still fail

    OpenAI released its next-generation flagship model GPT-6 Astra at 1 a.m. Beijing time on September 4. Leiphone compiled first-hand tests from users who received access early.

    Why it matters: Early user tests show GPT-6 Astra can check its own work on long tasks but still needs human oversight, revealing its capability limits.

9/29Tue
  1. Microsoft Research62

    Microsoft Research releases Quine, an AI research system for biological complexity

    Microsoft Research has released Quine, an AI research system designed for biological complexity. It combines a biological world model with an interactive harness connecting models, scientific tools, literature, and experimentalists. In work with the Broad Institute on pancreatic ductal adenocarcinoma, it predicted and prioritized thousands of compounds; the top-ranked compound produced the largest classical-to-ground-state transition in wet-lab experiments. The process from narrowing the search space to selecting a candidate for validation took one weekend.

    Why it matters: Microsoft Research describes Quine's multimodal biological world model and experimental loop, including wet-lab validation in pancreatic cancer compound screening.

  2. Latent Space78

    AINews: Opus 5.5 excels at generating explainer videos

    Latent Space's AINews roundup of developments from September 24–25 says community feedback was positive after Opus 5.5 launched that week, especially for generating explainer videos with code.

    Why it matters: The weekly roundup offers a quick look at model and agent-infrastructure developments, including community feedback on Opus 5.5's code-generated videos and how vendors position their new models.

9/25Fri
  1. Google DeepMind71

    Google DeepMind releases Gemini 3.8 Live with Live Avatar

    Google DeepMind has released Gemini 3.8 Live with Live Avatar, combining real-time video generation and voice to give Gemini's conversational model a near-real-time visual avatar, with accurate lip-sync, natural expressions, and smooth turn-taking.

    Why it matters: Gemini 3.8 Live adds real-time video avatars, giving readers a view of how multimodal conversation is being applied in enterprise settings and where the feature is available.

9/23Wed
  1. Google DeepMind71

    Google DeepMind releases Gemini 3.8 Flash TTS and Flash-Lite TTS

    Google DeepMind has released two speech-generation models: Gemini 3.8 Flash TTS for character design and fine-grained creative control, and Gemini 3.8 Flash-Lite TTS for high-volume voiceovers and voice agents.

    Why it matters: The article outlines the models' voice cloning, line-by-line direction, and benchmark results, helping readers assess their suitability for dubbing and podcast production.

  2. Qwen · X71

    Alibaba releases Qwen-Audio-3.1 family, including TTS-Next and ASR-Next

    Alibaba has released Qwen-Audio-3.1 with upgrades to ASR, TTS, and Realtime, plus two new models, TTS-Next and ASR-Next. Five models now cover audio understanding, generation, interaction, and creation.

    Why it matters: The article lists the capabilities of all five audio models and their price reductions, helping readers assess voice-stack choices and cost changes.

9/10Thu
  1. DeepSeek · X68

    DeepSeek releases DeepSeek-V4.1-Flash

    DeepSeek has released DeepSeek-V4.1-Flash, describing it as smarter, faster, and more efficient. It is the smallest model in a new architecture family, with native visual understanding, and is designed for stronger capabilities, faster inference, and higher throughput, with the architecture able to scale to larger models.

    Why it matters: As the smallest model in DeepSeek's new architecture family, its native visual understanding and higher inference throughput offer a view of its lightweight multimodal positioning.