Skip to content
K = 0.731
← All topics

Data and training

The training side: building datasets, synthetic data, pre- and post-training methods, compute and training cost.

48 picksLast 30 days 19 itemsCollected 227 items

New

Picks about Data and training

10/16Fri
  1. Hugging Face Blog82

    NVIDIA and AI4Bharat Introduce Nemotron-pretraining-Indic-v1 and an Open Data Generation Pipeline

    NVIDIA and AI4Bharat at IIT Madras introduce Nemotron-pretraining-Indic-v1, approximately 2.8 trillion tokens across 22 Indian languages, releasing soon alongside its open, GPU-accelerated generation pipeline. The corpus includes native-script text for all 22 languages and romanized text for twelve, translated from curated English sources with quality checks and source tracing. An Indic continued-pretraining stage improved five reported benchmarks, but combined synthetic and natural data, so the comparison does not isolate this corpus’s contribution.

    Why it matters: The workflow offers a reusable approach to multilingual data generation by separating translation, quality assessment, repair, and script conversion while preserving links to source documents.

10/13Tue
  1. MIT Technology Review · AI76

    AI labs are buying data from dead startups to train their models

    AI labs are buying defunct startups’ code and workplace archives through companies such as SimpleClosure and Sunset to train models and build environments where AI agents practice tasks. SimpleClosure’s Asset Hub has handled over 150 deals, while Google bought Spirit Airlines’ decades of data for $10 million this past August. The market raises questions about employee privacy, consent, and pricing, and no company has released tests comparing training results from real-world archives with alternatives.

    Why it matters: The sales expose a gap between who generates workplace data and who receives its resale proceeds, with employees’ private information entering transactions that typically benefit creditors and investors.

  2. Microsoft Research65

    Simulating student learning for better AI tutoring

    Microsoft Research introduces StudentSim, a research system that trains personalized student simulators from publicly available, de-identified learner records to evaluate and improve AI tutors. StudentSimEval measures behavioral fidelity and guidance responsiveness across chess, second-language English writing, and mathematics, where StudentSim outperforms leading baselines. In a chess tutor study, training with StudentSim feedback achieves 90.5% accuracy versus 75.7% without reinforcement learning and 71.6% with GPT-5.4 feedback; real learner studies remain essential, and the paper and code are publicly available.

    Why it matters: Evaluating both a simulated learner’s starting ability and response to guidance offers a concrete method for assessing tutor feedback beyond the correctness of an explanation.

10/11Sun
  1. 量子位88

    OpenAI Posts Two Kakeya Conjecture Manuscripts, Extending Work to Four Dimensions

    On October 7, OpenAI released 722 mathematics manuscripts, including two on the Kakeya conjecture: a 97-page paper claiming a three-dimensional maximal-function result stronger than Wang and Zahl's, and a 175-page paper claiming the four-dimensional Hausdorff dimension case. Both are attributed to OpenAI, are not Lean-formalized or peer-reviewed, and OpenAI's README says some unformalized results may be flawed.

    Why it matters: The piece traces how two OpenAI manuscripts extend Wang and Zahl's three-dimensional Kakeya work to a stronger maximal-function version and to four dimensions, while noting the proofs lack peer review.

10/10Sat
  1. The Verge · AI88

    OpenAI Releases Nearly 400 Mathematical Results at Once, Mathematicians Say It Will Take Years to Digest

    This week, OpenAI suddenly released nearly 400 AI-generated mathematical results, spread across more than 700 manuscripts and covering directions including combinatorics, geometry, number theory, theoretical computer science, algebra, topology, probability and statistical mechanics, and mathematical physics.

    Why it matters: Through first-hand reactions from dozens of mathematicians, this presents the impact of OpenAI's one-time release of nearly 400 results on the academic ecosystem and researchers' career paths.

10/8Thu
  1. Microsoft Research72

    Microsoft Research open-sources Agent Lightning v1.0, a 3,500-line lightweight agent RL framework for training with real harnesses

    Microsoft Research open-sourced Agent Lightning v1.0, a lightweight agent reinforcement-learning framework of about 3,500 lines. Its Harnessed Agentic RL approach lets the same agent harness used in deployment participate directly in reinforcement learning, without reimplementing the agent inside a training framework.

    Why it matters: The source gives the framework's code size, Kubernetes support, and training recipe, helping readers assess the cost of bringing a real harness into RL.

10/7Wed
  1. Hugging Face Blog62

    Nemotron fine-tuning reaches gold-medal level at both IOI 2026 and IMO 2026

    The NVIDIA team started with Nemotron 3 and used supervised fine-tuning, reinforcement learning, and a generate-verify-improve reasoning process to build specialized models that reached gold-medal thresholds at both IOI 2026 and IMO 2026.

    Why it matters: The source gives a complete recipe for fine-tuning Nemotron 3 to gold-medal-level results at IOI and IMO, which readers can use to reproduce its data and reasoning workflow.

  2. Latent Space88

    OpenAI's internal model releases 722 math papers, claiming to solve 90 of 500 open problems

    OpenAI published mathematical results from an unreleased internal frontier model through a blog, GitHub repository, and posts. The release includes 722 manuscripts grouped into 372 related result families, drawn from an evaluation of about 4,000 research questions. Each result used an average of about three hours of ChatGPT Pro reasoning compute.

    Why it matters: OpenAI is producing mathematical results in bulk with an internal model, giving readers a view of the scale, compute cost, and outside criticism surrounding the work.

10/3Sat
  1. AI at Meta · X66

    Meta publishes six papers on mathematicians collaborating with Muse Spark

    Meta said that after AI models achieved gold-medal-level performance in five math, physics, and chemistry competitions, it had mathematicians work with Muse Spark 1.1 and 1.2 in Thinking Mode through the meta.ai chat interface to solve open problems with no existing solution path and no custom research scaffolding.

    Why it matters: Meta made six papers from mathematicians' collaboration with Muse Spark public and described the division of work between people and AI and the review process.

  2. Hugging Face Blog62

    Ai2 open-sources AstaBrief 8B, a scientific report-generation model

    Ai2 open-sourced AstaBrief 8B, a model that turns research questions and retrieved literature excerpts into reports with citations. It was trained from Qwen3-8B using SFT and DPO and is now available as Asta's Fast mode.

    Why it matters: The source describes AstaBrief's training-data construction and citation-filtering details, which can be adapted to deploy other long-form generation models.

10/2Fri
  1. Hugging Face · X74

    Hugging Face publishes multi-harness RL guide: the same model scores 62% versus 33% with different harnesses

    The Hugging Face team published a multi-harness RL guide, showing that the same model with identical weights can score 62% with one agent harness and 33% with another.

    Why it matters: Scores for the same model differ by nearly half across agent harnesses. This open guide gives a multi-harness RL training path without changing the harness, along with measured comparisons.

10/1Thu
  1. Hugging Face Blog62

    AI2 releases Olmo-core 3, open and scalable training infrastructure for large MoE models

    AI2 released Olmo-core 3, a major upgrade to its large language model development framework. It redesigns the open MoE training system to scale MoE training to trillions of parameters while maintaining compute efficiency.

    Why it matters: The source provides throughput and memory comparisons for the MoE training stack, helping readers assess changes in the cost of large-scale MoE training.

9/30Wed
  1. Google DeepMind62

    Google DeepMind launches SynthID Bio to embed watermarks in AI-generated proteins

    Google DeepMind launched SynthID Bio, bringing watermarking to synthetic biology. It embeds imperceptible signatures directly into biological code so that generated content can be verified both in digital models and in synthesized physical proteins, while preserving their biological function.

    Why it matters: SynthID watermarking has expanded from images to synthetic biology, showing how AI-generated proteins can be marked and verified.

  2. NVIDIA Blog62

    NVIDIA and CoreWeave put Vera Rubin NVL72 into production; Cognition is first customer

    CoreWeave announced that NVIDIA Vera Rubin NVL72 systems and Spectrum-X 102.4T Ethernet are available on CoreWeave Cloud, making it one of the first cloud providers to deliver the platform. Cognition is the first customer to run Vera Rubin in production.

    Why it matters: CoreWeave launched Vera Rubin NVL72 and published measured Devin throughput, offering a view of compute costs for agentic coding workloads.

9/29Tue
  1. Hugging Face Blog62

    NVIDIA releases Kumo Tabular, a foundation model for tabular prediction

    NVIDIA has released Kumo Tabular, an open-source tabular foundation model that predicts labels for new rows in a single forward pass from labeled table rows, supporting classification and regression without training, tuning, or feature engineering. The model comes in three sizes from 28M to 215M parameters, was pretrained entirely on synthetic tables, is commercially usable under the OpenMDW-1.1 license, and ranks first on four benchmarks: TabArena, BeyondArena, TALENT, and ScoringBench.

    Why it matters: The source reports model sizes, training-data provenance, and rankings on four benchmarks, helping readers assess the accuracy and efficiency tradeoffs of tabular foundation models.

  2. Microsoft Research62

    Microsoft Research releases Quine, an AI research system for biological complexity

    Microsoft Research has released Quine, an AI research system designed for biological complexity. It combines a biological world model with an interactive harness connecting models, scientific tools, literature, and experimentalists. In work with the Broad Institute on pancreatic ductal adenocarcinoma, it predicted and prioritized thousands of compounds; the top-ranked compound produced the largest classical-to-ground-state transition in wet-lab experiments. The process from narrowing the search space to selecting a candidate for validation took one weekend.

    Why it matters: Microsoft Research describes Quine's multimodal biological world model and experimental loop, including wet-lab validation in pancreatic cancer compound screening.

  3. 量子位82

    AMD to acquire Fei-Fei Li's World Labs for $8.2 billion in an all-stock deal; Li to become chief scientist

    AMD announced an all-stock acquisition of Fei-Fei Li's World Labs for $8.2 billion. The deal is expected to close by the end of 2026, subject to regulatory approval. After closing, Li will join AMD as executive vice president and chief scientist, reporting directly to Lisa Su; co-founders Justin Johnson and Ben Mildenhall will continue leading the team as it joins AMD.

    Why it matters: The full deal details and the companies' year-and-a-half collaboration are laid out, helping readers understand why a chipmaker would acquire a spatial-intelligence team.

9/24Thu
  1. NVIDIA Blog60

    NVIDIA, Google DeepMind, and partners open predicted protein-complex structures for more than 2,800 viruses

    NVIDIA, Google DeepMind, EMBL-EBI, and other organizations have made predicted 3D structures of protein complexes from more than 2,800 viruses freely available to any scientist through the AlphaFold Database.

    Why it matters: NVIDIA, Google DeepMind, and collaborators have opened predicted protein-complex structures for more than 2,800 viruses, showing a concrete approach to open science for pandemic preparedness.

9/22Tue
  1. Latent Space78

    Xiaomi releases MiMo-V2.6-Pro 1T-A42B, an open-source multimodal model trained for $3 million

    Xiaomi has released the MiMo-V2.6 family, including the multimodal MiMo-V2.6-Pro and MiMo-V2.6-Flash, and introduced MiMo-V2.6-Pro-UltraSpeed, which delivers up to 20× faster output.

    Why it matters: Xiaomi's first entry in the frontier open-model lineup, with open RL environments and training recipes, provides a concrete example of post-training cost strategies.

9/16Wed
  1. NVIDIA Blog62

    NVIDIA Vera Rubin NVL72 makes its MLPerf Inference v6.1 debut with leading performance

    NVIDIA submitted preview results for Vera Rubin NVL72 for the first time in MLPerf Inference v6.1, reporting up to 3.7× the throughput of GB300 NVL72 on Qwen3-VL and up to 2.5× on DeepSeek-R1.

    Why it matters: MLPerf Inference v6.1 reports throughput, scaling efficiency, and software-optimization gains for Vera Rubin NVL72 versus GB300 NVL72, offering a basis for judging inference infrastructure value.