W26 AI Data Intelligence

One-line Summary

AllenAI released the BenchMIRT trio in one go on 2026-09-01 [P0], BAAI rolled out embodied and tool-safety data in a concentrated push from 2026-09-02 to 2026-09-03 [P0], and OpenAI brought code, cybersecurity, and "verifiability" to the forefront on 2026-09-03 [P0]. This week’s strongest data demand signal: robot collaboration / embodied planning data.

Key Findings

This week's 5 high commercial value findings

P0 AllenAI released the BenchMIRT trio in one go on 2026-09-01 [P0]

On 2026-09-01, allenai/BenchMIRT-item-statistics had 352 downloads and 1 like; allenai/BenchMIRT-model-statistics had 342 downloads and 0 likes; allenai/BenchMIRT had 327 downloads and 0 likes. The companion allenai/BenchMIRT-eval-data was released on 2026-08-25 and had 39 downloads. The official description explicitly states that this benchmark is used to measure LLM latent safety and general reasoning scores, and that the data includes prompt and model response statistics.

Business implication → This shows that the evaluation itself is becoming a high-value data asset, not a side product. BenchMIRT’s value is not in raw text volume, but in expert-defined scoring dimensions, task design, and statistical interpretation, all of which depend on human judgment. For Knowlyr, the opportunity is not only to collect tasks, but to build a reusable judgment framework, especially for safety, reasoning, and preference-alignment evaluation data.

P0 BAAI launched embodied and tool-safety data in a concentrated push from 2026-09-02 to 2026-09-03 [P0]

On 2026-09-02, BAAI/ToolPrivBench had 53 downloads. On 2026-09-03, BAAI/Orchestra-Bench had 186 downloads, and the description states that it includes 12,000 samples, collaborative planning for three robots, three-view inputs per sample, and three coordinated subtasks; BAAI/Discoverse-L had 129 downloads; BAAI/MobileVLA-CoT had 56 downloads.

Business implication → Chinese research institutions are shifting their data focus from single-task recognition to judgment scenarios that are much closer to real-world deployment, such as collaborative planning, tool permissions, and action chains. The rarest asset here is human judgment: which action is reasonable, which tool permission is overreaching, and which collaboration plan is safe. For data service providers, this wave is not about volume, but about scenario design, risk-boundary definition, and expert review capability.

P1 OpenAI put code, cybersecurity, and "verifiability" in the spotlight on 2026-09-03 [P0]

On 2026-09-03, OpenAI released GPT-6 Astra and a Safety overview. The official statement said it reached a new level in computer use, coding, cybersecurity, and science, and classified its safety capability as Critical under the Preparedness Framework. On the same day, Daybreak for Frontline Defenders was also released, announcing a $1 billion investment to support frontline defenders. In the accompanying GitHub ecosystem, openai/codex has 121,712 stars, openai/evals has 19,387 stars, and openai/openai-agents-python has 29,208 stars.

Business implication → The stronger the model capability, the more it depends on external safety evaluation, code review, red teaming, and scenario-based verification. Automated metrics cannot fully capture judgments such as "is it really safe" or "is it really controllable," so human judgment will move forward from the training stage to the safety validation stage. For Knowlyr, the most valuable areas to track are safety evaluation sets, code agent trajectories, over-privilege action judgments, and human review samples for high-risk tasks.

P1 From 2026-09-01 to 2026-09-03, papers focused on preference learning and synthetic judgment [P1]

On 2026-09-03, Subspace Inference Enables Efficient Active Reward Learning from Preferences discussed how to actively learn reward models from preferences. On 2026-09-01, Patterning in Practice: Debiasing Reward Models with Susceptibilities proposed using susceptibilities to debias reward bias. On 2026-09-01, StudentSim: Training LLM-based Student Simulators pointed out that real learner feedback is scarce and expensive. On 2026-09-03, Instruction Duplication as an Inference-Time Control Primitive showed that inference-time control can also serve as a structured intervention mechanism.

Business implication → The academic community is breaking the "human judgment is too expensive" problem into three parts: how to ask less, how to ask more precisely, and how to use simulated judgment to fill in scarce samples. This will directly increase demand for high-quality preference pairs, expert comparison pairs, and synthetic simulation pairs. For the data industry, what will be sold in the future is not raw answers, but a "trainable judgment distribution."

P2 EleutherAI strengthened retrain bank and reward-hacking data from 2026-08-30 to 2026-09-04 [P2]

On 2026-08-30, EleutherAI/djinn-problems-v1.0 had 95 downloads, and the description explicitly states that it is a dual-verifier reward-hacking environment; the fixed-djinn v2 build was updated again on 2026-09-04. In the same period, multiple retrain bank releases went live: EleutherAI/LDS-retrain-bank-adamw-wikitext2-N4656-bs8-seed1004 had 406 downloads, seed1006 had 383, seed1007 had 272, seed1005 had 174, and seed1008 had 531; EleutherAI/PARTIAL_LDS-retrain-bank-gpt2medium-16k-bs32 had 184 downloads.

Business implication → This kind of data shows that training data attribution, interpretability, and whether a model is exploiting loopholes are becoming a new track. Full retrain banks are costly, but they provide a data foundation closer to real causal judgment. For Knowlyr, this is an upstream opportunity in human judgment services: building high-value sample libraries around misattribution, reward cheating, and trustworthy rewriting.

Demand Signals

Infer training data demands from model releases

Robot collaboration / embodied planning data

extremely strong ↑ New

BAAI/Orchestra-Bench released 12,000 samples, with three-robot collaborative planning

Tool permission and safety judgment data

extremely strong ↑ New

BAAI/ToolPrivBench focuses on privileged tool selection; OpenAI GPT-6 Astra increases attention on cyber evaluation

Evaluation and preference statistics data

extremely strong ↑ New

AllenAI’s BenchMIRT trio went live, directly measuring safety and reasoning

Agent trajectories and terminal task data

strong ↑ New

NVIDIA/Open-SWE-Traces · NeMo-Gym EnterpriseOps continue expanding agent traces

Multilingual speech / TTS data

strong ↑ New

google/WaxalNLP 25,857 downloads; "Ready to Speak" emphasizes TTS-friendly text

Multimodal driving / 3D perception data

strong ↑ New

Qwen-Drive-1.0-4B moves into driving QA and motion planning; BAAI Discoverse-L / MobileVLA-CoT also launched

Preference learning / reward data

strong ↑ New

Active reward learning · Patterning · StudentSim papers appear frequently

Synthetic environment / anti-cheat data

strong ↑ New

EleutherAI/djinn-problems-v1.0 and retrain bank strengthen reward-hacking and attribution analysis

Time-series forecasting data

medium ↑ New

google/timesfm-3.0-pytorch has 123,025 downloads, and forecasting scenarios remain highly active

Code agent trajectories ↓ Dropped Present in previous issue, absent this issue

Multi-robot / embodied planning data ↓ Dropped Present in previous issue, absent this issue

Agent tool use and permission selection ↓ Dropped Present in previous issue, absent this issue

Preference learning / reward model data ↓ Dropped Present in previous issue, absent this issue

Multilingual speech and dialogue data ↓ Dropped Present in previous issue, absent this issue

Synthetic data for safety and identity verification ↓ Dropped Present in previous issue, absent this issue

Long-document and knowledge-work evaluation ↓ Dropped Present in previous issue, absent this issue

Cultural / multimodal understanding data ↓ Dropped Present in previous issue, absent this issue

Time-series forecasting data ↓ Dropped Present in previous issue, absent this issue

Want to discuss this issue?

Kai Founder & CEO

苏文 AI Documentation & Release Engineer

陆明哲 AI Product Manager

Auto-generated by AI Dataset Radar · Updated weekly

AI Dataset Radar →

BAAI Adds Three Embodied Datasetshuman judgment Shifts Toward Robot Collaboration and Tool Safety

Key Findings

Demand Signals

BAAI Adds Three Embodied Datasets
human judgment Shifts Toward Robot Collaboration and Tool Safety