Radar Brief Week 23, 2026 · 2026-07-02 — 2026-07-09

BioMysteryBench Reaches 2,408 Downloads
Non-Retrievable Tasks Validate the Scarcity of Human Judgment

This week scanned 86 HF orgs · 50 GitHub orgs · 71 blogs · 125 X accounts

0
Valuable Datasets
0
Related Papers
0
Blog Posts
0
Active Repos
One-line Summary

Anthropic’s BioMysteryBench-full, released on 2026-04-29, reached 2,408 downloads, pointing to demand for high-value judgment evaluation around “non-retrievable answers” [P0]; NVIDIA simultaneously scaled up robotics and physical-world datasets, with PhysicalAI-Robotics-Locomanipulation-GRAIL reaching 24,483 downloads as the strongest demand signal of the week [P0]; from 2026-07-01 to 2026-07-02, Meta consecutively released S-EMBER and r3d-bench, making video memory and 3D spatial reasoning new evaluation frontiers [P1]. Strongest data demand signal this week: robotics manipulation trajectories.

Key Findings

This week's 5 high commercial value findings

P0 Anthropic’s BioMysteryBench-full, released on 2026-04-29, reached 2,408 downloads, pointing to demand for high-value judgment evaluation around “non-retrievable answers” [P0]

Anthropic/BioMysteryBench-full was released on 2026-04-29 and currently has 2,408 downloads and 30 likes; Anthropic/BioMysteryBench-preview, released the same day, has 581 downloads and 14 likes. This benchmark contains 90 bioinformatics problems requiring real analysis based on anonymous biological data files, covering alignment, expression, variant calling, motif discovery, structure, and more, with the condition that the “source dataset cannot be looked up.”

Business significance → This is not simple knowledge Q&A data, but a high-difficulty evaluation set that requires humans to make judgments from raw evidence while eliminating retrieval shortcuts. For Knowlyr, this is direct validation of the model of “enabling people to earn income by contributing judgment”: high-value training and evaluation data will increasingly require experts to adjudicate evidence chains, reasoning processes, and answer verifiability. This creates a strong opportunity to expand task-oriented evaluation and preference data services in life sciences, healthcare, industrial analysis, and other domains where you cannot rely on memorized answers.
P0 NVIDIA simultaneously scaled up robotics and physical-world datasets, with PhysicalAI-Robotics-Locomanipulation-GRAIL reaching 24,483 downloads as the strongest demand signal of the week [P0]

nvidia/PhysicalAI-Robotics-Locomanipulation-GRAIL was released on 2026-04-28 with 24,483 downloads and 13 likes; nvidia/GR00T-N1.7-AppleToPlate was released on 2026-05-21 with 217 downloads; nvidia/Arena-G1-Static-PickNPlace-Task was released on 2026-05-06 with 14 downloads. All three focus on humanoid loco-manipulation, multimodal trajectories, video, actions, and human-object interaction; during the same period, the NVIDIA Robotics Blog emphasized advancing the LeRobot open-source ecosystem with Hugging Face this week.

Business significance → Robotics training has shifted from static visual data to integrated “state-vision-language-action” trajectories, requiring fine-grained judgment over success/failure, stage progress, grasp quality, and contact events. The human value here is not low-end box drawing, but judgment over action quality, task completion, anomaly recovery, and safety boundaries. Knowlyr can enter this space through robot demonstration replay review, failure case tiering, trajectory preference ranking, and multi-stage task completion evaluation.
P1 From 2026-07-01 to 2026-07-02, Meta consecutively released S-EMBER and r3d-bench, making video memory and 3D spatial reasoning new evaluation frontiers [P1]

facebook/S-EMBER was released on 2026-07-01 with 17 downloads and 0 likes, serving as a video QA benchmark for streaming egocentric memory retrieval; facebook/r3d-bench was released on 2026-07-02 with 301 downloads and 2 likes, containing 3,033 QA annotations and 57 Aria Digital Twin sequences for quantitative 3D spatial reasoning on natural first-person RGB-D video.

Business significance → The shared challenge in first-person video, memory retrieval, and 3D reasoning is that models must understand temporal continuity, spatial relationships, and task context, while the reliability of automatically generated pseudo-labels remains limited. Future high-value data demand will concentrate in event labeling for video segments, spatial relation adjudication, memory recall relevance judgment, and task-stage summarization. Knowlyr can design a “continuous video judgment task marketplace” that decomposes complex multi-round judgment into distributable units of human contribution.
P1 Agent evaluation is shifting from static Q&A to real trajectories and long-horizon decision-making, with Qwen/AgentWorldBench and AllenAI Asta data providing synchronized signals [P1]

Qwen/AgentWorldBench was released on 2026-06-22 with 1,989 downloads and 70 likes, built from real frontier-model trajectories across Tool Decathlon, Terminal-Bench, OSWorld-Verified, and others; allenai/asta-summary-citation-counts increased from 680 to 1,438 downloads, a gain of 758 or 111.5%; allenai/asta-bench-submissions rose from 78 to 94. Meanwhile, Snorkel published multiple articles this week on Agents’ Last Exam, continual learning, and agentic evaluation, while Senior SWE-Bench received 186 votes and 119 comments on Hacker News on 2026-07-02.

Business significance → Agent data demand is migrating from “instruction-response pairs” to full-chain records of “trajectory-outcome-citation-tool use-long-term performance.” What is scarcest here is human judgment over process quality: whether steps are reasonable, whether calls are necessary, whether citations are trustworthy, and whether failures are recoverable. Knowlyr should prioritize building capabilities for agent trajectory review, tool-use preferences, citation correctness verification, and segmented scoring for long-duration tasks, rather than focusing only on static SFT corpora.
P2 Alignment research this week concentrated on “noisy preferences, data selection as alignment, and failure-driven data evolution,” showing that high-quality human feedback remains the bottleneck [P2]

The 2026-07-03 paper “Unbiased Alignment for Large Language Models with Noisy Preferences” directly addresses noise in real preference data; the 2026-07-06 paper “Attention Limited Reward Learning” discusses how human comparative feedback is constrained by attention; the 2026-07-08 paper “Online Data Selection Is Implicit Alignment” argues that online data selection itself changes model preferences; the 2026-07-07 paper “CurateEvo: Data-Curation Evolving for Agentic Post-Training” emphasizes failure-driven data filtering and refinement; and a diffusion RLHF paper on 2026-07-08 highlights low feedback efficiency.

Business significance → The industry is increasingly recognizing that alignment problems are becoming human judgment collection and quality control problems. This is a structural tailwind for Knowlyr, because high-quality preferences, disputed sample review, disagreement modeling, expert arbitration, and noise correction will all become scarce capabilities. We recommend upgrading the service narrative from “data production” to “Human Judgment Infrastructure,” especially for preference learning, red teaming, safety, and agent post-training.

Demand Signals

Infer training data demands from model releases

Data Type Intensity Trend Related Signals
Robotics manipulation trajectories
Very strong ↑ New
nvidia/PhysicalAI-Robotics-Locomanipulation-GRAIL reached 24,483 downloads, with GR00T-N1.7-AppleToPlate and Arena-G1-Static-PickNPlace-Task deployed in parallel
Agent real trajectories and long-horizon evaluation
Very strong ↑ New
Qwen/AgentWorldBench was released on 2026-06-22 with 1,989 downloads; Snorkel discussed ALE, continual learning, and agent evaluation throughout the week
Scientific analysis evaluation data
Strong ↑ New
Anthropic/BioMysteryBench-full was released on 2026-04-29 with 2,408 downloads, requiring real analysis based on anonymous biological files
First-person video memory / 3D reasoning
Strong ↑ New
facebook/S-EMBER and facebook/r3d-bench were released consecutively on 2026-07-01/02
Speech multilingual QA evaluation
Strong ↑ New
google/svq has 2,060 downloads, covering 26 locales and 17 languages
Retrieval citation and scientific Agent evidence data
Strong ↑ New
allenai/asta-summary-citation-counts downloads increased from 680 to 1,438, up 111.5%
Preference learning and noise correction data
Strong ↑ New
From 2026-07-03 to 2026-07-08, multiple papers focused on noisy preferences, attention-limited feedback, and diffusion RLHF feedback efficiency
Data attribution and model memory analysis
Medium ↑ New
EleutherAI/bergson-wikitext-512-chunks and bergson-recall-9000 were released on 2026-07-08
Privacy and synthetic sensitive text
Medium ↑ New
nvidia/Privasis-USA and Meta’s privacy-aware infrastructure blog jointly reinforce demand for privacy data governance
Climate / simulation time-series environments
Medium ↑ New
Three nvidia/STRATA-SCREAM datasets were released on 2026-06-30 for weather and climate ML
Fixed-camera video understanding / cross-shot tracking data ↓ Dropped Present in previous issue, absent this issue
RL environment and task episode data ↓ Dropped Present in previous issue, absent this issue
Reward model and preference comparison data ↓ Dropped Present in previous issue, absent this issue
Agent tool-use trajectories ↓ Dropped Present in previous issue, absent this issue
Multilingual quality evaluation data ↓ Dropped Present in previous issue, absent this issue
Scientific and enterprise document reasoning data ↓ Dropped Present in previous issue, absent this issue
3D world models / robotics spatial annotation ↓ Dropped Present in previous issue, absent this issue
Safety and reliability evaluation data ↓ Dropped Present in previous issue, absent this issue
Scientific computing and industrial simulation data ↓ Dropped Present in previous issue, absent this issue
Video generation alignment data ↓ Dropped Present in previous issue, absent this issue

Download Movers

Datasets with the largest download changes this week

Dataset Downloads Weekly Growth
allenai/asta-summary-citation-counts 1,438 +111.5%
allenai/asta-bench-submissions 94 +20.5%
nvidia/HiLiftAeroML 12,597 +11.2%

Want to discuss this issue?

Kai
Kai Founder & CEO
苏文
苏文 AI Documentation & Release Engineer
陆明哲
陆明哲 AI Product Manager

Auto-generated by AI Dataset Radar · Updated weekly

AI Dataset Radar →