Goblin
News
AI news, distilled.
|
News
About
News
About
Filtered by:
benchmarks-and-evals
Clear
Titles
Summaries
6
LangChain Guide: How to Start and Structure Agent Evals, From First Dataset to Production
Products
1
1h ago
6
LangChain Guide: How to Start and Structure Agent Evals, From First Dataset to Production
Products
· 1 src · 1h ago
Discuss
8
SpaceXSI: Grok Voice 3.0 Launches at $0.08/min, Tops Artificial Analysis Speech-to-Speech Index
Products
1
9h ago
8
SpaceXSI: Grok Voice 3.0 Launches at $0.08/min, Tops Artificial Analysis Speech-to-Speech Index
Products
· 1 src · 9h ago
Discuss
7
Terrorism AI Benchmark: Only 2 of 134 Models Fully Pass; Open-Weight Qwen 3.8 Stripped of Guardrails Within a Day
Safety
1
17h ago
7
Terrorism AI Benchmark: Only 2 of 134 Models Fully Pass; Open-Weight Qwen 3.8 Stripped of Guardrails Within a Day
Safety
· 1 src · 17h ago
Discuss
7
MIT Study: OpenAI Models' Theory-Problem Accuracy Rose From ~2% to ~35% in Two Years
Research
1
9h ago
7
MIT Study: OpenAI Models' Theory-Problem Accuracy Rose From ~2% to ~35% in Two Years
Research
· 1 src · 9h ago
Discuss
6
Kombai Design Creativity Bench: Best AI Model Scores 65.5 vs 92.3 for Human Designers
Research
1
9h ago
6
Kombai Design Creativity Bench: Best AI Model Scores 65.5 vs 92.3 for Human Designers
Research
· 1 src · 9h ago
Discuss
6
Leash Bio: Releases Minos Agent Sandbox, Preprint and Open Dataset for Verifiable Scientific Reasoning
Research
1
9h ago
6
Leash Bio: Releases Minos Agent Sandbox, Preprint and Open Dataset for Verifiable Scientific Reasoning
Research
· 1 src · 9h ago
Discuss
6
Lakebase Search: Sizing Guidance from a 700-Deck Hybrid Search App and a 4M-Passage Test
Products
1
17h ago
6
Lakebase Search: Sizing Guidance from a 700-Deck Hybrid Search App and a 4M-Passage Test
Products
· 1 src · 17h ago
Discuss
7
Google DeepMind: Gemini 4 Argon for Robotics Claims 87% on ERQA Embodied Reasoning Benchmark
Models
3
1d ago
7
Google DeepMind: Gemini 4 Argon for Robotics Claims 87% on ERQA Embodied Reasoning Benchmark
Models
· 3 srcs · 1d ago
Discuss
7
LangChain: Harness-Only Changes Lift Deep Agents Coding Agent from Top 30 to Top 5 on Terminal Bench 2.0
Products
1
1d ago
7
LangChain: Harness-Only Changes Lift Deep Agents Coding Agent from Top 30 to Top 5 on Terminal Bench 2.0
Products
· 1 src · 1d ago
Discuss
6
Four Used Tesla V100s on NVLink Board Outrun Two DGX Sparks and an M5 Ultra on Local LLM Inference
Infra
1
1d ago
6
Four Used Tesla V100s on NVLink Board Outrun Two DGX Sparks and an M5 Ultra on Local LLM Inference
Infra
· 1 src · 1d ago
Discuss
6
Laplace Intelligence: Michigan Startup Automates Red-Teaming for Robotics and Autonomous Vehicles
Products
1
1d ago
6
Laplace Intelligence: Michigan Startup Automates Red-Teaming for Robotics and Autonomous Vehicles
Products
· 1 src · 1d ago
Discuss
6
Wikipedia Bans LLM Writing as AI Content Spreads and AI Overviews Reshape Search
Products
1
1d ago
6
Wikipedia Bans LLM Writing as AI Content Spreads and AI Overviews Reshape Search
Products
· 1 src · 1d ago
Discuss
6
Developer Turns Old Kindle Into Photo Frame; GPT-6 Astra Succeeds at Jailbreak Where Other Models Failed
Security
1
2d ago
6
Developer Turns Old Kindle Into Photo Frame; GPT-6 Astra Succeeds at Jailbreak Where Other Models Failed
Security
· 1 src · 2d ago
Discuss
6
Study: LLMs and Humans Both Unreliable at Annotating Machine Translation Quality
Research
1
2d ago
6
Study: LLMs and Humans Both Unreliable at Annotating Machine Translation Quality
Research
· 1 src · 2d ago
Discuss
6
Composo: DishwasherBench v0.1.0 Benchmarks Language Models on Physics-Simulated Dishwasher Loading
Research
1
1d ago
6
Composo: DishwasherBench v0.1.0 Benchmarks Language Models on Physics-Simulated Dishwasher Loading
Research
· 1 src · 1d ago
Discuss
7
Vibe Code Bench: Multi-Agent Teams Cost 1.8-5.1x More, Significantly Help Only One of Four Setups
Research
1
2d ago
7
Vibe Code Bench: Multi-Agent Teams Cost 1.8-5.1x More, Significantly Help Only One of Four Setups
Research
· 1 src · 2d ago
Discuss
7
DecepEval: New Benchmark Tests Whether LLM Agents Deceive Under Pressure, Incentive, Opportunity and Conflict
Research
1
2d ago
7
DecepEval: New Benchmark Tests Whether LLM Agents Deceive Under Pressure, Incentive, Opportunity and Conflict
Research
· 1 src · 2d ago
Discuss
7
Study: Public Benchmark Scores Poorly Predict How Prompt-Injection Detectors Perform on Real Agent Tool Outputs
Research
1
2d ago
7
Study: Public Benchmark Scores Poorly Predict How Prompt-Injection Detectors Perform on Real Agent Tool Outputs
Research
· 1 src · 2d ago
Discuss
7
Elorian and Scale AI: Humanity's Sixth Sense Benchmark Shows Frontier Models at 53.6% vs Humans' 93.1% on Implicit Visual Reasoning
Research
1
2d ago
7
Elorian and Scale AI: Humanity's Sixth Sense Benchmark Shows Frontier Models at 53.6% vs Humans' 93.1% on Implicit Visual Reasoning
Research
· 1 src · 2d ago
Discuss
7
Harness Design Study: Context Management, Planning and Tools Tested Across 176 Coding-Agent Settings
Research
1
2d ago
7
Harness Design Study: Context Management, Planning and Tools Tested Across 176 Coding-Agent Settings
Research
· 1 src · 2d ago
Discuss
7
Perplexity: pplx-decider-v1.1-27b Lifts Decision Index to 61.56 by Removing Causal Mask
Models
1
2d ago
7
Perplexity: pplx-decider-v1.1-27b Lifts Decision Index to 61.56 by Removing Causal Mask
Models
· 1 src · 2d ago
Discuss
6
StudentSim: Research System Trains Per-Student AI Simulators to Test AI Tutors Before Real-Learner Trials
Research
1
2d ago
6
StudentSim: Research System Trains Per-Student AI Simulators to Test AI Tutors Before Real-Learner Trials
Research
· 1 src · 2d ago
Discuss
7
Study: 10 Minutes of AI Assistance Impairs Focus and Persistence on Hard Tasks, Presented at CoLM
Research
1
3d ago
7
Study: 10 Minutes of AI Assistance Impairs Focus and Persistence on Hard Tasks, Presented at CoLM
Research
· 1 src · 3d ago
Discuss
6
Voice AI Execs Say Field Lacks Its 'ChatGPT Moment'; Speed and Transcription Accuracy Seen as Key Gaps
Products
1
3d ago
6
Voice AI Execs Say Field Lacks Its 'ChatGPT Moment'; Speed and Transcription Accuracy Seen as Key Gaps
Products
· 1 src · 3d ago
Discuss
6
Anthropic, OpenAI Seek Independent AI Evaluators as OpenAI Firing Dispute Raises Access Questions
Safety
1
3d ago
6
Anthropic, OpenAI Seek Independent AI Evaluators as OpenAI Firing Dispute Raises Access Questions
Safety
· 1 src · 3d ago
Discuss
7
Harvard Study: AI Coding Agents Raise Code Output 30% but Not Resolved Issues; Code Review Is Bottleneck
Research
1
5d ago
7
Harvard Study: AI Coding Agents Raise Code Output 30% but Not Resolved Issues; Code Review Is Bottleneck
Research
· 1 src · 5d ago
Discuss
7
Only 3.6% of Chinese AI Lab Releases Included Safety Results, SemiAnalysis Finds
Safety
1
5d ago
7
Only 3.6% of Chinese AI Lab Releases Included Safety Results, SemiAnalysis Finds
Safety
· 1 src · 5d ago
Discuss
6
ATLAS Benchmark: 547 Deep-and-Wide Search Tasks; No Sub-$1 Agent Run Tops 0.5 Row F1
Research
1
5d ago
6
ATLAS Benchmark: 547 Deep-and-Wide Search Tasks; No Sub-$1 Agent Run Tops 0.5 Row F1
Research
· 1 src · 5d ago
Discuss
6
Nikon Disqualifies Small World in Motion Winner Over AI Use; Nguyen Nam Nhat Named Winner
Creative
4
5d ago
6
Nikon Disqualifies Small World in Motion Winner Over AI Use; Nguyen Nam Nhat Named Winner
Creative
· 4 srcs · 5d ago
Discuss
6
Sakana AI: Japan-Specific LLM 'Namazu' Powers Iris's Evidence Finder Medical Search Tool
Enterprise
1
6d ago
6
Sakana AI: Japan-Specific LLM 'Namazu' Powers Iris's Evidence Finder Medical Search Tool
Enterprise
· 1 src · 6d ago
Discuss
7
TII Launches Falcon-ASR: State-of-the-Art Arabic Speech Recognition Model
Models
1
6d ago
7
TII Launches Falcon-ASR: State-of-the-Art Arabic Speech Recognition Model
Models
· 1 src · 6d ago
Discuss
7
Claude Haiku 5.5 Added to Devin Fusion, Scores 58.4% on FrontierCode at ~1/8 of Sonnet 5 Cost
Products
1
Oct 8
7
Claude Haiku 5.5 Added to Devin Fusion, Scores 58.4% on FrontierCode at ~1/8 of Sonnet 5 Cost
Products
· 1 src · Oct 8
Discuss
7
Arena: AI Model Ranking Platform Raises $200M Series B at $3.1B Valuation
Markets
3
6d ago
7
Arena: AI Model Ranking Platform Raises $200M Series B at $3.1B Valuation
Markets
· 3 srcs · 6d ago
Discuss
7
Developer Uses Claude Opus 5.5 to Port TypeScript Compiler to Rust for ~$24K, Beating OpenAI Models That Spent $420K+
Open Source
1
6d ago
7
Developer Uses Claude Opus 5.5 to Port TypeScript Compiler to Rust for ~$24K, Beating OpenAI Models That Spent $420K+
Open Source
· 1 src · 6d ago
Discuss
7
Nemotron Achieves Gold-Medal Level at Both IMO 2026 and IOI 2026
Research
1
Oct 7
7
Nemotron Achieves Gold-Medal Level at Both IMO 2026 and IOI 2026
Research
· 1 src · Oct 7
Discuss
6
Research: LLMs Spontaneously Rediscover Telegraph-Era 'Cablese' Compression, Cutting Tokens 40–48%
Research
1
Oct 7
6
Research: LLMs Spontaneously Rediscover Telegraph-Era 'Cablese' Compression, Cutting Tokens 40–48%
Research
· 1 src · Oct 7
Discuss
8
Liquid AI Adds Vision to d1 Decision Model, Undercutting Frontier Models by Up to 200x
Models
1
Oct 6
8
Liquid AI Adds Vision to d1 Decision Model, Undercutting Frontier Models by Up to 200x
Models
· 1 src · Oct 6
Discuss
6
Two-Year RCT Finds Khanmigo Raises Math Scores, But Student Engagement Is the Real Bottleneck
Research
1
Oct 6
6
Two-Year RCT Finds Khanmigo Raises Math Scores, But Student Engagement Is the Real Bottleneck
Research
· 1 src · Oct 6
Discuss
7
Only 11% of Businesses Can Forecast AI Spending as Microsoft Research Reveals 'Price Reversal' Phenomenon
Enterprise
1
Oct 5
7
Only 11% of Businesses Can Forecast AI Spending as Microsoft Research Reveals 'Price Reversal' Phenomenon
Enterprise
· 1 src · Oct 5
Discuss
7
DeepSeek V4.1 Flash Narrows US-China AI Performance Gap to ~3% on LiveBench
Models
1
Oct 5
7
DeepSeek V4.1 Flash Narrows US-China AI Performance Gap to ~3% on LiveBench
Top
Models
· 1 src · Oct 5
Discuss
7
Stack Overflow Survey: 60% of Developers Actively Try to Avoid AI Tools
Research
1
Oct 5
7
Stack Overflow Survey: 60% of Developers Actively Try to Avoid AI Tools
Research
· 1 src · Oct 5
Discuss
7
Safeworld Raises $12M Seed to Safety-Test Generative AI Robots
Safety
1
Oct 5
7
Safeworld Raises $12M Seed to Safety-Test Generative AI Robots
Safety
· 1 src · Oct 5
Discuss
7
Cantina Security Releases apex-flash-1: Open-Weight Model for Vulnerability Research Solves 40/60 Bug Tasks
Security
1
Oct 5
7
Cantina Security Releases apex-flash-1: Open-Weight Model for Vulnerability Research Solves 40/60 Bug Tasks
Security
· 1 src · Oct 5
Discuss
7
Google Platforms Named Primary AI Infrastructure by 48% of Production Enterprises — Twice OpenAI's Share
Enterprise
1
Oct 5
7
Google Platforms Named Primary AI Infrastructure by 48% of Production Enterprises — Twice OpenAI's Share
Enterprise
· 1 src · Oct 5
Discuss
6
GitHub Releases ReviewBench: Open Benchmark for Evaluating AI Code Review Agents
Open Source
1
Oct 5
6
GitHub Releases ReviewBench: Open Benchmark for Evaluating AI Code Review Agents
Open Source
· 1 src · Oct 5
Discuss
6
SciUniverse Benchmark Tests Frontier AI Models on Real Scientific Lab Tasks
Research
1
Oct 5
6
SciUniverse Benchmark Tests Frontier AI Models on Real Scientific Lab Tasks
Research
· 1 src · Oct 5
Discuss
6
HackerRank Launches Chakra AI Interviewer to General Availability After 500K Beta Interviews
Products
1
Oct 5
6
HackerRank Launches Chakra AI Interviewer to General Availability After 500K Beta Interviews
Products
· 1 src · Oct 5
Discuss
6
UniEvo-VL: Self-Distillation Framework Lets Multimodal Models Improve Without External Teacher
Research
1
Oct 5
6
UniEvo-VL: Self-Distillation Framework Lets Multimodal Models Improve Without External Teacher
Research
· 1 src · Oct 5
Discuss
7
Microsoft and Hugging Face Release ThinkingBox, an AI Agent Benchmark Grading Database Outcomes
Research
1
Oct 4
7
Microsoft and Hugging Face Release ThinkingBox, an AI Agent Benchmark Grading Database Outcomes
Research
· 1 src · Oct 4
Discuss
6
MIT and Sakana AI Release SIFT: Cheaper Evaluation for Self-Improving Coding Agents
Research
1
Oct 3
6
MIT and Sakana AI Release SIFT: Cheaper Evaluation for Self-Improving Coding Agents
Research
· 1 src · Oct 3
Discuss
Filters
Signal
Title
Category
Sources
Posted
Discuss
