Agents in Engineering

Google benchmark: frontier agents score under 50% on real engineering projects

Google's PLE, built with Mercor, has 85 long-horizon tasks in four engineering fields. Across 14 models and seven harnesses, the best mean score was 48.3%; only 41.3% of the top setup's deliverables passed every hard safety check.

Why it matters: Per Google, the model moved scores far more than the harness did. For engineering work, pick the model first and keep a licensed engineer signing off on every design.