CADGenBench: A New Benchmark for Evaluating AI-Generated 3D CAD Models
This open benchmark provides a standardized way for engineering teams to measure how well AI systems produce and edit precise, functional 3D parts for real-world use.
Practical Summary
CADGenBench offers a concrete, tool-agnostic method to evaluate AI models on generating and editing 3D CAD models. It defines two key tasks—generating a 3D model from a 2D drawing and editing a STEP file based on a change request—using metrics that assess geometric and functional accuracy, which are critical for engineering workflows.
Why It Matters
For teams integrating AI into product design, this benchmark moves evaluation beyond generic 'can it generate a 3D shape?' to 'can it generate a *functional* 3D part?'. It provides a reproducible, evidence-backed way to compare tools and track progress, helping to reduce the risk of investing in AI models that produce technically invalid or imprecise designs.
What is CADGenBench?
CADGenBench is a newly released, open benchmark designed to systematically evaluate how well AI models can generate and edit 3D CAD (Computer-Aided Design) models. It focuses on creating engineering-grade parts, not just visually plausible 3D shapes.
The Two Core Evaluation Tasks
The benchmark defines two practical tasks for AI models: 1. **Drawing-to-3D:** Given an engineering drawing, generate a valid 3D CAD model. 2. **Editing:** Given an existing STEP file and a change request, edit the file correctly. These tasks mirror common workflows in mechanical and product design.
Key Design Feature: Tool-Agnostic Evaluation
A critical feature of CADGenBench is that it is tool-agnostic. Submissions are standardized STEP files, so any CAD software stack (like Fusion 360, Onshape, SolidWorks, or open-source tools like build123d) can be used. This allows for a fair comparison of different AI model outputs regardless of the underlying software.
Scoring Metrics: Beyond Surface Appearance
Models are scored on four metrics that assess engineering validity, not just visual similarity: * **Geometric Accuracy:** How close is the shape to the target? * **Topology Correctness:** Is the model's structure (edges, faces, connections) sound? * **Interface Compatibility:** Will the part fit with other parts as intended? * **CAD Validity:** Is the file technically valid and usable in CAD software?