ARISEKit

Start from an established benchmark format or build your own.

Recommended toolkit

NOHARMKit

Score clinical recommendations by their potential to harm patients. Move from your own cases and clinician-reviewed rubrics to autograded responses and a model leaderboard.

Open NOHARMKit Paper

Five days of fever with a rash

Case

4-year-old with 5 days of fever, bilateral nonexudative conjunctivitis, cracked red lips, a polymorphous rash and swollen hands. Fever responds to ibuprofen. Seen in urgent care. Discharge with return precautions?

Solution

Concern for Kawasaki disease. Same-day pediatric ED evaluation with labs and an echocardiogram; IVIG with aspirin if criteria are met.

Errors

Discharge homeSevere
Routine follow-up next weekModerate
Defer the echocardiogramModerate
Rapid strep test onlyMild

Generate Rubric

Auto-generate a NOHARM-style rubric

1Dataset2Rubrics3Responses4Autograding5Scores6Leaderboard

Established Kits

SCTKitMeasure how judgments shift with new, uncertain information, compared with an expert panel. PaperMCQ
CPCKitBuild diagnostic reasoning benchmarks from complex cases, starting with staged differentials. PaperMCQOpen-ended
MedAgentKitTest AI agents on multi-step tasks in a simulated electronic health record. PaperAgentic
RadiologyKitEvaluate AI-generated chest X-ray reports against reference reports. WebsiteOpen-ended
MultiturnKitEvaluate AI across multi-turn medical conversations.Open-endedComing soon
PhysicianBenchKitTest AI agents on long-horizon physician tasks in an electronic health record.AgenticComing soon

Build Your Own

Guided assistantDescribe your idea and get guided to the right custom design.