https://sauravpanigrahi.com/tags/benchmarks/Recent content in Benchmarks on Saurav PanigrahiHugoen-usFri, 01 May 2026 00:00:00 +0000https://sauravpanigrahi.com/work/medmarks/Fri, 01 May 2026 00:00:00 +0000https://sauravpanigrahi.com/work/medmarks/<p>Medmarks is an open-source benchmark suite for evaluating medical capabilities in language models across a mix of verifiable and open-ended clinical tasks.</p> <h2 id="focus">Focus</h2> <ul> <li>Medical LLM evaluation.</li> <li>Verifiable and open-ended benchmark tasks.</li> <li>LLM-as-judge evaluation for non-verifiable tasks.</li> <li>Clinically relevant model capability tracking.</li> </ul> <h2 id="artifacts">Artifacts</h2> <ul> <li><a href="https://sophont.med/blog/medmarks">Medmarks v0.1</a></li> <li><a href="https://arxiv.org/pdf/2605.01417v1">Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks</a></li> </ul>