MaCBench: Multimodal Reasoning Benchmark for Chemistry and Materials Science

MaCBench is a benchmark designed to evaluate the multimodal reasoning capabilities of vision-language models (VLMs) in chemistry and materials science. It includes over 1,100 hand-crafted question-image pairs covering data extraction, experimental understanding, and results interpretation, using manually curated visuals from real-world scientific scenarios.

Citation: Alampara, Nawaf, Indrajeet Mandal, Pranav Khetarpal, Hargun Singh Grover, Mara Schilling-Wilhelmi, NM Anoop Krishnan, and Kevin Maik Jablonka. "Macbench: a multimodal chemistry and materials science benchmark." In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024). 2024.