Si2 Names NVIDIA, Synopsys Technologists to Lead New LLM Benchmarking Coalition

Estimated reading time: 2 minutes

The Silicon Integration Initiative today announced the chair and vice chair of the Si2 Large Language Model Benchmarking Coalition (LBC), a collaborative industry initiative and standards body advancing AI for silicon design and verification that will expedite the development of high-quality large language models for semiconductor design problems.

Modern chips are far more complex than a decade ago, and design teams face growing pressure to deliver reliable silicon on tight schedules. LLMs offer promising assistance—from drafting RTL to explaining verification failures—but today’s capabilities vary widely across tasks. A common, transparent way to measure progress helps the entire ecosystem learn what works, compare approaches fairly, and focus investment where it matters. Benchmarks also make it easier for new participants to contribute, accelerating practical innovation while keeping expectations grounded. Although LLMs sometimes fall short of expectations—especially on corner cases, reproducibility, and rigorous verification—clear metrics and open evaluation can steadily improve outcomes. The LLM Benchmarking Coalition’s corporate and academic members will build upon the RTL design and verification benchmarks provided in CVDP by extending problems to cover new categories and design domains, monitoring a leaderboard of results, refining benchmarking methods and metrics, and providing an interpretation of the results.

Nate Pinckney, a senior research scientist at Nvidia, was elected by Coalition members to become the coalition’s first chair. His work covers high-level synthesis methodologies, low-power VLSI design, and cryptographic accelerators, with over 40 publications in these fields. Recently, his research has focused on evaluating large language models and AI agents, including co-creating the VerilogEval and the open-source Comprehensive Verilog Design Problems benchmarks.

Igor Markov, a distinguished architect at Synopsys and an IEEE Fellow, was elected by Coalition members to become the coalition’s vice chair. He previously worked at Meta on AI platforms and at Google on Search. As a professor at the University of Michigan, he performed research on EDA, coauthored a textbook on physical design and co-edited a two-volume EDA Handbook. He won best paper awards at several conferences and published over 200 peer-reviewed papers. At Synopsys, he leads the AI Disruption Task Force.

LBC vice chair, Igor Markov, said, "Experience shows that rigorous benchmarking is the connective tissue between scientific validity and innovative progress. It ensures that claims are credible, comparisons are fair, and advances are real."

Nate Pinckney, LBC chair, added, "The Benchmarking Coalition is built on these very principles, ensuring that advances in chip and system design are grounded in evidence, transparency, and measurable impact."

The CEO of Si2, Robert Aslett, stated, “The election of Nate and Igor to chair the coalition is an important step and an example of the high quality of the current coalition participants. The coalition has ambitious plans, and I am very pleased with the number of industry and academic leaders who continue to join.”