Evaluating Modern LLMs for General Reasoning, Coding, and Math

Large Language Models (LLMs) have revolutionized natural language processing, offering powerful capabilities for text and code generation. Here we evaluate cutting-edge AI models from leading technology organizations, highlighting their specialized capabilities and measurable performance— with a particular focus on LLM as a general-purpose LLM and as a coder and mathematical reasoning.

This post is motivated by the rapid advancement of AI models, which has made it challenging to select the right one for specific needs. With this in mind, we aim to explore two key questions: To what extent are these AI models accurate enough to handle general-purpose tasks, and which AI models are best suited for specific applications?

To address these questions, we evaluate seven widely-used LLMs based on data available as of March 2025. These models include the state-of-the-art LLMs: DeepSeek-V3-0324, Qwen2.5-VL-32B, Llama-3.3 70B Instruct, Mistral Small 3.1-24B, Gemini 2.5 Pro, Claude-Sonnet 3.7 and GPT-4.5. Here, we focus on evaluating the versatility of LLMs across three key categories:
(i) General Knowledge and Reasoning, measured using the benchmarks MMLU-Pro and GPQA-Diamond ;
(ii) Code Generation and Programming, evaluated through the metrics LiveCodeBench; and
(iii) Mathematical Problem Solving, assessed using the benchmark MATH .

This approach provides a comprehensive view of LLM performance in both general-purpose and specialized task domains. Ultimately, these benchmarks help differentiate between strong and weaker models, providing clear insights into the strengths and limitations of each. This clarity empowers users to make informed decisions when selecting the most suitable model for their needs.

The results of the performance comparison are summarized in the Table 1 and further detailed in the following sections.

Category	Top Models
General Knowledge & Reasoning	GPT-4.5 & DeepSeek-V3-0324
Processing complex questions	Claude-Sonnet 3.7 & Gemini 2.5 Pro
Code generation	Gemini 2.5 Pro
Mathematical Problem Solving	Gemini 2.5 Pro

General Knowledge and Reasoning

MMLU-Pro

MMLU-Pro stands for Massive Multitask Language Understanding – Proficient-level ). MMLU-Pro is a dataset encompassing over 12,000 questions spaning 14 diverse domains including mathematics, physics, chemistry, law, engineering, psychology, and health. It integrates more challenging, reasoning-focused questions. MMLU-Pro is a comprehensive benchmark designed for proficient-level multi-discipline language understanding and reasoning. MMLU-Pro is designed to assess an LLM’s ability to tackle more advanced reasoning challenges…across diverse disciplines. It’s also used to evaluate the stability of LLM under varying prompts.

We present in Fig. 1 a comparative performance of various LLMs in handling general knowledge and reasoning tasks by measuring MMLU-Pro. LLMs include: DeepSeek-V3-0324, Qwen2.5-VL-32B, Llama-3.3 70B Instruct, Mistral Small 3.1-24B, and GPT-4.5. It is clearly shown from the chart that GPT-4.5 and DeepSeek-V3-0324 significantly outperforms the other LLMs achieving a score of 86.1 % and 81.2 % respectively. This represents almost a 24-30% improvement over other competing models.

The results of the comparison indicate that both GPT-4.5 and DeepSeek-V3-0324 are suitable for tasks related to general knowledge and reasoning.

GPQA-Diamand

GPQA stands for Graduate-Level Google-Proof Q&A Benchmark. It is a dataset of 448 multiple-choice questions covering  biology, physics, and chemistry. It measures the ability of LLMs to answer challenging questions of multiple-choice in the fields of biology, physics, and chemistry.  As a reference, experts who have PhDs in the corresponding domains reach only 65% accuracy. The term “Diamand” refers to highest quality subset which includes only questions where both experts answer correctly and the majority of non-experts answer incorrectly. All questions are classified as “Hard undergraduate level”. Measures how well an LLM comprehends and processes complex questions.

We summarize in Fig. 2 the performance of various LLMs in terms of processing complex questions by measuring GPQA-Diamand. The LLMs include: DeepSeek-V3-0324, Qwen2.5-VL-32B, Llama-3.3 70B Instruct, Mistral Small 3.1-24B, Gemini 2.5pro, Claude-Sonnet 3.7 and GPT-4.5. The chart clearly shows that both Claude-Sonnet 3.7 and Gemini 2.5 Pro dominate the score with a performance of about 84%.

The results indicate that both Claude-Sonnet 3.7 and Gemini 2.5 Pro are suitable for tasks related to processing complex questions.

Code Generation & Programming

LiveCodeBench

LiveCodeBench is a new benchmark introduced to evaluate the code capabilities of LLMs. It is designed to address key limitations observed in traditional coding benchmarks like HumanEval and MBPP, which are no longer sufficient as LLMs advance. Here, one of the critical issue is related to data contamination, which occurs when a LLM has been trained on the same or similar data that is later used to evaluate its performance, potentially misleading performance metrics. The LiveCodeBench benchmark mitigates this concern through a collection of over 500 fresh coding problems, enabling accurate assessment of model capabilities.

Based on the LiveCodeBench benchmark, we evaluate the coding performance of various LLMs, mainly DeepSeek-V3-0324, Gemini 2.5 Pro and GPT-4.5. This limited selection of LLMs is due to data availability at the time of the analysis (March 2025). As clearly illustrated in Fig. 3, Gemini 2.5 Pro demonstrates superior performance, achieving a score of 70.4 %. This represents a substantial 43–60% improvement over competing models.

The results of the comparison indicate that Gemini 2.5 Pro is suitable for code generation tasks.

Mathematical Problem-Solving

MATH

MATH is a dataset of 12,500 challenging competition mathematics problems. It also includes a contribution of pretraining dataset to help teach LLMs the fundamentals of mathematics. The benchmark is introduced to measure the ability of LLMs in solving mathematical problems.

In Fig. 4, we compare the mathematical reasoning performance of different LLMs by measuring the MATH benchmark. LLMs include: Qwen2.5-VL-32B, Llama-3.3 70B Instruct, Mistral Small 3.1-24B and Gemini 2.5 Pro. The results clearly show that high performance of Gemini 2.5 Pro, achieving a score of 91.8% and exceeding other competing models.

The results of the comparison indicate that Gemini 2.5 Pro is suitable for solving mathematical problems.

Conclusion

We have presented advanced benchmark metrics for various AI models that unify natural language processing (NLP) and coding workflows, including mathematical reasoning. These benchmarks highlight two key insights: first, that LLMs can serve as powerful tools across education, research, and business; and second, they help identify which AI models excel in specific domains. The accuracy of a large language model (LLM) varies depending on the specific domain in which it is evaluated. Notably, the highest observed scores are 86.1% for general knowledge, 84.8% for complex question processing, 70.4% for code generation, and 91.8% for mathematical reasoning. However, the current benchmark datasets used to evaluate these models may require further development to ensure a more accurate and comprehensive assessment of AI capabilities. We believe this content can support users in choosing the most suitable AI model for their particular needs.