DeepSeek R1 vs OpenAI o1: Distinct Architectures of GRPO and PPO

Key Highlights

Architectural Differences
DeepSeek R1: Uses Mixture-of-Experts (MoE) and GRPO, offers transparent chain-of-thought reasoning, and is open-source.
OpenAI o1: Likely uses transformers with PPO, has opaque reasoning processes, and is closed-source.

Cost Comparison
DeepSeek R1: Much cheaper with input at $4/million tokens and output at $4/million tokens on Novita AI.
OpenAI o1: Expensive, with input at $15/million tokens and output at $60/million tokens.

Performance and Use Cases
DeepSeek R1: Excels in reasoning-heavy tasks like math, coding, and domain-specific analysis (e.g., healthcare, finance).
OpenAI o1: Strong in general-purpose tasks like creative content, multilingual applications, and broad reasoning.

If you’re looking to evaluate the DeepSeek R1 on your own use-cases — Upon registration, Novita AI provides a $0.5 credit to get you started!

The field of large language models (LLMs) is advancing rapidly, with new models constantly redefining benchmarks. DeepSeek’s R1 model has emerged as a strong contender, particularly in reasoning tasks, challenging OpenAI’s o1 series. This article provides a practical and technical comparison of these two models, focusing on their features, performance, costs, hardware requirements, and use cases.

Basic Introduction of Model

To begin our comparison, we first understand the fundamental characteristics of each model.

DeepSeek R1

• Release Date: January 21, 2025
• Model Scale:
  • deepseek/deepseek-r1
  • deepseek/deepseek-r1-distill-qwen-14b
  • deepseek/deepseek-r1-distill-qwen-32b
• Key Features:
  • Model Size: 671B parameters (37B active/token)
  • Tokenizer: Enhanced tokenizer with self-reflection tags
  • Supported Languages: Multilingual with cultural adaptation
  • Multimodal: Text-only
  • Context Window: 128K tokens
  • Storage Formats: Q8/Q5 quantization support
  • Architecture: Mixture of Experts (MoE) + RL-enhanced training pipeline
  • Training Method: Built on V3 base with RL pipeline (SFT → RL → SFT → RL)
  • Training Data: V3 base + RL optimization data

OpenAI o1

• Release Date: December 5, 2024
• Key Features:
  • Model Size: 200 billion parameters,
  • Tokenizer: Tiktoken (BPE-based), shared with GPT-4o, supports large contexts.
  • Supported Languages: Strong in English, major languages (Mandarin, German, etc.)
  • Multimodal: Text-only
  • Context Window: 128K tokens
  • Architecture: Mixture of Experts (MoE) + RL-enhanced training pipeline
  • Training Method: RLHF-optimized reasoning, iterative CoT refinement, safety-aligned outputs.
  • Training Data: Public data (web, science), proprietary partnerships, custom datasets; filtered for quality/safety.

Key Difference of Models

OpenAI’s Reinforcement Learning Approach:

• Primarily based on the PPO (Proximal Policy Optimization) algorithm.
• Improved PPO-Clip variant applied in models like GPT-4.
• Integrated with the RLHF (Reinforcement Learning from Human Feedback) framework.

DeepSeek-R1’s Algorithm Innovations:

• Utilizes the proprietary GRPO (Grouped Relative Policy Optimization) algorithm.

• Core Innovations:

  • Grouped Contrastive Learning: Decomposes policy optimization into multiple subtask groups.
  • Multi-Objective Dynamic Weighting: Supports synergistic optimization of 8 independent reward signals.
  • Hybrid Offline-Online Training: Improves data utilization efficiency by ~40%.

Speed Comparison

If you want to test it yourself, you can start a free trial on the Novita AI website.

Try DeepSeek R1 Demo Now!

Speed Comparison

source from artificialanalysis

Cost Comparison

source from artificialanalysis

Overall, OpenAI o1 performs better in latency and output speed, while the DeepSeek-R1 model has an advantage in pricing.

Benchmark Comparison

Now that we’ve established the basic characteristics of each model, let’s delve into their performance across various benchmarks. This comparison will help illustrate their strengths in different areas.

Benchmark	DeepSeek-R1 (%)	OpenAI-o1 (%)
Codeforces	96.3	96.6
GPQA Diamond	71.5	75.7
MATH-500	97.3	96.4
MMLU	90.8	91.8

These data indicate that both DeepSeek-R1 and OpenAI-o1 are high-performing models, but they excel in different areas:

• OpenAI-o1 shows a slight edge in general-purpose tasks (GPQA Diamond) and multitask language understanding (MMLU).
• DeepSeek-R1 performs better in specialized reasoning tasks, such as math problem-solving (MATH-500).
• Their performance on programming challenges (Codeforces) is nearly identical, showing both are strong in problem-solving and algorithmic reasoning.

If you want to see more comparisons, you can check out these articles:

• Deepseek v3 vs Llama 3.3 70b: Language Tasks vs Code & Math
• Llama 3.2 3B vs DeepSeek V3: Comparing Efficiency and Performance.

Applications and Use Cases

• DeepSeek R1:
  Tailored for tasks requiring advanced reasoning:
  • Diagnosing complex problems
  • Multi-step scenario analysis
  • Mathematical problem-solving
  • Coding tasks
  • Specialized domains like healthcare, finance, and legal services
• OpenAI o1:
  Designed for broader general-purpose applications:
  • Creative content generation
  • Multilingual tasks
  • Complex reasoning across diverse contexts

Accessibility and Deployment through Novita AI

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.

Step 1: Log In and Access the Model Library

Log in to your account and click on the Model Library button.

Step 2: Choose Your Model

Browse through the available options and select the model that suits your needs.

Step 3: Start Your Free Trial

Begin your free trial to explore the capabilities of the selected model.

Step 4: Get Your API Key

To authenticate with the API, we will provide you with a new API key. Entering the “Settings“ page, you can copy the API key as indicated in the image.

Step 5: Install the API

Install API using the package manager specific to your programming language.

After installation, import the necessary libraries into your development environment. Initialize the API with your API key to start interacting with Novita AI LLM. This is an example of using chat completions API for python users.

from openai import OpenAI
  
client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key="<YOUR Novita AI API Key>",
)

model = "deepseek/deepseek_r1"
stream = True # or False
max_tokens = 2048
system_content = """Be a helpful assistant"""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }

chat_completion_res = client.chat.completions.create(
    model=model,
    messages=&#91;
        {
            "role": "system",
            "content": system_content,
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p,
    presence_penalty=presence_penalty,
    frequency_penalty=frequency_penalty,
    response_format=response_format,
    extra_body={
      "top_k": top_k,
      "repetition_penalty": repetition_penalty,
      "min_p": min_p
    }
  )

if stream:
    for chunk in chat_completion_res:
        print(chunk.choices&#91;0].delta.content or "", end="")
else:
    print(chat_completion_res.choices&#91;0].message.content)
  
  

Upon registration, Novita AI provides a $0.5 credit to get you started!

If the free credits is used up, you can pay to continue using it.

DeepSeek R1 and OpenAI o1 cater to different priorities in the AI landscape. DeepSeek R1 emphasizes open-source accessibility, cost efficiency, and specialized reasoning capabilities, making it ideal for domain-specific tasks. In contrast, OpenAI o1 excels as a versatile general-purpose model with strong multilingual support. The choice depends on specific use cases—whether one values transparency and affordability or broader adaptability.

Frequently Asked Questions

Is DeepSeek R1 better than OpenAI o1?

It depends on your needs. DeepSeek R1 excels in reasoning tasks at lower costs, while OpenAI o1 offers broader functionality.

What tasks are best suited for DeepSeek R1?

Reasoning-intensive tasks like math solving, coding, and industry-specific applications.

How are the models trained?

DeepSeek R1 is significantly cheaper than OpenAI’s o1 models.

Novita AI is the All-in-one cloud platform that empowers your AI ambitions. Integrated APIs, serverless, GPU Instance — the cost-effective tools you need. Eliminate infrastructure, start free, and make your AI vision a reality.

Recommend Reading

• DeepSeek V3: Advancing Open-Source Code Models, Now Available on Novita AI
• Deepseek v3 vs Llama 3.3 70b: Language Tasks vs Code & Math
• Llama 3.2 3B vs DeepSeek V3: Comparing Efficiency and Performance.