When launching cloud instances, developers often face a choice between On-Demand instances and Spot instances. On-Demand instances provide reliable compute capacity at a fixed price, whereas Spot instances offer the same hardware at steep discounts in exchange for potential interruptions.
This guide breaks down the fundamental differences between Spot and On-Demand instances, compares performance, discusses use cases (like machine learning and testing), evaluates costs with examples, and provides best practices for using Spot instances in real-world scenarios.
Difference Between Spot and On-Demand Instances
🟩 Availability and Interruptions
- On-Demand Instances
- Run continuously until you stop or terminate them
- Guaranteed availability except in very rare capacity errors
- Spot Instances
- Drawn from spare capacity and can be reclaimed at short notice
- Example: Some providers (e.g., Novita AI) give 1-hour interruption notice and a 1-hour minimum run guarantee for Spot GPU instances
- Key Trade-off: On-Demand ensures continuous availability; Spot does not.
🟩 Pricing Model
- On-Demand Pricing
- Fixed rate (per second or per hour) for a given instance type and region
- Stable, predictable pricing without risk of involuntary shutdowns
- Spot Pricing
- Dynamic and heavily discounted (typically 50%–90% lower than On-Demand)
- Example: Novita AI Spot GPU instances are ~50% off (e.g., RTX 4090 at ~$0.18/hr vs $0.35/hr On-Demand)
- Rates may fluctuate over time; instances may be terminated if capacity is needed
- Key Trade-off: On-Demand = stable and reliable; Spot = cheaper but volatile.
🟩 Use Cases
- On-Demand Instances
- Best for workloads that require uninterrupted service
- Production applications, databases, mission-critical systems
- Short-term jobs with unpredictable durations (no upfront commitment)
- Spot Instances
- Best for flexible, fault-tolerant workloads that can handle interruptions
- Examples: batch processing, data analysis, big data pipelines, CI/CD runners, rendering, background tasks
- Common Spot-friendly workloads: stateless web servers, containerized environments, HPC jobs, test/dev setups
- Key Trade-off: On-Demand = guaranteed uptime; Spot = cost savings if interruption is tolerable.
Spot vs On-Demand Iinstance Performance Benchmarks
Developers can expect equivalent performance on Spot and On-Demand instances for the same instance type. Plan for interruptions, but don’t worry about CPU speed or memory differences – Spot is a pricing model, not a performance tier.
On-Demand vs Spot Instances for Machine Learning or Testing
1. ML Training / Batch Jobs
Recommended: Spot Instances with Checkpointing
Why:
- Training jobs are fault-tolerant by nature (especially with saved checkpoints).
- Spot provides up to 90% cost savings.
- Perfect match for large-scale model training, hyperparameter tuning, or data processing.
2. ML Inference / Production Services
Recommended: On-Demand Instances for baseline + Spot Instances for extra capacity
Why:
- Real-time inference needs high availability.
- On-Demand ensures stability; Spot adds cost-effective scaling for non-critical tasks.
- Use Spot only if the service can tolerate delays or has failover mechanisms.
3. Testing / Development Environments
Recommended: Spot Instances, but only if you automate environment setup
Why:
- Dev/test workloads are temporary and restartable.
- Spot is highly cost-effective for CI/CD runners, staging environments, or sandboxes.
- For long-lived or stateful dev services, you need IaC or containerization to recover quickly from interruptions.
Spot Instances vs on-Demand Instances Cost Comparison
| Instance (GPU) | On-Demand Price | Spot Price |
|---|---|---|
| RTX 5090 | $0.50 per hour | $0.25 per hour |
| RTX 4090 | $0.35 per hour | $0.18 per hour |
| High frequency RTX 4090 | $0.69per hour | $0.35 per hour |
| H200 SXM | $3.25per hour | $1.63per hour |
| A100 SXM | / | $1.60per hour |
| B200 | $3.84per hour | $1.92per hour |
| H100 SXM | $1.00per hour | $0.90per hour |
Visualizing Cost Difference: If you were running a fleet of 10 such instances continuously for a month (720 hours), the On-Demand cost would be: 10 * $0.096 * 720 ≈ $691. The Spot cost (at $0.028) for the same would be: 10 * $0.028 * 720 ≈ $202.
Of course, cost isn’t everything – an interrupted instance might delay a job or cause downtime if not handled. But for many workloads, the cost trade-off is well worth it. The key is to maximize savings while mitigating the risks, which brings us to the question of Spot instances for more sensitive workloads like databases.
Are Spot Instances Suitable for My Database Workload?
Avoid using Spot-like instances for any mission-critical, stateful, or single-instance databases.
Use them only for resilient clusters, replicas, or non-critical environments to balance cost and reliability.
When They Might Be Acceptable
Use Spot-like compute only if:
- The database is distributed and replicated
- The system is resilient to node loss
- The workload is non-critical or for testing purposes
Examples:
- Using Spot for read replicas while keeping the primary on stable compute
- Distributed databases like CockroachDB or Cassandra that tolerate node failure
- Caching systems (e.g., Redis) where data loss is not critical
Best Practices to Reduce Risk
| Strategy | Description |
|---|---|
| Replication & Auto-Recovery | Use multi-node clusters that can auto-replace lost nodes |
| Frequent Snapshots | Take regular backups for rapid recovery after a failure |
| Isolate Primary Workloads | Run primary DB nodes on stable infrastructure; use Spot only for secondary roles |
| Automate Node Replacement | Use orchestration (e.g., Kubernetes) to quickly recreate lost database nodes |
Spot Instances Best Practices
If you’re using a platform like Novita AI for GPU compute, switching to Spot is often as easy as a UI toggle.
Step 1: Access Your Console
Log in to your Novita AI GPU Console
Step 2: Switch to Spot Billing
In the right sidebar under Filter, change Billing Method to “Spot” to see discounted prices
Step 3: Deploy
Select your GPU configuration and click “Deploy”
That’s it! Your Spot Instance will launch with:
- 1-hour protection period
- Up to 50% cost savings
- 1-hour advance interruption notice
Pro tip: Implement checkpointing in your application to handle potential interruptions gracefully..
For developers and teams, Spot instances represent a powerful cost-saving tool – essentially letting you rent cloud compute for pennies on the dollar. The fundamental trade-off is clear: you exchange the absolute guarantee of uptime for a much lower price. On-Demand instances remain the workhorse for critical, stateful, or unpredictable workloads where continuity is paramount. Spot instances, however, can unlock tremendous value for jobs that can handle a restart or two. By understanding the differences in availability and pricing, carefully selecting which workloads are suited to Spot, and following best practices like checkpointing and mixed instance deployments, you can confidently integrate Spot instances into your infrastructure.
Frequently Asked Questions
On-Demand instances provide stable, guaranteed uptime at a fixed price.
Spot instances are much cheaper but can be interrupted at any time.
Choose Spot when your workload is:
Fault-tolerant
Interruptible
Flexible in timing (e.g., training, testing, batch jobs)
No. Spot and On-Demand offer identical performance for the same instance type.
The difference is only in pricing and availability, not hardware.
Novita AI is the All-in-one cloud platform that empowers your AI ambitions. Integrated APIs, serverless, GPU Instance — the cost-effective tools you need. Eliminate infrastructure, start free, and make your AI vision a reality.
Recommend Reading
How Much to Run DeepSeek R1 0528? Discover Cost-Effective ASolutions with Novita AI
Trae or Claude Code: Which Is More Suitable to Use with Kimi K2?
DeepSeek R1 0528 Cost: API, GPU, On-Prem Comparison
Discover more from Novita
Subscribe to get the latest posts sent to your email.
