From Legacy VPS to a High Performance Proxmox Hybrid Cloud Architecture.
How we increased traffic management capacity by 274% and reduced data analysis time by 67% for a betting industry leader by migrating to a dedicated Proxmox VE & PBS solution.
1. Executive Summary)
A leading online betting company was faced with a critical challenge: a legacy infrastructure inadequate to support growth and load spikes, with slow and risky backup processes.
The solution was a hybrid cloud architecture on dedicated, geographically distributed servers based on Proxmox VE and Proxmox Backup Servers, with a maniacal focus on I/O performance through optimized ZFS configurations on NVMe Datacenter disks.
This transition produced outstanding results, far exceeding the set goals in terms of improved performance, increased traffic handling capacity, and dramatically reduced data analysis time, providing the customer with a robust platform for future innovation.
2. The Context and Challenges: When Infrastructure Becomes a Brake
The client is a leading online betting company, which is characterized by very high transaction volumes, unpredictable traffic spikes, and the need to process large volumes of data for real-time analysis and odds.
Initial Scenario: A Legacy Infrastructure Under Pressure
- Platform:Based on legacy VPS servers, which were showing their limitations in terms of I/O resources, CPU, and flexibility.
- Performance:Slow load times, difficulty in handling user spikes, resulting in an impact on the user experience. Data analysis was a long and cumbersome process.
- Backup:Dated file-level strategy (
Borg Backup
), with long restore times, lack of virtualization-level snapshots, and dependence on complex processes. - Flexibility and Scalability: Limited, making it difficult to adapt quickly to business needs.
- Reliability: High risk in case of single VPS provider failure, without a clear geographic disaster recovery strategy.
3. Project Goals: Toward Operational Excellence
The customer had defined ambitious goals for the new infrastructure:
- Reduce data analysis execution time by 50%.
- Increase traffic handling capacity by 200% during peaks.
- Improve page load time by 50%.
- Guarantee significantly better RPO (Recovery Point Objective) and RTO (Recovery Time Objective).
- Increase the overall security of the infrastructure.
- Gain more control and visibility over the operating environment.
4. The Designed Solution: A Tailored Dedicated Hybrid Cloud Architecture with Proxmox
The answer to the challenges was a hybrid architecture that leverages the power of dedicated servers combined with the flexibility of Proxmox virtualization and the security of a geographically distributed backup strategy.
Key Components of the Architecture:
- Primary Production Node (Hetzner):
- Hardware:Dedicated server with CPU AMD EPYC 9454P (48 Cores/96 Threads) and 256 GB RAM.
- Primary Storage:
rpool
(Root Pool for OS): ZFS Mirror on 2x NVMe SSD SAMSUNG MZQL21T9HCJR-00A07 (1.92TB each).datapool
(Storage for VM/DB): ZFS raidz1-0 on 3x NVMe SSDs MICRON MTFDKCC3T8TGP-1BK1DABYY (3.84TB each).
- Virtualization: Proxmox VE, using KVMs for VMs and LXC Containers for applications.
- Networking: Configuration of
vmbr0
for network bridge over dedicated public IP.
- Dedicated Backup Node (OVHcloud - Kimsufi):
- Strategic Choice: Dedicated servers on a different provider and in a distinct geographic region to maximize resilience.
- Software: Proxmox Backup Server (PBS) for block-level, deduplicated, atomic, and encrypted backups.
- Edge Network and Security (Cloudflare):
- Used for DNS management, CDN, DDoS protection, and Web Application Firewall (WAF).
- Database (MySQL):
- Optimized for high concurrency.
- Backups configured to include stored procedures and functions with
mysqldump --routines
.
5. The "Before and After": A Radical Transformation (Comparative Analysis)
This table directly compares the parameters of the previous infrastructure with the new one, using the data that emerged from the analysis.
Features | Legacy Infrastructure (VPS) | New Dedicated Proxmox Architecture |
---|---|---|
CPU | 48 Intel Xeon Gold 6130 Cores @ 2.10GHz | AMD EPYC 9454P (48 Cores/96 Thread) |
RAM | ~128 GB | 256 GB |
System Storage | HDD/SSD SATA (Ext4) | ZFS Mirror (2x NVMe 1.92TB SAMSUNG) - rpool |
Data Storage/DB | HDD/SSD SATA (Ext4) | ZFS raidz1 (3x NVMe 3.84TB MICRON) - datapool |
Perf. Sequential Write | Low (estimated by iostat ) | 31.0 GiB/s (FIO Test) |
Perf. Sequential Read | Low (estimated) | 26.7 GiB/s (FIO Test) |
Perf. Write Random 4k | ~100-400 IOPS (estimated by sar ) | 50.1k IOPS (FIO Test) |
Perf. Read Random 4k | Low (estimated) | 52.0k IOPS (FIO Test) |
Backup | File-level (Borg), slow, non-atomic | Block-level (PBS) atomic, deduplicated, geographically separated |
Snapshots | Absent at hypervisor level | Integrated ZFS and PBS snapshots |
Flexibility | Limited | High (LXC, KVM, Proxmox API) |
Technical Background on FIO Testing on the datapool
(ZFS raidz1):
Sequential Write Test:This test simulates loads such as writing large log files or massive data imports.
fio --filename=testfile-seq-50g --size=50G --direct=1 --rw=write --bs=4M --ioengine=libaio --numjobs=10 --iodepth=32 --name=seq-write-test --group_reporting --ramp_time=4
WRITE: bw=31.0GiB/s (33.3GB/s), 31.0GiB/s-31.0GiB/s (33.3GB/s-33.3GB/s), io=369GiB (396GB), run=11894-11894msec
The result of 31.0 GiB/s demonstrates an exceptional ability to absorb data quickly.
Random Write Test (4k):This test evaluates the typical performance of a transactional database.
fio --filename=testfile-rand-4g --size=4G --direct=1 --rw=randwrite --bs=4k --ioengine=libaio --numjobs=10 --iodepth=32 --name=rand-write-test --group_reporting --ramp_time=4 --time_based --runtime=300
write: IOPS=50.1k, BW=196MiB/s (205MB/s)(57.4GiB/300001msec); 0 zone resets
The result of 50.1k IOPS indicates a very high ability to handle concurrent transactions.
6. Tangible Results: Exceeding Customer Expectations
Implementation of the new architecture produced results that exceeded initial goals:
- Decrease in data analysis execution time: 67% (Goal: 50%).
- Increase in traffic handling capacity during peaks: 274% (Goal: 200%).
- Improved page load time: 57% (Goal: 50%).
Other Key Benefits Achieved:
- Dramatically Improved RPO and RTO:Backups completed in fractions of the previous time, with the ability to restore quickly.
- Increased Resiliency: Geographically distributed architecture for production and backups.
- More Overall Security: Thanks to Cloudflare, dedicated firewalls and hardening best practices on Linux.
- Reduction of Indirect Operating Costs: Less time spent on managing performance issues or complex backup procedures.
- Future Agility and Scalability: The Proxmox platform allows you to easily scale resources or add new services.
7. Why This Solution? Strategic and Technical Considerations
- Proxmox VE & PBS: Chosen for their open-source nature, robustness and comprehensiveness, avoiding expensive proprietary software licenses.
- Dedicated Servers (Hetzner & OVHcloud): They offer the best price/performance ratio for intensive workloads and geographic separation for disaster recovery.
- ZFS over NVMe: Essential for meeting I/O performance goals and for data integrity (checksums, copy-on-write).
- Container LXC: For efficiency and density, reducing overhead and speeding deployments.
- Cloudflare:To delegate perimeter security and content acceleration to a specialist.
8. Conclusion: A Technology Partner for Growth and Innovation
This project demonstrates how careful architectural design, choice of appropriate technologies, and deep expertise in Linux, virtualization, and DevOps can radically transform a company's operational capabilities. This was not just a technology upgrade, but providing the client with a robust, resilient, and scalable platform on which to confidently build their future success in the competitive online betting market.
Developing Proxmox Customized Virtualized Web Solutions for Extreme Performance and Resilience
Are you ready to transform your infrastructure and unlock your application's true potential? If you manage demanding workloads, need high availability, or want to modernize your backup strategy, contact me for a free Proxmox Consultation. We will analyze your challenges together and design a tailored Proxmox solution, just like the one described in this case study.