Keep the performance of your backend system being stable is such an interesting challenge for software engineer. Assume that your application growth exponentially with about 10K — 50K volume concurrent request, what’s your strategies to keep stable?
1. Optimize Backend System
Put your attention on:
– Connection Pooling
– Optimize queries
– Caching
– Queues
– Asynchronous Processing
2. Database Optimization
– Indexing
– Sharding & Replication
– Caching
3. Asynchronous Processing
Move time consuming task by using Message Queue, Background jobs etc
4. Scale Horizontal and Use Load Balancing
Distribute the traffic across multiple server instance, auto-scaling strategy is also recommended to implement on this case
5. Implement Rate limiting
Prevent abuse by setting limit on the number request of IP Address
6. API Gateway & Caching
Handling request routing & caching
Using nginx, memory caches like Redis
7. Cluster & Distributed system
Implement service oriented or micro services architecture
What’s the benefit of service-oriented architecture:
1. Scalability, it’s easier to build scalable system
2. Modularity, breaks down complex application into smaller and more manageable components
3. Interoperability, it’s easier to integrate system that provide related functionality
4. Adaptability, adapt to changing business needs
8. Scaling Database
Deploy read replicas for heavy read operations (Master & Slave)
Use database sharding to split large database across multiple server
By combining those strategies above, you can build your robust backend system that handle that huge request volume. If you have any advise based on your experiment for this case, please leave on comment.