As the digital landscape evolves, the demand for robust question-and-answer platforms continues to rise. Understanding the underlying architecture of such platforms, like Quora, is essential for engineers and developers aiming to create scalable and efficient systems. In this exploration, we dissect the functional and non-functional requirements, building blocks, workflow, and potential limitations of a Q&A site.
Functional Requirements
When designing a Q&A site, we only consider the following features and also ensure that, in the future, we can extend the features.
Questions and answers functionality
Commenting to the questions along with upvote and downvote capabilities
Recommendation system for personalized home feeds and advertising
Ranking mechanism for answers
Non Functional Requirements
The site should have the following characteristics due to the large number of users all over the world,
Scalability: The architecture should accommodate additional features and support a growing user base seamlessly.
Consistency: The questions and answers should be consistent with any set of users. This does not imply, that the newly added Q&A should be available to all the users right away.
Availability: The system should be highly available. It should handle a large number of concurrent requests and also be able to perform in case of a few server failures.
Performance: The system should serve the user without noticeable delay.
Resource Estimation
Consider 300 million active users. Each user will make 20 requests per day. With these assumptions, we are going to calculate the,
Servers
Each server can handle 8000 requests per second
Total request per day, TOTAL_REQUEST_PER_DAY = (3x10^6) * 20
Requests per second, TOTAL_REQUEST_PER_SECOND = (TOTAL_REQUEST_PER_DAY) / (24 60 60)
Number of servers required (TOTAL_REQUEST_PER_SECOND / 8000)
Storage
15% Q&A will have the image
5% Q&A will have video
For a Q&A there will be either an image or video or nothing. Both video and image can not be in a single Q&A
Consider 1 question from each active user
Two responses from each question
Calculate the storage
Let's assume
Image Storage
Video Storage
Text Content
Each of the day, we will require 11 + 75 + 30 = ~116 TB of storage
Bandwidth
Incoming Bandwidth
Outgoing Bandwidth
Consider 300 million active user
Consider each user sees 20 questions each day
Consider that 15% of Q&A has an image
Consider 5% of Q&A has video
Now total size of data: 600 + 225 + 1500 = 2325 TB ~ 2500 TB ~ 2500000 GB ~ 20000000 Gb
Outgoing bandwidth 20000000 / (24 hours in seconds) = 231 Gb/s
Building Blocks
Load Balancer: Distribute traffic between servers and services
Database: Store textual content in the DB
Distributed Caching: Schedule tasks and reduce loads to db and services
Blob Store: Store the images and videos
Web and Application Server
To handle requests
Application servers maintain an in-memory queue to process different user requests
A router library between the web and application server
Manager process enqueued the tasks
Application process dequeued the tasks
Data Stores
A relational database MySQL for storing Q&A and comments, as it has a high level of consistency
An hBase to store metadata as it has a very high throughput in storing and retrieving data. Use these
Stats for recommendation later
Blob storage for the images and videos
Distributed Cache
memcacheD for critical data caching
Redis for upvote type data as it has in-store increment
CDN for serving images and videos
Computer Servers
For recommendation engine or ranking
The process will be online and offline
Probably running some ML operations
May have lots of memory and high processing power
Workflow
Posting Q&A and comments
The web server receives the request and passes it to the application server
Web servers also manipulate the web page like the request is in progress
The worker process will manipulate the database, ex: fetch or save data
The task will be prioritized by different queue
Images and videos will be stored in the blob storage
The answer ranking system will rank the answers based on upvotes, views, dates, and some other properties. An ML engine will back it up.
Extract and store metadata of answers, comments, images, and videos in hBase and feed these metadata to ML and rank offline
Recommendation System
Search
Build index and store in DB and keep frequently used in hBase
Make an index by tokenizing from Q&A, level, and comments
Limitations
Latencies of web and application servers: Latency of web and application servers communications.
In Memory Queue Failure: Tasks are queued in the queue. If a queue is failed, a lot of manual engineering will be required. Replicating the queue can be a solution but it will require extra memory. Tasks like view count should not hamper comparatively more important tasks like saving answers or questions.
MySQL QPS: Since we offer a lot of services, it is possible to encounter a lot of queries in our MySQL server. This will result in a huge latency in getting the query results.
HBase Latency: Although the HBase has a high throughput, it has slow latency. On top of that, since, we rely on the ML, at one point, it will have a poor performance.
Adjustment for the mentioned limitations
Latencies of web and application servers: Using a service host. A large powerful machine to handle all web and application processes at once in a single place.
In Memory Queue Failure: Use Kafka instead.
MySQL QPS: Use vertical sharding. If there are joins involved between two tables, put them in the same shard.
hBase Latency: Use MyRocks instead. Allows improved latency and data transfer tools between RocksDB and MySQL.
Conclusion
Understanding the intricate system design of a Q/A site like Quora offers invaluable insights into creating scalable, efficient, and reliable platforms in the digital age. By addressing functional and non-functional requirements, designing robust building blocks, establishing efficient workflows, and implementing mitigation strategies for potential limitations, developers can craft high-performance systems capable of meeting the demands of millions of users worldwide.
