Distributed Computing: Core Concepts
Distributed computing systems enable coordinated actions across multiple independent nodes, utilizing message passing and leader election for efficient communication, fault tolerance, and decision-making, despite inherent challenges like scalability and security.
Core Principles
- Message Passing: The fundamental communication method in distributed systems, enabling nodes to exchange information for coordination, synchronization, and data sharing.
- Synchronous vs. Asynchronous Message Passing: Synchronous ensures sender-receiver synchronization, while asynchronous allows concurrent execution.
- Leader Election: A critical process for selecting a coordinator node to manage operations, simplify decision-making, and enhance fault tolerance and scalability.
- Consensus: Achieving agreement among nodes, especially in the presence of faulty nodes (crash or Byzantine), is essential for reliable decision-making.
- Distributed Models: Systems can be structured using physical, architectural, or fundamental models to define hardware, interactions, and overall design.
Action Steps
- Implement message passing protocols (like TCP or UDP) to enable communication between nodes.
- Choose between synchronous and asynchronous message passing based on system requirements for synchronization and concurrency.
- Employ leader election algorithms (e.g., Bully, Ring) to designate a coordinator for decision-making and task management.
- Utilize consensus algorithms (e.g., Paxos, Raft) to ensure agreement among nodes, especially in fault-prone environments.
- Design systems with fault tolerance and security in mind, addressing potential failures and threats.
Key Terms
- Message Passing: The communication method used by nodes in a distributed system to exchange information and coordinate actions.
- Synchronous Message Passing: A communication model where the sender waits for the receiver's confirmation before proceeding, ensuring tight synchronization.
- Asynchronous Message Passing: A communication model where the sender continues execution without waiting for the receiver's response, allowing for concurrent operations.
- Leader Election: A process where nodes in a distributed system select a single node to act as a coordinator for decision-making and management.
- Consensus: The process by which a group of nodes in a distributed system reach an agreement on a single value or decision, even with faulty nodes.
- Bully Algorithm: A leader election algorithm where the node with the highest identifier initiates an election if it detects a leader's absence.
- Ring Algorithm: A leader election algorithm where nodes are organized in a logical ring, and election messages are passed to successors.
- Paxos: A consensus protocol designed to achieve agreement among nodes, ensuring safety and liveness.
- Raft: A consensus protocol designed for leader election and log replication, focusing on simplicity and understandability.
- Byzantine Fault: A fault where a node behaves maliciously or unpredictably, potentially sending different values to different nodes.
- Logical Clocks: Timestamps assigned to events in a message-passing system to provide a notion of time and order.
Real World Examples
- An e-commerce platform processing orders.: Asynchronous message passing is used by the order system to send payment requests to a payment processing service, allowing it to continue processing other orders without waiting for immediate confirmation.
- A stock trading app.: Synchronous message passing is used when a user wants to buy or sell shares; the app sends a request to the trading server, waits for confirmation, and then allows further actions, ensuring transaction accuracy.
- Distributed databases (e.g., Apache Cassandra, MongoDB).: Leader election guarantees data consistency and fault tolerance, with the leader overseeing transactions, consistency checks, and data replication.
- Cloud computing platforms (e.g., GCP, AWS).: Leader election is used for resource management and orchestration, managing functions like load balancing and auto-scaling.
- Messaging systems (e.g., Apache Kafka, RabbitMQ).: Leader election is used to control message brokers and provide high availability, managing message routing, segmentation, and replication.
Related cheat sheets
More like this