- Published on
System Design Fundamental Concepts
- Author
- Name
- Robert Kirby
- Title
- Lead Full Stack Developer at Y-Squared
Blog Post 5
System Design Fundamental Concepts
1 Introduction
2 Client-Server Model
- 2.1 Client & Server
- 2.2 IP Addresses & DNS
- 2.3 Ports
3 Network Protocols
- 3.1 IP - Internet Protocol
- 3.2 TCP - Transmission Control Protocol
- 3.3 HTTP - Hyper Text Transfer Protocol
- 3.4 HTTPS - Hyper Text Transfer Protocol Secure
4 Storage Overview
5 Latency And Throughput
6 Availability
7 Caching
8 Proxies
- 8.1 Proxies
- 8.2 Reverse Proxies
9 Load Balancers
10 Databases
11 Replication And Sharding
12 Leader Election
13 Polling and Streaming
14 Rate Limiting
15 Logging
16 Contact, Socials & GitHub
This article is also available to read and in audio form on Medium
1 Introduction:
Chart source:https://hackernoon.com/10-tips-for-using-diagrams-to-ace-the-system-design-interview-906p3609__
In this article we will be covering a broad range of system design concepts! We won't be covering everything in system design but this is a good high-level overview to start with as your foundation
2 Client-Server Model:
Chart:https://currentmillis.com/standard/specification/client-centric-time/
The client server model is the foundation of the modern internet. We use this technology everyday, in fact we are using it right now !
2.1 Client & Server:
The client is a machine that requests data or a service from a server and the server is another machine that provides data or a service for clients by listening for incoming network calls.
An example of this would be when you enter Y-Squared.Blog to view this article your browser is acting as the client and it requests the page data from the Y-Squared server.
2.3 IP Addresses & DNS:
An IP address is given to each machine that is connected to the public internet and DNS(Domain Name System) is like the phonebook of the internet, basically DNS translates domain names like Y-Squred.Blog to IP addresses of machines so browsers can load the data.
An example of how this works is when you write Y-Squared.Blog into the url bar, the browser makes a DNS query and goes to a specific name server associated with that domain to ask what is the IP address of the server where Y-Squared.Blog is located and then it gives back the IP address.
When the browser gets the IP address, it makes a HTTP or HTTPS request to the servers.
We will cover HTTP/HTTPS in detail later but essentially it is a packet of data your client sends to the server with the requested information in. It also contains your device or browser IP address called the source address, this is so the server knows what IP address to send the response back to (IP addresses are usually granted by the IP provider).
2.5 Ports:
A server usually listens for requests on specific ports. This allows multiple programs to listen for network requests on the same machine without colliding.
Any machine that has an IP address has more than 60,000 ports. So as a client when communicating with a server we need to specify the port. Think of the IP address is the mailbox to an apartment complex the port is the apartment numbers
Most clients know the port depending on the protocol we are using for example in the port 80: HTTP port 443: HTTPS.
3 Network Protocols:
What is a protocol ? A protocol is just a set of rules agreed on by two parties.
3.1 IP - Internet Protocol:
The internet effectively runs on IP, all other protocols are built on top of this, when one machine sends data to another it does so in the form of IP Packets.
IP Packets or network packets are made up of bytes(smallest unit of memory) and contain two main sections: the IP header (Contains source and destination IP addresses) and the Payload (Contains the data).
There are two main versions of IP, Internet Protocol Version 4 (IPv4), is the dominant protocol of the Internet and Its successor Internet Protocol Version 6 (IPv6).
Internet protocol alone has issues, it is limited by size so most pieces of data are split into many different packets and sent across the network, IP alone cannot guarantee packets are delivered error free and read in an ordered fashion. This is where TCP comes in.
3.2 TCP - Transmission Control Protocol:
TCP or Transmission Control Protocol is built on top of the internet protocol and was designed to solve the issue above. TCP makes sure all the data packets are in order and are error free
Inside the IP packet payload we have a TCP header, this contains important info about the ordering of packets.
Before the client and server send data to each other, they will create a TCP connection by sending packets to one another confirming the connection through something called a handshake. TCP handshakes usually have a window of time to send data after the connection and will timeout otherwise. Either party can also cut the connection by sending a special packet.
TCP is a wrapper for IP packets but it only contains abirtrey data. It lacks a robust framework that is well defined with easy to use communication channels for developers to use, this is where HTTP comes in.
3.3 HTTP - Hyper Text Transfer Protocol:
HTTP is built on top of TCP and introduces the request response paradigm. The request response paradigm allows us to easily build robust and easy to maintain systems, where machines can communicate with each other easily.
When dealing with HTTP we forget about IP and TCP and only focus on HTTP Requests and Responses.
In summary Where IP and TCP are quite low level and just send data, HTTP allows us to introduce business logic in our systems!
3.4 HTTPS - Hyper Text Transfer Protocol Secure
Note:We will not be covering encryption in this section or article as it deserves its own article so we will be just taking in a high level overview of HTTPS
HTTPS is an extension of HTTP that we use for secure communication online, it requires servers to have trusted certs (SSL certificates) and uses TLS (Transport Layer Security). TLS is a security protocol built on top of TCP and is used to encrypt data between a client and server.
An SSL Certificate is granted by a certificate authority(A trusted entity that signs digital certs) and it contains the public key that is used in a process called the TLS handshake.
The TLS Handshake is the process in which a client and a server communicate with each other to establish a secure connection.
HTTPS was developed to defend against a Man-In-The-Middle-Attack. The best way to explain MITMA is by example let's say you are on a website and you purchase something with your credit card and the website is only using HTTP. Somebody could intercept that data while it’s going from the client to the server and just read your card details in plain text.
On the other hand HTTPS encrypts this information and only the other trusted party can decrypt it with a key, so if the attacker intercepted your data they would only see ciphertext which is essentially useless without the key to decrypt it.
4 Storage Overview:
Image source:https://unsplash.com/photos/aWslrFhs1w4
Database:A database is just a program on a server somewhere that allows us to do two things; write data and read data on disk or memory.
Persistent Storage: Persistence Refers to a form of storage that will keep the data even if the program managing it dies.
Memory or RAM - When you are running a program Random Access Memory (RAM) is used to store things and retrieve them quickly while a program is running it is not persistent when a program dies it is lost.
Hard Drive Storage - Writing or storing memory on a HDD(Hard-disk drive ) or SSD(Solid-state drive) is persistent. When the program managing it dies it's still there.
5 Latency And Throughput:
Diagram source: https://kinsta.com/blog/network-latency/
Latency is the time it takes for data to get from one point in a system to another point.
We would describe the time it takes for a network request to complete from a client to a server as latency or the time it takes for a program to read data from memory or a hard drive disk.
When designing a system you want to try to optimize for latency but there is usually a trade off between speed and the system requirements for the project.
Throughput is the number of operations a system can handle in a given amount of time ie. RPS (Requests Per Second) or QPS (Queries per second).
For example the throughput of a networks is stated like this 150Mbps (150 Megabytes of data per-second) or 1Gbps(1 Gigabyte of data per-second)
6 Availability:
Image source:https://www.marketing91.com/wp-content/uploads/2018/07/Uptime-Monitoring-1.jpg
Availability is the odds of a server being up and running at any time.
We measure availability in the amount of downtime systems have. Downtime per year is the most important data metric we go by. We only measure availability in high percentages because low percentage downtime is not something that should be on any application.
For example in the case of critical software where uptime could mean life and death 99% availability is actually very poor performance, over a year it would mean 3.65 days of downtime which is unacceptable.
We usually measure availability using the nines system - 5 nines or more is classes as high availability, the nines system goes all the way up to nine nines!
Nines over a year:
- 99% - Two 9’s - 3.65 days downtime
- 99.9 - Three 9’s - 8.77 hours downtime
- 99.99 - Four 9’s 52.6 minutes downtime
- 99.999 - Five 9’s - 5.26 minute downtime
- 99.9999999% - Nine 9’s - 31.56 millisecond downtime
Creating higher availability and fail safes will usually incur some cost so depending on what we are building we need to categorize what parts of our system need to be highly available.
For example if we built a Web App with that sells thousands of products per day but also has a blog where we talk about upcoming products, it's important that the store and payment system have high availability because if it goes down it will cause major problems with losses in revenue and angry customs, but if the blog goes down it's not that big of a deal as it's not critical to business operations.
We create high availability by creating redundancy in our system. This is usually done by avoiding creating single points of failure, we can defend against this by using a distributed system with multiple versions of our code across many servers.
7 Caching:
Image source:https://www.enjoyalgorithms.com/blog/caching-system-design-concept
A cache is a software (or hardware) component of a system that is used to store frequently used data.
The most common type of caching we are all familiar with is caching of HTML pages, images and files of websites on our browsers.
This means we don't have to request data from the database every time instead we check the cache first for the data, it improves speed and efficiency.
This is not the only type of caching available, caches can be placed all over our application:
- Web Caching (Browser/Proxy/Gateway)
- Data Caching
- Application/Output Caching
- Distributed Caching
Caching is limited by memory and so we use cache eviction policies to make space:
- LRU - Least recently used
- FIFO - First in first out
- LFU - Least frequently used
8 Proxies:
Chart source: https://www.upguard.com/blog/proxy-server
8.1 Proxy:
A proxy (Forward proxy) is a server that sits between a client and server, a proxy acts on behalf of the client.
If a client wants to communicate with a server instead of going directly to the server it goes to the proxy and then goes to the server. When the server responds it goes back through the proxy then back to the client.
A proxy can serve as a way to hide the client from the server. A VPN works this way but it also encrypts your data.
8.2 Reverse Proxy:
A reverse proxy also sits between the client and the server, but it acts on behalf of the server.
So when a client sends a request to the server it actually goes to the reverse proxy instead and then goes to the server. The request then goes back the same route.
A reverse proxy can be very useful in system design ranging from security(eg. filtering requests, rate limiting), caching(eg. store HTML pages) as well as acting as a load balancer.
9 Load Balancers:
Chart source:https://www.nginx.com/resources/glossary/load-balancing/
A load balancer does exactly what it says in its name, it balances load! (Like we previously mentioned we can use reverse proxies as our load balancers!).
Before we get into load balancers we should understand scaling our infrastructure.**We can scale vertically meaning we increase the power of our server this could be adding RAM to give our single server more power**or we can scale horizontally which means adding more servers.
Note:__We usually scale horizontally in almost all cases, especially if we are dealing with a platform of growing users! An example of when we might scale vertically is if we are using a single program like a complex machine learning algorithm that needs more power to run on a single device.
So let's say for example our business is scaling and we now have four servers that do the same thing, we would add a load balancer in between our client requests and our servers because no one server can handle all the traffic.
Using our load balancer we can distribute traffic across our servers evenly or depending on the power of each server and ensure any single server won't get overloaded due to our requests.
A load balancer is not just reserved for client to server requests, they can be placed throughout the system, we can also have multiple load balances in the same location to stop the load balancers from getting overloaded as well this helps with our apps availability!
10 Databases:
Chart source: https://miro.medium.com/max/1400/1*oupfNXy-3ud6VjFghWEM5w.png
There are many different types of databases out there that are optimized for different use cases but they are generally split into two main categories based on their structure, these are relational databases and non-relational databases.
Relational databases are structured databases that follow a table structure and impose a very strict structure on the data. Most relational databases support SQL (Structured Query Language ) so relational databases are often just called SQL Databases even though not all of them do!
Non-relational databases are any databases that are databases that do not have a tabular-like structure and are often called NoSQL databases.
There is a lot to cover on databases so I'm planning on writing a stand-alone comprehensive article on databases in the near future that I will link here when it's done!!
11 Replication And Sharding:
Sharding example diagram:https://www.digitalocean.com/community/tutorials/understanding-database-sharding
Replicationis when we duplicate data from one database server to another, this increases the redundancy of your system incase of a failure and also allows us to decrease latency by moving the data closer to clients in different regions !
Sharding, also called data partitioning, is when we spit the database into multiple parts called shards! We might also shard based on the type of data being sent.
12 Leader Election:
Diagram source: https://patterns.arcitura.com/microservice-patterns/design_patterns/leader_node_election
Leader election what is it ? In a nutshell, When we have multiple servers in a distributed system that can all carry out the same task, we can use this process called leader election where the servers will elect one server to carry out a primary operation.
To do this we must use a consensus algorithm to elect the server, the consensus algorithm must also be able to elect a new leader if for instance the current leader cant carry out the operation.
Like hashing algorithms we would rarely write these ourselves as there are already very robust consensus algorithms out there for us to use like Paxos and Raft.
13 Polling and Streaming:
Diagram source:https://www.geeksforgeeks.org/polling-and-streaming-concept-scenarios/__
Polling and streaming are two techniques we can use when we need to give data to our clients.
Polling is the act of fetching data on regular intervals to make sure the data does not go stale, a real life example of this would be updating price information on a company stock chart every 5 seconds.
Streaming is the act of continuously receiving data by keeping an open connection between two parties. An example of this would be a live stream of a video.
14 Rate Limiting
Chart source:https://blog.getambassador.io/rate-limiting-for-api-gateways-892310a2da02
Rate limiting is the act of limiting the amount of requests coming in from a client to a server. You can set up rate limiting by configuring some business logic on your reverse proxy that sits between the client and your server.
We mostly use rate limiting in order to prevent DoS or DDOS attacks. We can enforce this in many different ways including by blocking the source IP address, User accounts or regional requests.
DoS or Denial-of-Service Attack is when someone tries to bring down your system by overloading the server with requests.
DDoS or Distributed Denial-of-Service attack is the same as DoS but instead of the requests coming from one source, they are coming from many different sources making it harder to defend against.
15 Logging:
Logging is the act of collecting and storing useful information about events that happen in our system. We can use logging monitoring to track key metrics or alert admins of critical system errors.
16 Contact, Socials & GitHub:
Please contact us if you see any mistakes on this post, have any suggestions or business inquiries.
Please fill out the form on https://y-squared.com/contact or email us at team@y-squared.com.
The Author:This post was written by Robert from Y-Squared:
Click here to follow me on Github @Rob1818
Blog Site:
Medium:(Follow me on medium)
https://medium.com/@Robert-Y-Squared
_[_This article is also available to read and in audio form on Medium](https://medium.com/@Robert-Y-Squared/system-design-fundamental-concepts-df3f35d6dd7b "https://medium.com/@Robert-Y-Squared/system-design-fundamental-concepts-df3f35d6dd7b")
Business Site:(Fill in the form if you have any business inquires)