Showing posts with label microservices. Show all posts
Showing posts with label microservices. Show all posts

Thursday, March 21, 2019

Remote Procedure Call, gRPC, Protocol Buffers

Let me say to be honest "I had no idea of the impact of RPC in Distributed Systems" until I watched the UW session on Distributed Systems on Youtube. I asked myself, why is it that that RPC is a module in itself. Why every other place it is dedicated lecture. I couldn't understand it before.

But now that I understand that it is a foundation for creating a truly distributed system, I want to bring in the resources which I felt are important to learn this topic.

What benefits do you get from using Protocol Buffers over other contemporary techniques? Specially when it comes to System Design and Architecture, what leverage do I get here that I cannot find elsewhere?


First, let's learn about Why RPC?
<Fill in the details here>

Second, why Protocol Buffers?
<Fill in the details here>


Third, Implement gRPC and ProtoBuf in Python

Protocol Buffers Basic using ToDo Project


Creating a Server and Client Code using above lecture material and gRPC.

This is a good place to start understanding what is gRPC.
https://grpc.io/docs/guides/



Intro to gRPC in C# 4/19/2020
I actually liked the tutorial because it was more hands on and the others I posted before.

Here's one article that I think could handy as well : https://medium.com/@karuppiah7890/how-i-started-learning-grpc-and-made-a-grpc-client-in-nodejs-and-grpc-server-in-golang-cc7e3901ed54

Sunday, December 17, 2017

System Design : Message Broker




RabbitMQ : A 5 min walk through.

High Scalibility Blogs



Plenty of Fish
http://highscalability.com/plentyoffish-architecture

A small calculation on storage I scribbled go like this:

  • Each User has avg 5 pictures of .5 MB each. = 2.5 MB of Data Per User. ~ Not counting Textual Data.
  • That means a 10TB Disk can host 4 million users. I think it is just amazing how storage is cheap. ~ 400$ Amazon Link   . This really opened my eyes when I think of how insignificant hardware cost is when compared to scale of application.

ButterCMS

I think article does a great job in going to details about where they made mistakes and what they did to overcome that. I personally wish to print it out and read it in more depth as I find it very valuable. I guess it is more important to understand one system design better than reading 100's. 

Friday, December 15, 2017

My Frustrations with System Design - Try2

Prologue : The aptness of this title is known to me only after I lost my first draft of this article by accident. Sure enough it is frustrating but let's focus on my deal with Software System Design. The second writeup will not have any dramatic entries I had in my first one but focus on MVP.

The knowledge of Big Data System Design has many components which can be happily group into various categories. My plan is to write one blog per category so that I understand them in more detail. This is to help me store all my learning at one place and help as a quick guidebook when times comes for Job Interviews or Project Execution.

The various categories that come to my mind so far are:-
  1. Data Orchestration : Hbase, Flume, Zookeeper
  2. Search : Solr, Elasticsearch
  3. Data Access :  Pig, Hive, Sqoop
  4. Hadoop : HDFS, MapReduce, Yarn
  5. Spark : Stream Processing 
  6. NoSQL Database : MongoDB, Casandra
  7. Load Balancers : HAProxy, Nginx
  8. Message Broker : RabbitMQ, Kafka
  9. High Availability & Scalability
  10. Container Orchestration Engine : Mesos, Kubernetes or Swarm


Monday, November 27, 2017

Sunday, November 26, 2017

It's OK to be NOT-OK.

02/22/2021  Last week I was just un-productive and was not doing enough to talk about. I also wanted to say that weeks before the last, I ha...