What does Kafka do?

Tram Ho

At the beginning of this series of articles, I also introduced the LinkedIn problem and the specific requirements clearly

And just like last time, please prepare yourself a cup of tea, sit down and enjoy it to wake up and increase your knowledge about Apache Kafka

To answer the title of the article, let’s learn from the first brick to build it. After starting to work and up to now, the gods in the Kafka development team have built “Architecture Style” for it as follows:

image.png

Kafka is designed and deployed according to a distributed model consisting of Servers and Clients interacting with each other via TCP protocol:

Servers one or more personal computers, virtual machines in an on-premise system or cloud server servers with Kafka installed called Broker connected together into a cluster, so we call it Kafka Cluster and advertised as after:

  • High scalable (easily expandable) ie you just need to install Kafka on your machine and configure it to add it to the cluster is ok done .
  • High reliable : I temporarily explain it as after a not very good day outside, there is a storm level 11 12 big waves and strong winds, inside the house the wind blows cold dev or System Admin after a while Tired working day come home, just want to go to bed soon, wrap a blanket as a “nest” for me to sleep, squeeze my tongue, then a shark or a rat with itchy teeth jumps in to bite something to make a server inside. Cluster cluster falls unconscious, Alert is firing loudly in vain, other machines in that cluster will take over the work of the server that has “faded” and ensure continuous operation without data loss. Whether .
  • High durable : Events are stored on the hard drive for the time you want.
  • High performance (high throughput): Broker can handle TBs of data without much performance issues (yeah great man sun)

(Exactly as required by the topic )

The client simply understands here that your Applications / Services are written in languages ​​such as Java, Go, Python…. using libraries (Open Source or Paid) that are community or a organization created to support connecting and interacting with the Kafka servers in the Cluster mentioned above.
The interaction here can be to produce, consume or process stream events to the Kafka Cluster

And here, the title question of the article has been partly answered

I summarize the combination of 3 main functions as follows:

  1. Publish (write) and Subscribe (Read/receive) event
  2. Process event streams when an event occurs (when an event is published to the Kafka cluster).
  3. Store the event (Event) with the time you want.

It sounds quite confusing but that’s okay, that’s an overview that I go into detail and specificity one by one.

Please read my detailed articles.

Thank you guys very much.

Share the news now

Source : Viblo