System Design Series — Part 38

Imagine you're using Uber.

You request a ride.

Within seconds, your request reaches an available driver near you.

But here's the interesting question.

How did Uber know which driver service to contact?

In a modern microservices application, services are constantly being created, destroyed, scaled, and moved across servers.

Their IP addresses change all the time.

So how does one service find another?

The answer is Service Discovery.

It's one of the most important building blocks of a production-grade microservices architecture.

Let's understand it in the simplest way possible.

The Real Problem

Imagine an e-commerce application built with microservices.

It contains:

• User Service

• Product Service

• Inventory Service

• Order Service

• Payment Service

• Notification Service

Now suppose the Order Service needs to call the Inventory Service.

Should it store the Inventory Service's IP address?

What happens if the Inventory Service is moved to another server?

Or Kubernetes creates a new instance?

Or an auto-scaling event launches five new containers?

The stored IP becomes invalid.

The application starts failing.

Hardcoding service addresses simply doesn't work in distributed systems.

This is exactly why Service Discovery exists.

A Simple Real-World Analogy

Imagine calling your friend.

You don't remember everyone's phone number.

Instead,

you search their name in your contacts.

Your phone finds the latest number automatically.

Service Discovery works the same way.

Applications ask for a service name, not an IP address.

The Service Discovery system returns the correct address.

What is Service Discovery?

Service Discovery is a mechanism that allows services to find and communicate with each other automatically.

Instead of hardcoding IP addresses,

services register themselves with a discovery server.

Whenever another service wants to communicate,

it asks the discovery server for the current location.

The discovery server always knows which service instances are healthy and available.

Why Service Discovery is Needed

Modern applications use:

• Microservices

• Containers

• Docker

• Kubernetes

• Cloud infrastructure

• Auto Scaling

All these technologies create dynamic environments where service locations change frequently.

Without Service Discovery:

❌ Hardcoded IPs

❌ Frequent failures

❌ Manual configuration

❌ Difficult scaling

❌ High maintenance

Service Discovery solves all of these problems.

How Service Discovery Works

Step 1

A new service starts.

↓

Step 2

It registers itself with the Service Registry.

↓

Step 3

The registry stores:

• Service Name

• IP Address

• Port

• Health Status

↓

Step 4

Another service requests:

"Where is Inventory Service?"

↓

Step 5

The registry returns an available instance.

↓

Step 6

The services communicate successfully.

Service Registry

The Service Registry is a central directory that keeps track of every running service.

It stores information such as:

• Service Name

• Instance ID

• IP Address

• Port Number

• Health Status

• Metadata

Whenever a service starts or stops, the registry updates automatically.

Real-World Example

Imagine Amazon's checkout process.

When you click Place Order, multiple services work together:

Checkout Service

↓

Inventory Service

↓

Payment Service

↓

Shipping Service

↓

Notification Service

Instead of remembering IP addresses, every service discovers the others through the Service Registry.

This enables Amazon to scale thousands of service instances without changing application code.

Client-Side Discovery

In Client-Side Discovery:

The client asks the Service Registry for available instances.

Then,

the client selects one instance (often using load-balancing algorithms like Round Robin).

Flow:

Client

↓

Service Registry

↓

Service Instance

This approach gives clients more control but increases client complexity.

Server-Side Discovery

In Server-Side Discovery:

The client sends requests to a Load Balancer.

The Load Balancer queries the Service Registry.

It selects a healthy service instance and forwards the request.

Flow:

Client

↓

Load Balancer

↓

Service Registry

↓

Service Instance

This simplifies client applications and is widely used in cloud environments.

Health Checks

A discovery system continuously verifies whether services are healthy.

If a service crashes:

❌ It is removed from the registry.

Future requests are automatically routed to healthy instances.

This improves reliability and fault tolerance.

Popular Service Discovery Tools

Common tools include:

• Netflix Eureka

• Consul

• Apache ZooKeeper

• etcd

• Kubernetes Service Discovery

• AWS Cloud Map

Each provides automatic registration, health monitoring, and service lookup.

Advantages

✔ Dynamic service discovery

✔ No hardcoded IP addresses

✔ Easy horizontal scaling

✔ Automatic failover

✔ Better fault tolerance

✔ Simplifies microservices communication

✔ Works well with containers and Kubernetes

Challenges

Single Point of Failure

The Service Registry itself must be highly available.

Registration Delays

New service instances must register before receiving traffic.

Additional Infrastructure

Running a Service Registry introduces another component that needs monitoring and maintenance.

Production-Level Insight

In Kubernetes,

developers rarely think about service IP addresses.

Pods are created and destroyed constantly.

Instead,

applications communicate using stable Service DNS names like:

• product-service

• order-service

• payment-service

Kubernetes automatically resolves these names to healthy Pods behind the scenes.

This is built-in Service Discovery.

Interview Tip

A common System Design interview question is:

"Why do microservices need Service Discovery?"

A strong answer should explain:

• Service instances are dynamic.

• IP addresses frequently change.

• Services register themselves automatically.

• Other services discover them using a registry.

• Health checks ensure traffic reaches only healthy instances.

Mentioning Kubernetes or Eureka demonstrates practical production knowledge.

Key Takeaways

✔ Service Discovery allows services to locate each other dynamically.

✔ It removes the need for hardcoded IP addresses.

✔ Services register themselves with a central registry.

✔ Health checks prevent traffic from reaching failed instances.

✔ It is essential for cloud-native and microservices architectures.

✔ Kubernetes provides built-in Service Discovery using DNS and Services.

One of the biggest lessons in System Design is this:

In distributed systems, service locations are temporary—but service names remain stable.

Service Discovery bridges that gap, enabling scalable, resilient, and self-healing microservices.

This is Part 38 of the System Design Simplified series.

Next Article: Part 39 — API Gateway Explained Simply

#SystemDesign #Microservices #ServiceDiscovery #Kubernetes #Docker #CloudComputing #SoftwareArchitecture #BackendDevelopment #DistributedSystems #SystemDesignInterview #DevOps #CloudNative #SoftwareEngineering #TechArchitecture