Node.js Microservices: Build Scalable and Efficient Applications

Once your applications begin to grow and surpass a certain threshold, it is time you settle on an architecture that will make adding new features and fixing existing ones easy to do and without disruptions to other parts of your web applications. Furthermore, the service-oriented architecture should easily scale out and meet the varying demands of the digital product life cycle.

With Node.js microservices, there is this exciting way of breaking down applications into smaller, manageable services, where each caters to a particular need. Each microservice then can be independently built, deployed, and managed, which shortens software development lifecycles.

Node.js is a good option for building microservices, with its lightweight and event-driven architecture, delivering the speed and scalability that modern web application development require.

What are Microservices?

One of the most widely used architectural patterns in developing modern applications is microservices. This technique produces smaller and independent services designed to provide a single functionality of an application. They are usually considered diametrically opposed to monolithic applications since they have only one codebase for all their functionalities.

Why Choose Node.js to Develop Microservices?

Community Support and Huge Resources: Node.js has a huge community and huge developer resources that allow your engineers to find the right tools and build microservices apps.

Seamless Integration with JavaScript Ecosystem: Node.js plays nicely with a range of popular frontend frameworks, and because it’s written top-to-bottom in JavaScript, your engineers can work on the frontend and backend development simultaneously.

Cross-Platform Compatibility: Node.js Architecture supports multiple operating systems and, therefore, is a great candidate for different contexts of deployment.

Insight: In the survey conducted by Statista, more than 40% of developers are using Node.js for developing highly scalable microservice-based applications.

Monolithic vs Microservices in Node.js

There are two fundamental ways of building backend applications, namely monolithic and microservices. Each of these methodologies provides advantages and some disadvantages when targeted for development using Node.js. Let’s compare both ways side by side for understanding Microservices architecture and Monolithic Architecture. 

Monolithic vs Microservices Architecture – Which is Best?

Before we start, why should you care? System design decisions are important because it’s hard to reverse. Careful consideration is needed to ensure you’re meeting your system’s requirements.

Monolithic vs microservices… pic.twitter.com/IqeEdnCJo6

— Chris Staudinger (@ChrisStaud) November 21, 2023

Key Benefits of Using Node.js for Microservices

By using Node.js in microservice architecture, the following wondrous gains can be obtained: 

High Performance: Node.js is built on an event-driven, non-blocking I/O mechanism that allows for the smooth processing of several incoming requests at once. This non-blocking architecture thus supports highly performance-oriented applications dealing with enormous volumes of real-time data.

Lightweight and Fast: A significant benefit of Node.js is lightweight, enhancing load times and response rates across microservices without any tuning.

Large Community and Ecosystem: Node.js features a heavy ecosystem of open-source libraries. In fact, it also hosts one of the most robust and active software developer communities, which makes NodeJS development Services pretty quick and smooth.

Efficient Resource Management: The application of Node.js is good for efficient management since it is single-threaded, lessening the overhead of managing threads and context. It also benefits when it comes to cloud microservices environments.

Rapid Prototyping: Node.js for MVC apps allows quick prototyping and the production of MVPs faster, meaning testing and iteration cycles will get done faster. This is especially helpful for teams who continually explore and test new ideas.

Seamless JSON Handling: Node.js has JavaScript, making it easy to process JSON. In addition, connecting it to the front-end apps simplifies data interchange in microservices with Node.js.

Cross-Platform Compatibility: With Node.js Development, deployment can be done on several platforms, ranging from Windows to Linux, since you will have to concentrate on micro-services design instead of deployment-related issues.

Easy Scalability: It is easy to scale Node.js horizontally based on the demand of each microservice. Consequently, this fosters dynamic scaling and flexibility in apps.

Native API Support: Node.js naturally supports REST APIs and also has support for GraphQL, which allows microservices to communicate and exchange data with ease.

Less Development Cost: Since the codebase is shared for both the frontend and backend (in JavaScript), Node.js reduces the learning curve and the development cost of your projects since you end up requiring fewer developers to build and operate the application.

Basic Building Blocks of Node.js Microservices Architecture

There are several diverse individual components of Node microservice architecture. So let’s have a look below at them:

Service Registry and Discovery

A service registry keeps track of each microservice and its locations. Furthermore, each microservice has to register itself with a service registry. This is an essential thing when you build large-scale systems where multiple services need to locate each other dynamically based on need.

API Gateway

A Simple API Gateway is typically a central hub from which all the client requests route to various different services. It makes real-time communication from clients to multiple services much easier and provides a single external entry point for all client requests. API gateways also handle load balancing across instances of a service, rate limits, and throttling requests.

Load Balancer

The name indicates that a load balancer balances the incoming request load and redirects these to various instances of the microservice. This will ensure optimum performance and prevent any single instance from becoming a bottleneck.

Data Management

In a microservice architecture, each service should be responsible for its data and database. This way, you will keep data for a service connected with it, apart from allowing data storage solutions to be customized to meet each service’s needs.

Practical Implementation Steps with Tools for Node.js Microservices

If your aim is implementing microservices with Node.js, an approach needs to be derived considering the right set of steps as well as tools. In the section to come, let’s explore a step-by-step guide to develop, test, and then deploy microservices with Node.js.

Step 1. Set Up the Basic Environment

Set up the Development Environment First download Node.js and npm from the page, and install Node.js along with npm on your local system. Additionally, for your individual service, you need to create a new project with the use of npm.

Install Node.js and npm: Download and install Node.js (v16+) and npm.

Initialize a Project: Run the npm init command to create a package.json file. This will be your dependency manager for the microservice project.

mkdir microservices-project

cd microservices-project

npm init -y

Step 2. Define Microservices Boundaries

Now, it is time to define boundaries for your microservice by breaking down the application for the specific functionality.

Service Identification: Identify responsibilities that can be chunked out to be developed as independent services from the monolithic application being in existence. Examples may include User Service, Order Service, and Product Service.

Definition of Responsibilities: Each of the services must be responsible for just one functionality; this NodeJS best practice should not be deviated from. For example, the User Service may be able to handle only authentication and user data; it must not be used to render the products.

Step 3. Set Up Each Microservice

How you set up the directory structure also plays an important role in how easily you can develop, test, and deploy your microservice project. In this step, you should create the directory structure and set up the best Node.js framework to be used in the app. 

Directory Structure: Create individual folders for each microservice in your architecture to isolate dependencies and configuration files and build them independently.

Express Framework: Use express framework in each microservice to handle incoming and external HTTP request. You can Install it by running the following command:

npm install express

Create Basic Server: Once you have installed the Express framework, you can start a basic express server for each microservice through the following code:

const express = require('express');

const app = express();

const PORT = process.env.PORT || 3000;

app.listen(PORT, () => console.log(`Service running on port ${PORT}`));

Add Routes: The Routes object helps you define different endpoints and serve various functionalities from your microservices. You can define a routes module in separate route files.

const express = require('express');


const router = express.Router();


router.get('/users', (req, res) => {


    res.json({ message: 'User list' });


});


module.exports = router;

Register the route in index.js:

const userRoutes = require('./routes/user');

app.use('/api', userRoutes);

Step 4. Database Integration

When building standalone microservices that have database integrations, you need to choose the right types of databases. In this step, we will configure a database for your microservice and fetch data from it through the following code. 

Choose Database: Determine your service’s needs and select the right one between a SQL and NoSQL database.

npm install mongoose

Connect Database: Once you have selected the database, look for relevant Node JS packages and create database connections in the service.

const mongoose = require('mongoose');

const userSchema = new mongoose.Schema({

    name: String,

    email: { type: String, unique: true },

    password: String

});

module.exports = mongoose.model('User', userSchema);

Set Up Models: Once the database connections are ready, it is time to create data models and schemas that will help the service communicate with the database.

const mongoose = require('mongoose');

mongoose.connect('mongodb://localhost:27017/users', {

    useNewUrlParser: true,

    useUnifiedTopology: true

}).then(() => console.log('Database connected'))

  .catch(err => console.error('Database connection error:', err));

Step 5. Service Communication

Cross-service communication is pretty common in microservices, and it is also needed to create appropriate responses for client requests sometimes. Hence, you should set up service communication paths and protocols.

Set Up API Calls Between Services: If one service needs to get some data from another service, use HTTP requests between the services. You can use axios or node-fetch.

npm install axios

Example API Call (from Order Service to User Service): In order-service, create services/userService.js

const axios = require('axios');

async function getUserById(userId) {

try {

const response = await axios.get(`http://localhost:3001/api/users/${userId}`);

return response.data;

} catch (error) {

console.error('Error fetching user:', error);

throw error;

}

}

module.exports = { getUserById };

Step 6. Implement Authentication

Authentication is an important step in securing your apps. Without proper authentication mechanisms, you may expose your app to harmful security issues. Hence, you should not compromise on this front. 

API Gateway Authentication (recommended): It is always a better choice to authenticate at API gateway levels for microservices, so you don’t have to write the same authentication source code in every service. 

JWT Tokens: Use jsonwebtoken and issue tokens that can be verified in each service. These tokens will help the service determine whether the request is coming from authorized users only or not. Below is an example of using JWT tokens in a Node.js app.

Install jsonwebtoken:

npm install jsonwebtoken

Generate JWT Token: In user-service/controllers/authController.js:

const jwt = require('jsonwebtoken');

const secret = 'your_jwt_secret';

function generateToken(user) {

return jwt.sign ({ 

id: user._id, email: user.email }, secret, { expiresIn: '1h'

});

}

async function login(req, res) {

    const { email, password } = req.body;

    // Verify user and password in DB (not shown here)

    const token = generateToken(user);

    res.json({ token });

}

module.exports = { login };

Protect Routes: You can also add middleware to verify JWT. By adding middleware you don’t have to verify the token inside each route individually. 

const jwt = require('jsonwebtoken');

const secret = 'your_jwt_secret';

function authenticateToken(req, res, next) {

const token = req.header('Authorization')?.split(' ')[1];

    if (!token) return res.sendStatus(403);

    jwt.verify(token, secret, (err, user) => {

        if (err) return res.sendStatus(403);

        req.user = user;

        next();

    });

}

module.exports = authenticateToken;

Also Read: Creating Middleware in Node JS & Express JS

Step 7. Add Logging and Monitoring

Logging and monitoring requests help you understand application usage patterns and what happens when the service processes requests. It is also helpful in finding errors before they cause application-level issues. 

Implement Logging: To log different statements while running the app, you can use libraries like winston or Morgan as they can log at various levels and to different destinations.

Install Winston for logging.

npm install winston

Configure Logger in user-service/logger.js:

const winston = require('winston');

const logger = winston.createLogger({

    level: 'info',

    format: winston.format.json(),

    transports: [

new winston.transports.File({filename: 'error.log', level: 'error'}),

new winston.transports.File({ filename: 'combined.log' }),

],

});

if (process.env.NODE_ENV !== 'production') {

    logger.add(new winston.transports.Console({

        format: winston.format.simple(),

    }));

}

module.exports = logger;

Step 8. Testing Microservices

At this stage, your microservices are ready, and it is time to test them. You should never deploy anything to controlled environments without thorough testing. While testing can take a lot of time in microservices, you can make the development process easier by writing programmatic unit and integration tests.

Unit Tests: To write unit tests for your service, you can use mocha or jest. These frameworks can help you test your services independently and verify their functionalities.

Integration Tests: To test communication between services and databases, you need to write integration tests. You can write integration tests with the mocha or chai framework. 

Install Mocha and Chai

npm install mocha chai

Create Tests in test/userService.test.js

const chai = require('chai');

const chaiHttp = require('chai-http');

const app = require('../index'); // Import your app instance

chai.use(chaiHttp);

describe('User Service', () => {

it('should fetch all users', (done) => {

        chai.request(app)

            .get('/api/users')

            .end((err, res) => {

                chai.expect(res).to.have.status(200);

                done();

            });

    });

});

Step 9. Containerize Microservices

After testing and verifying functionalities, the last step is to containerize your microservices and deploy them independently so you can use them.

Dockerize Each Service: To containerize your service, you need to write a Dockerfile for each microservice in your root services folder. Here’s a sample Dockerfile:

FROM node:16

WORKDIR /app

COPY package.json .

RUN npm install

COPY . .

CMD ["node", "server.js"]

Use Docker-Compose: After writing the docker file, you need to compose it using docker-compose. In your docker-compose.yml, you will include all your services so that inter-service communication and service discovery are also possible.

version: '3'

services:

  user-service:

    build: ./user-service

    ports:

      - '3001:3001'

  product-service:

    build: ./product-service

    ports:

      - '3002:3002'

  order-service:

    build: ./order-service

    ports:

      - '3003:3003'

Step 10. Deploy and Scale

Finally, you must deploy and scale your application so that it can function in a multi-cloud architecture. You’ll need to build up a continuous deployment pipeline and a scaling service to keep all of your services running at peak performance. 

Continuous Deployment: You may configure your CI/CD pipelines using technologies such as Jenkins or Github activities. They can assist you with automated testing and deployment of your services.

Scaling Services: Set up some scaling strategies for Kubernetes or your cloud service provider depending on incoming traffic and service performance indicators.

CI/CD Pipelines (GitHub Actions Example)

name: Node.js CI/CD

on: [push]

jobs:

  build:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      - name: Setup Node.js

        uses: actions/setup-node@v2

        with:

          node-version: '16'

      - run: npm install

      - run: npm test

Security Considerations in Node.js Microservices

Role-based Access Control (RBAC)

As the name implies, role-based access control is a more effective strategy in which individuals are allocated roles with varied levels of access. This prevents unwanted access and makes it easy to give consistent degrees of access to a large number of users. 

Data Encryption and SSL

Data encryption and SSL ensure that your data is secure while in transit and at rest. This allows you to safeguard the information of your users and apps at all times while also complying with privacy requirements. 

Secure API Communication

You can use JWT tokens to protect communication between microservices and clients. These tokens are unique to each user and have an expiration time, making it easy to manage sessions and offer access limits.

Testing and Monitoring in Node.js Microservices

Test Strategies for Microservices

Testing microservices is critical since you never know what’s missing and might create problems in the program. The simplest method to test your microservices is to unit test them individually. You may also develop integration tests to see how microservices connect with the rest of your entire system and whether they work properly. 

Continuous Integration/Deployment (CI/CD)

As the project expands, manually deploying microservices every time a change is made might become inefficient. It is no longer necessary to perform it manually using contemporary tools. You should set up continuous integration and independently deployable pipelines to test, develop, and deploy your microservice applications. These pipelines will guarantee that you have the best infrastructure and error-free builds every time. 

Real-Time Monitoring and Logging

Establish real-time monitoring, logging, and tracking of your microservices. This shall give you better judgments regarding performance, health, and response times of your microservices, along with detailed insight into the faults in the apps. Utilities like the ELK stack, Prometheus, and Grafana will enable you to monitor microservices in real-time. 

Also Read: Diving Deep Into Top 10 Node.js Performance Tips

Popular Node.js Frameworks for Microservices Development

Express.js: Express.js is a lightweight and adaptable node.js framework that’s being used to create a lot of different types of RESTful services. It is ideal for building microservices because it’s fast.

Also Read: Is Express JS Framework An Idea Choice For Developing Enterprise Applications?

NestJS: NestJS is a future framework for developing highly scalable, enterprise-grade apps and microservices. It internally uses TypeScript and has a modular framework which microservices are compatible with. TypeScript also supports type-checking and removes datatype problems.

Seneca.js: Seneca.js is designed for microservices primarily. It allows developers to easily create and deploy separate services. It also has built-in plugins for logging and monitoring.

Moleculer: Moleculer is a full-featured microservices framework that provides many features out of the box, such as service discovery, load balancing, and fault tolerance. It’s one of the best options for big applications.

Hire NodeJS Developer from eSparkBiz possessing core expertise in leveraging the capabilities of Express.js, NestJS, Seneca.js, Moleculer and other NodeJS frameworks to build your desired microservice solutions.

Communication Patterns of Node.js Microservices

Synchronous communication (HTTP)

As the name says, synchronous communication means services will communicate over RESTful APIs and wait until the other services respond. This adds a bit more latency to your application’s performance.

Asynchronous communication using message queues

In Asynchronous communication, you have to use message queues between your services. Using this can decouple the services because they can just send the message to the queue and not wait for any response. Moreover, the consuming service can consume the request from the message queue based on its performance.

Event-Driven Architecture

In this setup, your services will be able to respond to different events occurring on the event bus. They need not wait for other services to send them requests and respond with something. Your service can become independent, and they can process events and requests according to their bandwidth.

Best Practices of Node.js Microservice

Here are some of the crucial NodeJS Microservices best practices:-

Use lightweight Communication Protocols: Always look at lightweight communication protocols like gRPC or message brokers. The implementation of these will lead to low-latency distributed systems. 

Handle Errors Better: Add error handling and retry mechanism in each service so that errors in one service don’t break down the entire system.

Add Centralized Logging: Implement a centralized logging space where each service puts its logs. This will help you in log analysis and also make things easier for operations teams.

Use API Gateways: Add API key gateways in front of your services, which will enable different features like rate limiting, API throttling, security, and observability of usage. 

Common Challenges in Building Microservices with Node.js and How to Overcome Them

Data Consistency Across Microservices

Since each service is working with its database, getting data consistency across microservices can be tough initially. 

Solution:

Leverage event-driven microservices and rely on eventual consistency between microservices for better data consistency in your entire system. 

Managing Service Dependencies

A service being dependent upon availability or input data from another service is a common problem in microservices.

Solution:

You can manage the service dependencies by tracking every service and its operation. Furthermore, if a service goes down, a new instance should be automatically created for it, and the dependent service should handle errors during those downtimes. 

Complex Deployment and Versioning

As you continue to add more services, the complexities will grow, and it will be difficult to manage the deployments if you do not have a standardized process. 

Solution: 

Automate the building and deployment of your services using Jenkins or GitHub Actions. Furthermore, you may leverage docker containers to scale and deploy correct versions of your apps at all times. 

Real-World Examples of Node.js Microservices Implementations

Netflix: Netflix replaced its core streaming platform with thousands of Node.js microservices, and today, it streams millions of requests in a day with zero downtime.

Uber: Uber uses Node.js microservices to track everything in real-time on its platforms.

eCommerce Platforms: Different e-commerce platforms use Node.js microservices in event-driven architecture for up stocks, payment processing, and events by a user independently.

Also Read: Node JS Use Cases: When, Why & How to Use Node.js?

Chintan Gor

CTO, eSparkBiz

Enthusiastic for web app development, Chintan Gor has zeal in experimenting with his knowledge of Node, Angular & React in various aspects of development. He keeps on updating his technical know-how thus pinning his name among the topmost CTO's in India. His contribution is penned down by him through various technical blogs on trending tech. He is associated with eSparkBiz from the past 14+ years where one can get premium services.