Clusters and Child Processes
Scaling Node.js Applications
Although single-threaded and non-block is pretty good. But a process in a CPU will not be enough to handle the growing workload of an application. No matter how powerful your server can be, what single-threaded support can be limited. The fact that a node runs in single-threaded does not mean that we cannot take advantage of many processes and of course, so do many servers. Using multiple processes is the only way to extend a Nodejs application. Nodejs is designed to build distributed applications with multiple nodes. This is why it is named Nodejs.
Scalability is brought into the platform. Workloads are the most common reason we expand our applications, but that’s not the only reason. We also expanded our applications to increase usability and failure tolerance. Before we talk about scaling a node, there are three things we can do to extend an application:
- The easiest way to extend a large application is to clone it multiple times and let each mirrored instance handle part of the workload. This does not take a long time to develop and it is highly effective.
- In addition, we can extend an application by separating it based on functions and services. This means that there are many different applications with different codes and sometimes even a separate database. This strategy is often associated with the term microservice, in which the microphone indicates that the services should be as small as possible, but in reality, the scale of the service is not important, but the implementation. Loose connection and high cohesion between the services. Implementing this strategy is often not easy and can lead to unexpected long-term problems, but when done correctly, the advantages are huge.
- The third expansion strategy is to divide the application into cases where each case is only responsible for a portion of the application’s data. Data partitioning requires a lookup step before each operation to determine which case of the application to use. For example, we may want to partition users based on their country or language. We need to look up that information first.
Successfully extending a large application will eventually implement all three strategies and nodes. js makes it easy to do so. Next, we will talk about the built-in tools available in Nodejs to implement the clone strategy.
Child Processes Events and Standard IO
We can easily generate a child process by using child_ process modules and the child process can easily communicate with each other by a messaging system. The child_ process module allows us to access Operating System functions by running any system command within a child process, controlling input stream and listening to its outout stream. We can control the arguments passed to the command and we can do whatever we want with its output. For example, we can convert the input of one command to another, because all the inputs and outputs of these commands can be displayed to us using streams.
There are four different ways we can use to create a child process in Node: Spawn, fork, exec, and execFile. Each function will have a difference that different uses:
execFile
1 2 | child_process.execFile(file[, args][, options][, callback]) |
Execute a function on a new process. Input is the name or path to a file, which can be passed as an argument and a callback. Output is when done, the callback will be called with data as buffer or an error.
1 2 3 4 5 6 7 8 9 | const execFile = require('child_process').execFile; const child = execFile('node', ['--version'], (error, stdout, stderr) => { if (error) { console.error('stderr', stderr); throw error; } console.log('stdout', stdout); }); |
spawn
1 2 | child_process.spawn(command[, args][, options]) |
Execute a function on a new process. Input is a command, can be passed arguments. Output is 1 stream I / O,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | const spawn = require('child_process').spawn; const fs = require('fs'); function resize(req, resp) { const args = [ "-", // use stdin "-resize", "640x", // resize width to 640 "-resize", "x360<", // resize height if it's smaller than 360 "-gravity", "center", // sets the offset to the center "-crop", "640x360+0+0", // crop "-" // output to stdout ]; const streamIn = fs.createReadStream('./path/to/an/image'); const proc = spawn('convert', args); streamIn.pipe(proc.stdin); proc.stdout.pipe(resp); } |
exec
1 2 | child_process.exec(command[, options][, callback]) |
Execute a function on a new process. Input is a command, can be passed arguments and callbacks. Output is when done, the callback will be called with data as buffer or an error.
1 2 3 4 5 6 7 8 9 10 11 | const exec = require('child_process').exec; exec('for i in $( ls -LR ); do echo item: $i; done', (e, stdout, stderr)=> { if (e instanceof Error) { console.error(e); throw e; } console.log('stdout ', stdout); console.log('stderr ', stderr); }); |
fork
1 2 | child_process.fork(modulePath[, args][, options]) |
The child_ process.fork () method is a special case of child_ process.spawn () that is specially used to generate new Nodejs processes. Like child_ process.spawn (), a ChildProcess object is returned. The returned ChildProcess will have an integrated communication channel that allows messages to be passed between the parent process and the child process.
1 2 3 4 5 6 7 8 9 10 11 12 13 | //parent.js const cp = require('child_process'); const n = cp.fork(`${__dirname}/sub.js`); n.on('message', (m) => { console.log('PARENT got message:', m); }); n.send({ hello: 'world' }); //sub.js process.on('message', (m) => { console.log('CHILD got message:', m); }); process.send({ foo: 'bar' }); |
Refer
https://dzone.com/articles/understanding-execfile-spawn-exec-and-fork-in-node