Nodejs is a popular web development platform today because of the benefits and convenience it offers. But there are a few things that it was not supported as machine learning, deep learning and libraries for Python AI tao.That sewing supports all of these things and sometimes many other things that nodejs no. People will wonder why not use python’s Django Framework to build web applications that can integrate machine learning and artificial intelligence. This seems to be fine if we build that application from scratch. But what if our application is Nodejs and it’s running, but now we want to add machine learning. It’s okay to connect Nodejs and Python as usual, using the child process module for Nodejs
.
We will have 3 ways to interact between python and nodejs but first we need setup
Setup
- Create a simple Express project:1234npm ini -ynpm i express nodemon --save
Create simpleserver.js
file123456789<span class="token comment">// Server.js</span><span class="token keyword">var</span> express <span class="token operator">=</span> <span class="token function">require</span> <span class="token punctuation">(</span> ‘express’ <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">var</span> app <span class="token operator">=</span> <span class="token function">express</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>app <span class="token punctuation">.</span> <span class="token function">listen</span> <span class="token punctuation">(</span> <span class="token number">3000</span> <span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span>console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> ‘server running on port <span class="token number">3000</span> ’ <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">)</span>
In thepackage.json
file1234567891011121314151617181920<span class="token comment">// package.json</span><span class="token punctuation">{</span><span class="token string">"name"</span> <span class="token punctuation">:</span> <span class="token string">"node_integrate_python"</span> <span class="token punctuation">,</span><span class="token string">"version"</span> <span class="token punctuation">:</span> <span class="token string">"1.0.0"</span> <span class="token punctuation">,</span><span class="token string">"description"</span> <span class="token punctuation">:</span> <span class="token string">""</span> <span class="token punctuation">,</span><span class="token string">"main"</span> <span class="token punctuation">:</span> <span class="token string">"index.js"</span> <span class="token punctuation">,</span><span class="token string">"scripts"</span> <span class="token punctuation">:</span> <span class="token punctuation">{</span><span class="token string">"start"</span> <span class="token punctuation">:</span> <span class="token string">"nodemon server.js"</span> <span class="token punctuation">,</span><span class="token string">"test"</span> <span class="token punctuation">:</span> <span class="token string">"echo "Error: no test specified" && exit 1"</span><span class="token punctuation">}</span> <span class="token punctuation">,</span><span class="token string">"keywords"</span> <span class="token punctuation">:</span> <span class="token punctuation">[</span> <span class="token punctuation">]</span> <span class="token punctuation">,</span><span class="token string">"author"</span> <span class="token punctuation">:</span> <span class="token string">""</span> <span class="token punctuation">,</span><span class="token string">"license"</span> <span class="token punctuation">:</span> <span class="token string">"ISC"</span> <span class="token punctuation">,</span><span class="token string">"dependencies"</span> <span class="token punctuation">:</span> <span class="token punctuation">{</span><span class="token string">"express"</span> <span class="token punctuation">:</span> <span class="token string">"^4.17.1"</span> <span class="token punctuation">,</span><span class="token string">"nodemon"</span> <span class="token punctuation">:</span> <span class="token string">"^2.0.2"</span><span class="token punctuation">}</span><span class="token punctuation">}</span>
Try running the project before continuing12npm start
The result is as follows1234567[nodemon] 2.0.2[nodemon] to restart at any time, enter `rs`[nodemon] watching dir(s): *.*[nodemon] watching extensions: js,mjs,json[nodemon] starting `node server.js`server running on port 3000
Next we need a file that does not handle pythonprocess.py
:123456<span class="token keyword">import</span> sys<span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">"Output from Python"</span> <span class="token punctuation">)</span><span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">"First name: "</span> <span class="token operator">+</span> sys <span class="token punctuation">.</span> argv <span class="token punctuation">[</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">"Last name: "</span> <span class="token operator">+</span> sys <span class="token punctuation">.</span> argv <span class="token punctuation">[</span> <span class="token number">2</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>
Method 1: Call Python script from Node child process
- We need to modify the Express file server to make it call the Python script child process123456789101112131415161718192021<span class="token comment">// server.js</span><span class="token operator">...</span>app <span class="token punctuation">.</span> <span class="token keyword">get</span> <span class="token punctuation">(</span> <span class="token string">'/name'</span> <span class="token punctuation">,</span> callName <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">function</span> <span class="token function">callName</span> <span class="token punctuation">(</span> req <span class="token punctuation">,</span> res <span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">var</span> spawn <span class="token operator">=</span> <span class="token function">require</span> <span class="token punctuation">(</span> <span class="token string">'child_process'</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> spawn <span class="token punctuation">;</span><span class="token comment">// E.g : http://localhost:3000/name?firstname=van&lastname=nghia</span><span class="token keyword">var</span> process <span class="token operator">=</span> <span class="token function">spawn</span> <span class="token punctuation">(</span> <span class="token string">'python'</span> <span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token string">'./process.py'</span> <span class="token punctuation">,</span>req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> firstname <span class="token punctuation">,</span>req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> lastname<span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>process <span class="token punctuation">.</span> stdout <span class="token punctuation">.</span> <span class="token function">on</span> <span class="token punctuation">(</span> <span class="token string">'data'</span> <span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> data <span class="token punctuation">)</span> <span class="token punctuation">{</span>console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> data <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>res <span class="token punctuation">.</span> <span class="token function">send</span> <span class="token punctuation">(</span> data <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span>
The function used to interact between nodejs and python is child_process.spawn () as defined by the definition:12child_process.spawn(): This method helps us to spawn child process asynchronously.
Thechild_process.spawn()
function will have the following parameters12345<span class="token keyword">var</span> process <span class="token operator">=</span> <span class="token function">spawn</span> <span class="token punctuation">(</span> <span class="token string">'python'</span> <span class="token punctuation">,</span> <span class="token punctuation">[</span> <span class="token string">'./process.py'</span> <span class="token punctuation">,</span> arg1 <span class="token punctuation">,</span> arg2 <span class="token punctuation">,</span> <span class="token operator">...</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token operator">-</span> Tham số đầu tiên sẽ là file python script<span class="token operator">-</span> theo sau sẽ là các arg tham số ta muốn truyền vào
The parameters passed will be retrieved in the correct order123456<span class="token comment"># process.py</span><span class="token punctuation">.</span> <span class="token punctuation">.</span> <span class="token punctuation">.</span><span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">"First name: "</span> <span class="token operator">+</span> sys <span class="token punctuation">.</span> argv <span class="token punctuation">[</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">"Last name: "</span> <span class="token operator">+</span> sys <span class="token punctuation">.</span> argv <span class="token punctuation">[</span> <span class="token number">2</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>
Run the application and then enter the example linkhttp://localhost:3000/name?firstname=van&lastname=nghia
the result:in the console:
Method 2: Call Python script from the Node child process using the python-shell package
- This can be interpreted as way 1.1 because we are still using the child process but for now we will use python-shell to include deployment details (as well as provide error handling and other utilities including). deploy a simple messaging to communicate directly with a single script).We will rewrite the file server as follows:1234567891011121314151617<span class="token comment">// server.js</span>app <span class="token punctuation">.</span> <span class="token keyword">get</span> <span class="token punctuation">(</span> <span class="token string">'/name'</span> <span class="token punctuation">,</span> callName <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">function</span> <span class="token function">callName</span> <span class="token punctuation">(</span> req <span class="token punctuation">,</span> res <span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">let</span> <span class="token punctuation">{</span> PythonShell <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token function">require</span> <span class="token punctuation">(</span> <span class="token string">'python-shell'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">var</span> options <span class="token operator">=</span> <span class="token punctuation">{</span>args <span class="token punctuation">:</span> <span class="token punctuation">[</span> req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> firstname <span class="token punctuation">,</span> req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> lastname <span class="token punctuation">]</span><span class="token punctuation">}</span> <span class="token punctuation">;</span>PythonShell <span class="token punctuation">.</span> <span class="token function">run</span> <span class="token punctuation">(</span> <span class="token string">'process.py'</span> <span class="token punctuation">,</span> options <span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> err <span class="token punctuation">,</span> data <span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">if</span> <span class="token punctuation">(</span> err <span class="token punctuation">)</span> res <span class="token punctuation">.</span> <span class="token function">send</span> <span class="token punctuation">(</span> err <span class="token punctuation">)</span> <span class="token punctuation">;</span>console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> data <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>res <span class="token punctuation">.</span> <span class="token function">send</span> <span class="token punctuation">(</span> data <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span>
Install package
python-shell
:12npm i python-shell --saverun to test we get the result:
Method 3: Use a message broker
- We will now run the Python and Node applications as separate processes and use a message broker to communicate between them. Here we use RabbitMQ , along with amqplib to integrate in node and pika for Python.Follow RabbitMQ installation if you haven’t already, on OSX you can simply run:12345678sudo apt updatesudo apt -y install rabbitmq-serverpip install pikanpm install amqplib --save
Then start
RabbitMQ
:12sudo systemctl start rabbitmq-server.serviceNow we’ll edit files
server.js
. Use amqplib in the node to connect to the currently running RabbitMQ instance:123456789101112131415161718192021222324252627282930<span class="token comment">//server.js</span>app <span class="token punctuation">.</span> <span class="token keyword">get</span> <span class="token punctuation">(</span> <span class="token string">'/name'</span> <span class="token punctuation">,</span> callName <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">function</span> <span class="token function">callName</span> <span class="token punctuation">(</span> req <span class="token punctuation">,</span> res <span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">var</span> amqp <span class="token operator">=</span> <span class="token function">require</span> <span class="token punctuation">(</span> <span class="token string">'amqplib/callback_api'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">var</span> input <span class="token operator">=</span> <span class="token punctuation">[</span> req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> firstname <span class="token punctuation">,</span> req <span class="token punctuation">.</span> query <span class="token punctuation">.</span> lastname <span class="token punctuation">]</span> <span class="token punctuation">;</span>amqp <span class="token punctuation">.</span> <span class="token function">connect</span> <span class="token punctuation">(</span> <span class="token string">'amqp://localhost'</span> <span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> err <span class="token punctuation">,</span> conn <span class="token punctuation">)</span> <span class="token punctuation">{</span>conn <span class="token punctuation">.</span> <span class="token function">createChannel</span> <span class="token punctuation">(</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> err <span class="token punctuation">,</span> ch <span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token keyword">var</span> simulations <span class="token operator">=</span> <span class="token string">'simulations'</span> <span class="token punctuation">;</span>ch <span class="token punctuation">.</span> <span class="token function">assertQueue</span> <span class="token punctuation">(</span> simulations <span class="token punctuation">,</span> <span class="token punctuation">{</span> durable <span class="token punctuation">:</span> <span class="token boolean">false</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token keyword">var</span> results <span class="token operator">=</span> <span class="token string">'results'</span> <span class="token punctuation">;</span>ch <span class="token punctuation">.</span> <span class="token function">assertQueue</span> <span class="token punctuation">(</span> results <span class="token punctuation">,</span> <span class="token punctuation">{</span> durable <span class="token punctuation">:</span> <span class="token boolean">false</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>ch <span class="token punctuation">.</span> <span class="token function">sendToQueue</span> <span class="token punctuation">(</span> simulations <span class="token punctuation">,</span> Buffer <span class="token punctuation">.</span> <span class="token keyword">from</span> <span class="token punctuation">(</span> <span class="token constant">JSON</span> <span class="token punctuation">.</span> <span class="token function">stringify</span> <span class="token punctuation">(</span> input <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>ch <span class="token punctuation">.</span> <span class="token function">consume</span> <span class="token punctuation">(</span>results <span class="token punctuation">,</span><span class="token keyword">function</span> <span class="token punctuation">(</span> msg <span class="token punctuation">)</span> <span class="token punctuation">{</span>console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> msg <span class="token punctuation">.</span> content <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span>res <span class="token punctuation">.</span> <span class="token function">send</span> <span class="token punctuation">(</span> msg <span class="token punctuation">.</span> content <span class="token punctuation">.</span> <span class="token function">toString</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">,</span><span class="token punctuation">{</span> noAck <span class="token punctuation">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token function">setTimeout</span> <span class="token punctuation">(</span> <span class="token keyword">function</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span>conn <span class="token punctuation">.</span> <span class="token function">close</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">,</span> <span class="token number">500</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span><span class="token punctuation">}</span>We write a simple message handler in Python to handle request messages, call process.py and publish response messages – message.py :
1234567891011121314151617181920<span class="token keyword">import</span> pika<span class="token keyword">import</span> process<span class="token keyword">import</span> jsonconnection <span class="token operator">=</span> pika <span class="token punctuation">.</span> BlockingConnection <span class="token punctuation">(</span> pika <span class="token punctuation">.</span> ConnectionParameters <span class="token punctuation">(</span> <span class="token string">'localhost'</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span>channel <span class="token operator">=</span> connection <span class="token punctuation">.</span> channel <span class="token punctuation">(</span> <span class="token punctuation">)</span>channel <span class="token punctuation">.</span> queue_declare <span class="token punctuation">(</span> queue <span class="token operator">=</span> <span class="token string">'simulations'</span> <span class="token punctuation">)</span>channel <span class="token punctuation">.</span> queue_declare <span class="token punctuation">(</span> queue <span class="token operator">=</span> <span class="token string">'results'</span> <span class="token punctuation">)</span><span class="token keyword">def</span> <span class="token function">callback</span> <span class="token punctuation">(</span> ch <span class="token punctuation">,</span> method <span class="token punctuation">,</span> properties <span class="token punctuation">,</span> body <span class="token punctuation">)</span> <span class="token punctuation">:</span>requestParams <span class="token operator">=</span> json <span class="token punctuation">.</span> loads <span class="token punctuation">(</span> body <span class="token punctuation">.</span> decode <span class="token punctuation">(</span> <span class="token string">'utf-8'</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span>firstname <span class="token operator">=</span> <span class="token builtin">str</span> <span class="token punctuation">(</span> requestParams <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>lastname <span class="token operator">=</span> <span class="token builtin">str</span> <span class="token punctuation">(</span> requestParams <span class="token punctuation">[</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>results <span class="token operator">=</span> process <span class="token punctuation">.</span> simulate <span class="token punctuation">(</span> firstname <span class="token punctuation">,</span> lastname <span class="token punctuation">)</span>channel <span class="token punctuation">.</span> basic_publish <span class="token punctuation">(</span> exchange <span class="token operator">=</span> <span class="token string">''</span> <span class="token punctuation">,</span> routing_key <span class="token operator">=</span> <span class="token string">'results'</span> <span class="token punctuation">,</span> body <span class="token operator">=</span> json <span class="token punctuation">.</span> dumps <span class="token punctuation">(</span> results <span class="token punctuation">,</span> ensure_ascii <span class="token operator">=</span> <span class="token boolean">False</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span>channel <span class="token punctuation">.</span> basic_consume <span class="token punctuation">(</span> queue <span class="token operator">=</span> <span class="token string">'simulations'</span> <span class="token punctuation">,</span> on_message_callback <span class="token operator">=</span> callback <span class="token punctuation">,</span> auto_ack <span class="token operator">=</span> <span class="token boolean">True</span> <span class="token punctuation">)</span>channel <span class="token punctuation">.</span> start_consuming <span class="token punctuation">(</span> <span class="token punctuation">)</span>Next We need to make some small changes to process.py – now we will pass the parameters in, instead of using sys.argvs and expose to a method we can call from the message. py :
12345678910111213141516<span class="token comment"># process.py</span><span class="token keyword">import</span> sys<span class="token comment"># print("Output from Python") </span><span class="token comment"># print("First name: " + sys.argv[1]) </span><span class="token comment"># print("Last name: " + sys.argv[2])</span><span class="token keyword">def</span> <span class="token function">simulate</span> <span class="token punctuation">(</span> firstname <span class="token punctuation">,</span> lastname <span class="token punctuation">)</span> <span class="token punctuation">:</span>results <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token string">'firstname'</span> <span class="token punctuation">:</span> firstname <span class="token punctuation">,</span><span class="token string">'lastname'</span> <span class="token punctuation">:</span> lastname<span class="token punctuation">}</span><span class="token keyword">return</span> resultsThe steps to run will first run
message.py
file:12python message.pyThen run to the server node:
12npm startCheck the result
Conclude
This is a simple combination of Python and Node.js, which is useful for situations where you want to use Python’s computing capabilities but want to take advantage of the Node.js application. Hopefully, through this article you can use and develop your application better from these basics.
Source: https://github.com/ngovannghia1997kma/Node_integrate_Python
Source :
https://medium.com/@HolmesLaurence/integrating-node-and-python-6b8454bfc272 https://www.geeksforgeeks.org/run-python-script-node-js-using-child-process-spawn-method/