p5.js là gì?
p5.js là một thư viện Javascript, thường thì nó sẽ sử dụng để dùng làm những thứ linh tinh với đồ họa các thứ sử dụng canvas. Các bạn có thể sử dụng p5.js bằng p5 Web Editor ở đây.
Sơ qua về p5.js
Ở trong p5.js thì sẽ có hai function quan trọng. Đó là setup() và draw().
setup()
Đây là function sẽ chạy ngay lập tức khi chạy. Thường thì hay dùng để config trước khi chạy thực tế.
draw()
Function này sẽ chạy ngay sau thằng setup() ở trên. Đây là function chính của p5.js.
ml5.js là gì?
ml5.js là một thư viện bao gồm các thuật toán và pre-trained models cho browser. ml5.js được build trên nền của tensoeflow.js. Vì thế, mình có thể dùng ml5.js để build một số thứ hay ho dựa trên các pre-trained models có sẵn.
Lý do mình dùng p5.js trong bài này là vì thằng ml5.js này chơi thân với thằng p5.js nên nó dễ sử dụng hơn.
Ví dụ
Bây giờ thì mình làm linh tinh một cái gì đó để ví dụ về cái ml5.js này cho vui. Mình ở đây sẽ làm một cái project vui vui sử dụng poseNet pre-trained model.
poseNet pre-trained model
poseNet pre-trained model là một pre-trained model dùng để mô phỏng vị trí tay, chân, mắt, mũi, miệng, bla bla bla của con người.
Cấu trúc Project
Về cấu trúc thì đơn giản như sau.
1 2 3 4 | index.html nancy.mp4 script.js |
Trong file index.html thì import p5.js và ml5.js vào thôi
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <span class="token doctype"><!DOCTYPE html></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>html</span> <span class="token attr-name">lang</span><span class="token attr-value"><span class="token punctuation">=</span><span class="token punctuation">"</span>en<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>title</span><span class="token punctuation">></span></span>Getting Started with ml5.js<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>title</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation">=</span><span class="token punctuation">"</span>https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script language-javascript"/><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation">=</span><span class="token punctuation">"</span>https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script language-javascript"/><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation">=</span><span class="token punctuation">"</span>https://unpkg.com/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="1875742d5828362c3629">[email protected]</a>/dist/ml5.min.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script language-javascript"/><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation">=</span><span class="token punctuation">"</span>script.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script language-javascript"/><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>html</span><span class="token punctuation">></span></span> |
Load Video bằng function createVideo của p5.js
Để load Video thì trong p5.js mình dùng function createVideo(). Vì là load video thì mình viết nó trong setup(), sau khi load xong rồi thì mới hiển thị đó ra ở draw() function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | <span class="token keyword">let</span> video<span class="token punctuation">;</span> <span class="token keyword">let</span> poses <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token keyword">function</span> <span class="token function">setup</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">createCanvas</span><span class="token punctuation">(</span><span class="token number">406</span><span class="token punctuation">,</span> <span class="token number">720</span><span class="token punctuation">)</span><span class="token punctuation">;</span> video <span class="token operator">=</span> <span class="token function">createVideo</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token string">"nancy.mp4"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> video<span class="token punctuation">.</span><span class="token function">loop</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> video<span class="token punctuation">.</span><span class="token function">volume</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> video<span class="token punctuation">.</span><span class="token function">size</span><span class="token punctuation">(</span>width<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token punctuation">;</span> video<span class="token punctuation">.</span><span class="token function">hide</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">function</span> <span class="token function">draw</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">image</span><span class="token punctuation">(</span>video<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> width<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> |
Lúc này kết quả như sau.
Load poseNet pre-trained model
Công việc tiếp theo là mình load tiếp pre-trained model vào. Mình sẽ viết tiếp trong function setup() như sau.
1 2 3 4 5 6 7 8 | poseNet <span class="token operator">=</span> ml5<span class="token punctuation">.</span><span class="token function">poseNet</span><span class="token punctuation">(</span>video<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Model is ready"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Listen to new 'pose' events</span> poseNet<span class="token punctuation">.</span><span class="token function">on</span><span class="token punctuation">(</span><span class="token string">"pose"</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span>results<span class="token punctuation">)</span> <span class="token punctuation">{</span> poses <span class="token operator">=</span> results<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> |
Vẽ skeleton lên Video
Sau khi lắng nghe sự kiện pose ở trên. Nó sẽ trả một mảng các pose, bao gồm các properties như sau:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | <span class="token punctuation">[</span> <span class="token punctuation">{</span> pose<span class="token punctuation">:</span> <span class="token punctuation">{</span> keypoints<span class="token punctuation">:</span> <span class="token punctuation">[</span><span class="token punctuation">{</span>position<span class="token punctuation">:</span><span class="token punctuation">{</span>x<span class="token punctuation">,</span>y<span class="token punctuation">}</span><span class="token punctuation">,</span> score<span class="token punctuation">,</span> part<span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token operator">...</span><span class="token punctuation">]</span><span class="token punctuation">,</span> leftAnkle<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">289.23709728843284</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">739.7039085522032</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.02065359242260456</span><span class="token punctuation">}</span> leftEar<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">259.3518055698328</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">113.62603513818037</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.17528685927391052</span><span class="token punctuation">}</span> leftElbow<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">312.8670815584952</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">355.85845010322436</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.3681596517562866</span><span class="token punctuation">}</span> leftEye<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">242.82734312090957</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">98.04618166204084</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9990793466567993</span><span class="token punctuation">}</span> leftHip<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">285.8621941282038</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">423.13189556724143</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.2725003957748413</span><span class="token punctuation">}</span> leftKnee<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">275.85972595214844</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">682.6726759525767</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9468103647232056</span><span class="token punctuation">}</span> leftShoulder<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">291.67652035596075</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">222.69961507696854</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9864554405212402</span><span class="token punctuation">}</span> leftWrist<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">315.4616281609786</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">308.55148850825793</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.31114229559898376</span><span class="token punctuation">}</span> nose<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">230.66232273034882</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">115.1411611992016</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9995276927947998</span><span class="token punctuation">}</span> rightAnkle<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">187.2091764483535</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">734.9885826780085</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.12924212217330933</span><span class="token punctuation">}</span> rightEar<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">177.03415412233585</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">108.54197920414438</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.705632746219635</span><span class="token punctuation">}</span> rightElbow<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">88.15446646171702</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">364.6860253183465</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.6543225646018982</span><span class="token punctuation">}</span> rightEye<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">209.1365689729389</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">97.66484712299547</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9994242787361145</span><span class="token punctuation">}</span> rightHip<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">177.5815705416495</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">426.5434666683799</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.25370630621910095</span><span class="token punctuation">}</span> rightKnee<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">191.22615238658165</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">676.1598981890762</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.7883720397949219</span><span class="token punctuation">}</span> rightShoulder<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">127.5505650838216</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">220.96515889753374</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.9299540519714355</span><span class="token punctuation">}</span> rightWrist<span class="token punctuation">:</span> <span class="token punctuation">{</span>x<span class="token punctuation">:</span> <span class="token number">119.36843028821443</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">311.3493300320809</span><span class="token punctuation">,</span> confidence<span class="token punctuation">:</span> <span class="token number">0.6887825727462769</span><span class="token punctuation">}</span> score<span class="token punctuation">:</span> <span class="token number">0.6017089601167861</span>zz <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> |
Từ đây thì mình lặp hết và vẽ skeleton lên thôi.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | <span class="token comment">// A function to draw the skeletons</span> <span class="token keyword">function</span> <span class="token function">drawSkeleton</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token comment">// Loop through all the skeletons detected</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> poses<span class="token punctuation">.</span>length<span class="token punctuation">;</span> i<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> skeleton <span class="token operator">=</span> poses<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>skeleton<span class="token punctuation">;</span> <span class="token comment">// For every skeleton, loop through all body connections</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> j <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> j <span class="token operator"><</span> skeleton<span class="token punctuation">.</span>length<span class="token punctuation">;</span> j<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> partA <span class="token operator">=</span> skeleton<span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token keyword">let</span> partB <span class="token operator">=</span> skeleton<span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token function">stroke</span><span class="token punctuation">(</span><span class="token number">255</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">strokeWeight</span><span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">line</span><span class="token punctuation">(</span> partA<span class="token punctuation">.</span>position<span class="token punctuation">.</span>x<span class="token punctuation">,</span> partA<span class="token punctuation">.</span>position<span class="token punctuation">.</span>y<span class="token punctuation">,</span> partB<span class="token punctuation">.</span>position<span class="token punctuation">.</span>x<span class="token punctuation">,</span> partB<span class="token punctuation">.</span>position<span class="token punctuation">.</span>y <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> |
Sau đó để function drawSkeleton() này vào function draw()
1 2 3 4 5 | <span class="token keyword">function</span> <span class="token function">draw</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">image</span><span class="token punctuation">(</span>video<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> width<span class="token punctuation">,</span> height<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">drawSkeleton</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> |
Kết quả sẽ như sau
Kết luận
Trên đây là một ứng dụng cơ bản của thằng ml5.js. Ngoài poseNet pre-trained model, thằng ml5.js còn cung cấp một số pre-trained model khác cũng hay ho, ví dụ như là Image Classifier sử dụng Mobilet Net, FaceApi, hay YOLO. Các bạn cứ ghé vào trang chủ của ml5.js để ngó qua nhé.
Mình cũng có viết một bài về Nhận diện gương mặt với face-api-js. Các bạn có thể xem thêm nhé.
Các bạn có thể xem source code ở đây: https://github.com/nguyen47/nancy-skeleton-ml5-js
Hiện tại mình đang tập tọe viết Blog ở địa chỉ https://htknguyen.com/. Nếu bạn nào có hứng thú thì ghé vào blog của mình nghe mình chém gió loạn lên dưới góc độ một thằng Developer cùi nhé. Bắn tim