Preface
Recently, I have been very interested in WebRTC so I will do some articles to learn about this WebRTC. The article is based on my limited knowledge, I hope the a / c / e will give me more suggestions. Please. We also get started.
content
WebRTC stands for Web Real-Time Communication very popular with programmers. WebRTC allows browsers to communicate with each other in real time. For example: phone calls, videos, play games, … In addition, WebRTC is a product of the World Wide Web Consortium (W3C). It has the ability to support real-time communication browser via Video Call, Voice Call or transfer data P2P (peer-to-peer), no need for plugins or other software.
In general, WebRTC has a lot to consider and research. So in today’s article, I will only introduce one of the main components of WebRTC .
MediaStream APIs
To initiate a Video call, we first need access to the webcam (to record video), the microphone (to record audio). And to do this WebRTC provides us MediaStream API. It is designed to easily access the media streams of the computer or phone we are using. Usually these are called via the navigator.mediaDevices object. From this object we can list all the connected devices, listen for device changes (when the device is connected or disconnected) and use a device to access the Media Stream ( see below).
1 2 3 4 5 6 7 8 9 10 11 12 | <span class="token keyword">const</span> constraints <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token string">'video'</span> <span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">,</span> <span class="token string">'audio'</span> <span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">getUserMedia</span> <span class="token punctuation">(</span> constraints <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">then</span> <span class="token punctuation">(</span> <span class="token parameter">stream</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> <span class="token string">'Got MediaStream:'</span> <span class="token punctuation">,</span> stream <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">catch</span> <span class="token punctuation">(</span> <span class="token parameter">error</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console <span class="token punctuation">.</span> <span class="token function">error</span> <span class="token punctuation">(</span> <span class="token string">'Error accessing media devices.'</span> <span class="token punctuation">,</span> error <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> |
Calling the getUserMedia function will ask the user (the users of the system) to ask for permission to access the device’s camera and microphone.
- If the user allows, the
promisewill be passed along with aMediaStreamincluding the video and audio tracks - If the user does not allow it, a
PermissionDeniedErrorwill be fired - If the device does not have video or audio hardware, a
NotFoundErrorwill be fired.
1: Querying media devices
In more complex applications or devices with multiple media devices (For example, phones with two cameras, for example, front and back cameras). We will get access to more hardware and our job is to let the user choose the appropriate hardware. This can be done using the enumerateDevices() function.
1 2 3 4 5 6 7 8 9 10 | <span class="token keyword">function</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token parameter">type <span class="token punctuation">,</span> callback</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">enumerateDevices</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">then</span> <span class="token punctuation">(</span> <span class="token parameter">devices</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> filtered <span class="token operator">=</span> devices <span class="token punctuation">.</span> <span class="token function">filter</span> <span class="token punctuation">(</span> <span class="token parameter">device</span> <span class="token operator">=></span> device <span class="token punctuation">.</span> kind <span class="token operator">===</span> type <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token function">callback</span> <span class="token punctuation">(</span> filtered <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token string">'videoinput'</span> <span class="token punctuation">,</span> <span class="token parameter">cameras</span> <span class="token operator">=></span> console <span class="token punctuation">.</span> <span class="token function">log</span> <span class="token punctuation">(</span> <span class="token string">'Cameras found'</span> <span class="token punctuation">,</span> cameras <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> |
It will return a promise and go bad with an array of MediaDevicesInfo (Each of these objects will describe the information of the Media Device that the user provides). And from this information we can give the user permission to choose the media device they want to use. Each MediaDevicesInfo contains an attribute named kind whose value is audioinput , audiooutput , videoinput to identify the media device type.
2: Listen for change of devices.
Most computers today support multiple media devices plugged in. It can be a webcam via USB, Micro and headphone via Bluetooth or an external audio-visual device. To support this, the web app can listen for changes in media devices. We can do this using the devicechange event in navigator.mediaDevices :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | <span class="token comment">// Updates the select element with the provided set of cameras</span> <span class="token keyword">function</span> <span class="token function">updateCameraList</span> <span class="token punctuation">(</span> <span class="token parameter">cameras</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> listElement <span class="token operator">=</span> document <span class="token punctuation">.</span> <span class="token function">querySelector</span> <span class="token punctuation">(</span> <span class="token string">'select#availableCameras'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> listElement <span class="token punctuation">.</span> innerHTML <span class="token operator">=</span> <span class="token string">''</span> <span class="token punctuation">;</span> cameras <span class="token punctuation">.</span> <span class="token function">map</span> <span class="token punctuation">(</span> <span class="token parameter">camera</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> cameraOption <span class="token operator">=</span> document <span class="token punctuation">.</span> <span class="token function">createElement</span> <span class="token punctuation">(</span> <span class="token string">'option'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> cameraOption <span class="token punctuation">.</span> label <span class="token operator">=</span> camera <span class="token punctuation">.</span> label <span class="token punctuation">;</span> cameraOption <span class="token punctuation">.</span> value <span class="token operator">=</span> camera <span class="token punctuation">.</span> deviceId <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> <span class="token function">forEach</span> <span class="token punctuation">(</span> <span class="token parameter">cameraOption</span> <span class="token operator">=></span> listElement <span class="token punctuation">.</span> <span class="token function">add</span> <span class="token punctuation">(</span> cameraOption <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token comment">// Fetch an array of devices of a certain type</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token parameter">type</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> devices <span class="token operator">=</span> <span class="token keyword">await</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">enumerateDevices</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token keyword">return</span> devices <span class="token punctuation">.</span> <span class="token function">filter</span> <span class="token punctuation">(</span> <span class="token parameter">device</span> <span class="token operator">=></span> device <span class="token punctuation">.</span> kind <span class="token operator">===</span> type <span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token comment">// Get the initial set of cameras connected</span> <span class="token keyword">const</span> videoCameras <span class="token operator">=</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token string">'videoinput'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token function">updateCameraList</span> <span class="token punctuation">(</span> videoCameras <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token comment">// Listen for changes to media devices and update the list accordingly</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">addEventListener</span> <span class="token punctuation">(</span> <span class="token string">'devicechange'</span> <span class="token punctuation">,</span> <span class="token parameter">event</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newCameraList <span class="token operator">=</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token string">'video'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token function">updateCameraList</span> <span class="token punctuation">(</span> newCameraList <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> |
3: Media constraints.
In some complex applications, media devices with different specifications may be required. This object is built on top of MediaStreamConstraints . This object is used as a param in getUserMedia() that allows us to request the correct devices as required by the app.
Requirements can be very loose as basic as including audio and video or very complex (what device ID must be or camera resolution). As follows :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token parameter">type</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> devices <span class="token operator">=</span> <span class="token keyword">await</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">enumerateDevices</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token keyword">return</span> devices <span class="token punctuation">.</span> <span class="token function">filter</span> <span class="token punctuation">(</span> <span class="token parameter">device</span> <span class="token operator">=></span> device <span class="token punctuation">.</span> kind <span class="token operator">===</span> type <span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token comment">// Open camera with at least minWidth and minHeight capabilities</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">openCamera</span> <span class="token punctuation">(</span> <span class="token parameter">cameraId <span class="token punctuation">,</span> minWidth <span class="token punctuation">,</span> minHeight</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> constraints <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token string">'audio'</span> <span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string">'echoCancellation'</span> <span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span> <span class="token punctuation">,</span> <span class="token string">'video'</span> <span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string">'deviceId'</span> <span class="token operator">:</span> cameraId <span class="token punctuation">,</span> <span class="token string">'width'</span> <span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string">'min'</span> <span class="token operator">:</span> minWidth <span class="token punctuation">}</span> <span class="token punctuation">,</span> <span class="token string">'height'</span> <span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token string">'min'</span> <span class="token operator">:</span> minHeight <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token keyword">await</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">getUserMedia</span> <span class="token punctuation">(</span> constraints <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">const</span> cameras <span class="token operator">=</span> <span class="token function">getConnectedDevices</span> <span class="token punctuation">(</span> <span class="token string">'videoinput'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span> cameras <span class="token operator">&&</span> cameras <span class="token punctuation">.</span> length <span class="token operator">></span> <span class="token number">0</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token comment">// Open first available video camera with a resolution of 1280x720 pixels</span> <span class="token keyword">const</span> stream <span class="token operator">=</span> <span class="token function">openCamera</span> <span class="token punctuation">(</span> cameras <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> deviceId <span class="token punctuation">,</span> <span class="token number">1280</span> <span class="token punctuation">,</span> <span class="token number">720</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> |
4: Local playback
Once the user has given us permission to use the media device, the Web Browser has been allowed to start and use the device. Then we have a MediaStream that can be used. And our job is to tag them video and audio in the browser and play it in the browser:
1 2 3 4 5 6 7 8 9 10 11 | <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">playVideoFromCamera</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">try</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> constraints <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token string">'video'</span> <span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">,</span> <span class="token string">'audio'</span> <span class="token operator">:</span> <span class="token boolean">true</span> <span class="token punctuation">}</span> <span class="token punctuation">;</span> <span class="token keyword">const</span> stream <span class="token operator">=</span> <span class="token keyword">await</span> navigator <span class="token punctuation">.</span> mediaDevices <span class="token punctuation">.</span> <span class="token function">getUserMedia</span> <span class="token punctuation">(</span> constraints <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token keyword">const</span> videoElement <span class="token operator">=</span> document <span class="token punctuation">.</span> <span class="token function">querySelector</span> <span class="token punctuation">(</span> <span class="token string">'video#localVideo'</span> <span class="token punctuation">)</span> <span class="token punctuation">;</span> videoElement <span class="token punctuation">.</span> srcObject <span class="token operator">=</span> stream <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span> error <span class="token punctuation">)</span> <span class="token punctuation">{</span> console <span class="token punctuation">.</span> <span class="token function">error</span> <span class="token punctuation">(</span> <span class="token string">'Error opening video camera.'</span> <span class="token punctuation">,</span> error <span class="token punctuation">)</span> <span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> |
The HTML section needs a typical video through the getUserMedia function. We should use 2 attributes:
autoplay: Let the video auto be played every time a new stream is added to the same elementplaysinline: Allows video to play inline, instead of just full screen, on some mobile browsers
You should also use control = "false" for live strean streams, unless the user can pause them:
1 2 3 4 5 6 7 | <span class="token tag"><span class="token tag"><span class="token punctuation"><</span> html</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span> head</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span> title</span> <span class="token punctuation">></span></span> Local video playback <span class="token tag"><span class="token tag"><span class="token punctuation"></</span> video</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span> head</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span> body</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span> video</span> <span class="token attr-name">id</span> <span class="token attr-value"><span class="token punctuation">=</span> <span class="token punctuation">"</span> localVideo <span class="token punctuation">"</span></span> <span class="token attr-name">autoplay</span> <span class="token attr-name">playsinline</span> <span class="token attr-name">controls</span> <span class="token attr-value"><span class="token punctuation">=</span> <span class="token punctuation">"</span> false <span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span> body</span> <span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span> html</span> <span class="token punctuation">></span></span> |
Conclude.
Ok, so I’ve finished my little research today. The article is still flawed. Hope everyone comments and suggestions for me. Thank you for watching all of your posts
References
https://www.tutorialspoint.com/webrtc/webrtc_architecture.htm