JP6532159B2 - Real time video communication quality evaluation method and system - Google Patents

Description

  The present invention relates to a method and system for evaluating quality of real-time video communication, and more particularly, to a method and system for evaluating quality suitable for evaluating quality of real-time video communication by WebRTC.

  Conventionally, in order to make a video call using an information terminal such as a PC, a smartphone or a tablet, it was necessary to separately install dedicated software on the terminal, and this is an introduction for users who are not used to operating the information terminal. The threshold was high.

  However, in recent years, WebRTC (Web Real-Time Communication) attracts attention as a technology for enabling real-time video communication of audio and video easily with only a web browser, and is disclosed in, for example, Non-Patent Documents 1 and 2 There is.

  WebRTC is an API that enables real-time communication with a web browser, and a web browser is usually installed on a PC or a smartphone from the beginning, so the user does not have to install it anew. Also, according to WebRTC, a device such as a microphone or a camera connected to or built in a PC, a smartphone, or a tablet terminal can be accessed via JavaScript (registered trademark) API.

  Therefore, as shown in FIG. 6, the user must execute real-time video communication service such as video call using WebRTC technology only by accessing Web page including JavaScript (registered trademark) describing WebRTC service. Can.

  The standard of WebRTC is formulated in W3C (World Wide Web Consortium), browser compatible API (mainly implemented by JavaScript (registered trademark)), and communication protocol between browsers in IETF (Internet Engineering Task Force).

JP 2014-90364 A

WebRTC 1.0: Real-time Communication Between Browsers [http://www.w3.org/TR/webrtc/] Rtcweb Status Pages [http://tools.ietf.org/wg/rtcweb/]

  Not limited to WebRTC, when a user uses a general video call service or application, tests such as connection confirmation and video transmission / reception are often performed in advance. For example, as general test items, (1) Outgoing test (can you establish a connection with the other party by yourself?) (2) Incoming test (can you accept the outgoing call from the other party and establish a connection with the other party? ), (3) Video transmission test (Can video be transmitted with appropriate quality to the other party on the established connection), and (4) Video reception test (Receive video from the other party with the appropriate quality on the established connection) Can you do it?

  Then, through the above-mentioned series of tests, confirmation of connection and video quality, and fault isolation when testing is unsuccessful are performed. Here, in order to conduct the incoming test in (2) above, in general, it is necessary for a person's terminal to make an outgoing call operation by a person, so when carrying out the above series of tests, The schedule adjustment of, the arrangement of the person who monitors and operates the other party's terminal, etc. are required, and it takes time and effort to confirm.

  Patent Document 1 discloses a technology that enables connection confirmation of a video of a transmission terminal at only one site. According to this technology, no adjustment with the other party is required, and the test can be performed only by the sender.

  However, Patent Document 1 assumes that video is transmitted via a relay device installed in a network, and as in P2P, it is assumed that video may be transmitted and received directly between transmitting and receiving terminals without using a relay device. I did not.

  Further, in the video transmission / reception test according to Patent Document 1, the call origination / reception terminal is the same terminal, and the video is looped back from the self-side terminal by the relay device and returns to the self-side terminal. For this reason, it is not possible to confirm the connection from the opposite terminal as the actual communication partner or the quality of the video received from the opposite terminal.

  The object of the present invention is to solve the above technical problems, and in the video communication test in the video communication, it is not necessary to adjust each test of the above (1) to (4) with the other side. It is to be able to carry out simply by the operation of.

  In particular, by using WebRTC technology, it is not necessary for the user to execute terminal updates, software updates, etc. by themselves, and it is not necessary to provide the terminal with an automatic update function, and JavaScript (registered trademark) The purpose is to solve the above technical problems simply by changing.

  In order to achieve the above object, the present invention is characterized in that it has the following configuration in a method of evaluating the quality of a camera image transmitted by the own terminal to the opposite terminal in the own terminal.

  (1) A procedure of establishing a first real-time video communication channel between the web browser of the self-side terminal and the web browser of the other-party terminal and transmitting camera video, and of the other-party terminal receiving the camera video A procedure of establishing a second real-time video communication path with the web browser of the self-side terminal led by the web browser, and returning and transmitting the received camera image, and the web browser of the self-side terminal being sent back And a procedure for receiving and displaying a camera image.

  (2) The web browser of the terminal on the local side is configured to display the transmitted camera video and the back-transmitted camera video.

  (3) The self-side terminal acquires quality statistical information of the transmitted camera video and back-transmitted camera video, and displays each quality statistical information together with the camera video.

  According to the present invention, the following effects are achieved.

  (1) In video communication using WebRTC technology, the other party's terminal that receives the camera image transmitted by the own-side terminal automatically sends it back to the own-side terminal. The reproduction state of the camera image can be confirmed without user operation on the other terminal. Therefore, the quality evaluation of real-time video communication can be performed easily and without adjustment with the other party.

  (2) Since the own terminal displays not only the camera image sent back but also the camera image sent to the other terminal by the own terminal, it is possible to evaluate the quality change of the video communication from a subjective viewpoint become.

  (3) Since the terminal on the own side displays not only the transmitted camera video and the camera video sent back but also the quantitative quality statistical information at the time of transmitting and receiving each camera video, the quality of the video communication can be increased. It is also possible to evaluate objectively, and it is also possible to perform simple cause isolation when a problem occurs in communication.

It is a functional block diagram of a WebRTC video communication system to which the present invention is applied. It is a figure showing signs that each terminal of a self-party and a partner side acquires a test Web page from a Web server. It is the figure which showed the evaluation method of quality statistical information. It is the sequence flow of the video communication between the self-party and the other party's terminal. It is the figure which showed the example of a display of the evaluation result of the communication quality in a self-side terminal. It is a figure for demonstrating WebRTC.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of the main part of a WebRTC video communication system to which the present invention is applied.

  The system according to the present embodiment includes an own terminal Ns mounted or connected with a camera function, an opposite terminal Nd displaying a camera image, a web server providing a test web page to each terminal, and a web browser In the network, a signaling server that mediates information exchange for establishing an RTC Peer Connection and a relay server that relays P2P communication between web browsers are connected to a network.

  The camera function of the self-side terminal Ns is not limited to hardware, and virtual camera software may be installed.

  As shown in FIG. 2, the self-side terminal Ns and the other-side terminal Nd access the web server using the web browser installed in advance, and use the html file constituting the web page and the web RTC technology. Download a JavaScript (registered trademark) file for video communication. In addition, each Web browser of the self-side terminal Ns and the other-side terminal Nd registers in advance a browser ID for identifying each browser in the signaling server.

  The own terminal Ns specifies the browser ID of the other terminal Nd and starts real-time video communication with the other terminal Nd. At this time, the web browser of the self terminal Ns and the partner terminal Nd sends and receives information for establishing a video communication path (RTC Peer Connection) by the WebRTC via the signaling server.

  If possible, video communication between the own terminal Ns and the other terminal Nd is performed by P2P communication without passing through a server or the like, and when P2P communication is not possible, it is performed through a relay server. For communication protocols and server operations for realizing communication via relay servers, Traversal Using Relay NAT (TURN) standardized as RFC5766 in IETF can be used.

  In the web browser of the own terminal Ns, the communication path establishment unit 101 takes the initiative in establishing a first real-time video communication path (RTC Peer Connection) with the web browser of the opposite terminal. The video communication unit 102 transmits the camera video (transmission video) through the first real-time video communication channel to the web browser of the remote terminal Nd, and is loop-back transmitted from the remote terminal Nd through a second real-time video communication channel described later. Camera image (received image) is received.

  The video evaluation unit 103 displays the transmission video and the reception video on a display (not shown). The quality statistical information evaluation unit 104 acquires quality statistical information of the transmission video and the reception video, and displays them together with the transmission video and the reception video.

  In the present embodiment, as shown in FIG. 3, the frame rate, the transmission rate, the RTT, and the packet loss rate are evaluated as quality statistical information of video communication through the first real-time video communication channel. Also, the frame rate, the reception rate and the packet loss rate are evaluated as quality statistical information of video communication through the second real time video communication channel.

  In the web browser of the opposite terminal Nd, the communication channel establishment unit 201 establishes itself as a second real-time video communication channel (RTC Peer Connection) between itself and the web browser of the own terminal. The return transmission unit 202 receives the transmission video from the web browser of the self-side terminal Ns through the first real-time video communication path, and loops back and transmits this to the self-side terminal Ns through the second real-time video communication path.

  FIG. 4 is a sequence flow diagram showing a procedure of processing performed between the start of the test and the video communication between the own terminal Ns and the opposite terminal Nd, and at time t1 to t6, a camera image is displayed. Prior to transmission and reception, as shown in FIG. 2, a procedure is performed in which the self-side terminal Ns and the other-side terminal Nd acquire a test Web page from the Web server.

  At time t1, a page acquisition request is transmitted in advance from the opposite terminal Nd to the Web server. At time t2, a page response is transmitted from the Web server to the opposite terminal Nd. The Web server provides a WebRTC video transmission / reception test service, and the page response includes a script (JavaScript (registered trademark)) for executing video communication using WebRTC technology.

  The script may be directly described in the test page, or a link to the script file may be included in the test page. At time t3, an ID unique to the browser of the opposite terminal Nd is registered in the signaling server by the opposite terminal Nd.

  Thereafter, at time t4, a page acquisition request is transmitted from the self terminal Ns to the Web server. At time t5, a page response is transmitted from the Web server to the own terminal Ns. At time t6, an ID unique to the browser of the self-side terminal Ns is registered in the signaling server. The connection to the signaling server is executed upon execution of the script.

  In the other party's terminal Nd, if a series of procedures up to ID registration are executed at an arbitrary timing in advance before the start of the test, no human operation is required thereafter.

  In addition, the connection between the own terminal Ns and the other terminal Nd and the signaling server is always established to transmit and receive the SDP message described later, and in the signaling server, the ID transmitted from each terminal at the time of connection establishment. And the connection.

  Next, according to the procedure defined in the WebRTC standard, a procedure for establishing a communication path (RTCPeerConnection) for real-time video communication between the own terminal Ns and the opposite terminal Nd is executed.

  That is, at time t7, the SDP offer message is transmitted from the own terminal Ns to the opposite terminal Nd via the signaling server. This SDP (Session Description Protocol) message includes information such as media type (audio, video), codec format, IP address and port number, and protocol used for data transfer (TCP, UDP, etc.).

  When the correspondent terminal Nd receives the SDP offer from the own terminal Ns, it transmits similar information to the own terminal Ns in an SDP answer message at time t8. At time t9, a connection check is performed between the self terminal Ns and the remote terminal Nd, and a pair of an IP address and a port number that can be communicated by P2P is determined.

  In addition, when it can not communicate by P2P, it becomes communication via relay server. As for the details of the procedure, a method called ICE (Interactive Connectivity Establishment) is standardized in the IETF (RFC 5245).

  When an RTC Peer Connection (first real-time video communication channel) is established between the self terminal Ns and the remote terminal Nd according to the procedure of time t9, at time t10, the self terminal Ns directs the remote terminal Nd to the remote terminal Nd. The camera image is transmitted (in FIG. 2, only the P2P case is described, but when P2P communication can not be performed, the camera image is transmitted via the relay server).

  Here, the camera image may be an input image from a physical camera device, or an image file obtained from the outside may be input via a virtual camera.

  When receiving the camera image, the opposite terminal Nd attempts to establish an RTC Peer Connection toward the terminal Ns at time t11. The procedure in the opposite terminal Nd is the same as the procedure in the case of trying from the own terminal Ns.

  When RTC Peer Connection (second real-time video communication channel) is established at time t13, the other party terminal Nd uses the RTC Peer Connection at time t14 to make a camera picture received from itself. It sends back to the side terminal Ns.

  According to the above procedure, the self terminal Ns can check whether or not its own video has reached the other terminal. That is, the transmission test of the above (1) and the video transmission test of the above (3) can be carried out without the need for the operation at the opposite terminal Nd.

  At the same time, it is possible to check whether or not the own terminal Ns can be called from the opposite terminal Nd, and whether the own terminal Ns can receive an image transmitted from the other terminal Nd after the call. That is, the incoming test of (2) and the video reception test of (4) can also be performed without the need for an operation at the opposite terminal Nd.

  FIG. 5 is a display example of evaluation results in the own terminal Ns of real-time video communication, and a camera image (transmission image) V1 transmitted from the own terminal Ns to the opposite terminal Nd and the other terminal from the opposite terminal Nd A camera image (received image) V2 sent back to Ns is displayed in parallel.

  Furthermore, in the present embodiment, quality statistical information (here, frame rate, transmission rate, RTT and packet loss rate) when camera video is transmitted from the own terminal Ns to the opposite terminal Nd is displayed, and further the other terminal Quality statistical information (here, a frame rate, a reception rate, and a packet loss rate) when the camera image is sent back from the Nd to the own terminal Ns by RTC Peer Connection is also displayed.

  In the Web RTC technology, a JavaScript (registered trademark) API capable of acquiring statistical information of RTC Peer Connection is defined, and in a script acquired by the self-side terminal Ns from a Web server, a program for periodically calling the API is described By doing this, it is possible to periodically acquire and update statistical information in the RTC Peer Connection directed from the self side to the other side in the self-side terminal Ns and in the RTC Peer Connection directed from the other side to the self side.

  As described above, by displaying the statistical information of the camera video, it is possible to subjectively and quantitatively check not only whether the own video has arrived at the opposite terminal but also the quality thereof.

  In addition, in the case where a problem occurs in communication, it is also possible to easily isolate the cause. For example, when the received video is not displayed, if quality statistical information of the transmitted video is displayed, it is known that a problem has occurred in the communication from the opposite side to the own side.

  101 ... communication channel establishment unit, 102 ... video communication unit, 103 ... video evaluation unit, 104 ... quality statistical information evaluation unit, 201 ... communication channel establishment unit, 202 ... return transmission unit, Ns ... own terminal, Nd ... other party Terminal

Claims (7)

In a method of evaluating the quality of a camera image transmitted by the own terminal to the opposite terminal in the own terminal,
A procedure of establishing a first real-time video communication path by the WebRTC between the web browser of the other party's terminal and the web browser of the own party's own terminal,
A procedure for transmitting a camera image on the first real-time image communication channel from the web browser of the own terminal to the web browser of the other terminal;
A step of establishing a second real-time video communication path by the WebRTC between the terminal and the web browser of the self-side terminal , led by the web browser of the other-side terminal that has received the camera video;
A procedure of returning and transmitting the received camera image through the second real-time image communication channel from the Web browser of the opposite terminal to the Web browser of the own terminal;
And d. A method of evaluating the quality of real-time video communication, including the step of displaying the camera video that has been sent back by the web browser of the self-side terminal.   The method for evaluating quality of real-time video communication according to claim 1, wherein the web browser of the self-side terminal displays the transmitted camera video together with the folded back transmitted camera video. The method further includes: acquiring the quality statistical information of the transmitted camera image and the back-transmitted camera image by the self-side terminal;
The method for evaluating quality of real-time video communication according to claim 2, wherein the Web browser of the self-side terminal displays each of the quality statistical information together with the transmitted camera video and the camera video sent back. In a system in which the quality of the camera image transmitted from the own terminal to the opposite terminal is evaluated in the own terminal,
The web browser of your own terminal,
A means for establishing a first real-time video communication path by the WebRTC between the other party's terminal and the web browser;
Means for transmitting a camera image over the first real time image communication channel,
A means for establishing a second real-time video communication path by the WebRTC between the Web browser of the opposite terminal and the Web browser of the own terminal;
Means for returning the received camera image through the second real-time image communication channel;
Furthermore, the web browser of the self-side terminal
A system for evaluating the quality of real-time video communication, comprising means for receiving and displaying the camera image sent back.   The system for evaluating quality of real time video communication according to claim 4, wherein the means for displaying the camera video displays the transmitted camera video together with the folded back transmitted camera video. The self-side terminal further comprises means for acquiring quality statistical information of the transmitted camera video and the back-transmitted camera video.
The system for evaluating quality of real-time video communication according to claim 5, wherein the Web browser of the self-side terminal displays the respective pieces of quality statistical information together with the transmitted camera video and the camera video sent back.   The quality evaluation system for real-time video communication according to any one of claims 4 to 6, characterized in that the own terminal includes a camera that generates the camera video.