JP5651872B2 - Receiving device, screen frame transmission system and method - Google Patents

Description

  The present invention relates to a receiving apparatus, a screen frame transmission system, and a method, and more particularly, to a receiving apparatus, a screen frame transmission system, and a method capable of adjusting a latency delay.

  Screen sharing is one of the important applications of information technology. For example, remote control is performed by a remote desktop, or presentation is performed by connecting to a projector by a wireless communication method. The screen sharing techniques known so far are only suitable for the transmission of static content due to the low frame rate. As computer performance, image compression technology, and wireless network bandwidth continue to evolve, current screen sharing technology has enabled wireless networks to smoothly display video played on remote desktops. In this field, the screen frame data is obtained losslessly by the operating system through a software program.

  When a known remote desktop screen frame is transmitted, the receiving end normally does not buffer the screen frame data, so the latency from the transmission end transmitting the screen frame to the receiving end displaying the screen frame is transmitted. Delay is reduced. However, when it is necessary to smoothly transmit a desktop including video content in a wireless communication environment, it is necessary to buffer screen frame data at the receiving end. In most network transmission environments, especially wireless networks, the arrival time of the transmission package at the receiving end is not constant, so buffering screen frame data at the receiving end is smooth and jittery at the receiving end. It is essential to maintain no playback speed. However, when the receiving end buffers the screen frame data, if the transmitting end transmits the screen frame, the latency delay between the receiving end displaying the screen frame increases. If the latency delay increases during remote control, the immediate control desired by the user cannot be achieved. For example, when the user hits the keyboard remotely, after only a few seconds, the display screen responds and the user cannot easily control the remote host.

  Therefore, it is necessary to provide a screen frame transmission method and system, smooth the video being played back, and provide low latency delay control when operating the host.

  The present invention provides a receiving apparatus, a screen frame transmission system and method, and an object thereof is to improve the above-described problems.

  In order to achieve the above object, the present invention provides a receiving apparatus, a screen frame transmission system and method, detects a user operation, controls the system, switches between an operation mode and a video mode, Since less screen frame data is buffered, the latency delay to display the screen is reduced, thereby providing the user with low latency delay control. During the video mode, a lot of screen frame data is buffered, which facilitates video playback.

According to an embodiment, a screen frame transmission system includes a transmission device and a reception device. The transmission device is electrically connected to a capture module for capturing screen frame data for screen sharing , a first communication interface for converting and transmitting the screen frame data to a communication signal, and the capture module and the first communication interface. A first processing unit for processing the screen frame data and transmitting it via the first communication interface. The receiving device buffers a second communication interface that receives a communication signal, buffers the screen frame data, and causes the display device to update the screen according to the screen frame data. A detection module that outputs a mode switching signal corresponding to the video mode, and a second communication interface, a buffer, and a detection module are electrically connected, the buffer is accessed, and the receiving device is controlled based on the mode switching signal And a second processing unit that switches to the operation mode or the video mode. The buffer buffers the screen frame data of the first reproduction period in the video mode, and the second processing unit in the operation mode. Buffer screen frame data Ngushi and first regeneration period is longer than the second reproduction period.

In another embodiment, a receiving device and a transmission device are combined in a screen frame transmission system. The receiving device includes a communication interface, a buffer, a detection module, and a processing unit. The communication interface receives a communication signal including screen frame data for screen sharing from the transmission device. The buffer buffers the screen frame data, and the display device updates the screen based on the screen frame data. The detection module detects a user operation and outputs a mode switching signal corresponding to the operation mode or the video mode. The processing unit is electrically connected to the communication interface, the buffer and the detection module, accesses the buffer, controls the receiving device according to the mode switching signal, switches to operation mode or video mode, the buffer is in video mode The screen frame data of the first reproduction period is buffered, the screen frame data of the second reproduction period is buffered in the reproduction mode, and the first reproduction period is longer than the second reproduction period.

According to still another embodiment, a screen frame transmission method includes a step of capturing screen frame data for screen sharing at a transmission end and transmitting the screen frame data at a reception end, detecting a user operation, an operation mode, or Outputting a mode switching signal corresponding to the video mode; receiving and buffering screen frame data at the receiving end; and updating the screen based on the screen frame data by the display device; and receiving end In accordance with the mode switching signal, the mode is switched to the operation mode or the video mode. During the video mode, the screen frame data in the first playback period is buffered. And a first reproduction period Longer than the second playback period.

  Receiving device, screen frame transmission system and method detect user's operation, control system, switch between operation mode and video mode, during operation mode, less screen frame data is buffered and screen frame The latency latency to display is short and provides the user with low latency delay control. During the video mode, a lot of screen frame data is buffered and the video playback is smooth.

1 is a diagram illustrating a screen frame transmission system according to an embodiment of the present invention. FIG. 5 is a flowchart of a screen frame transmission method according to an embodiment of the present invention.

  FIG. 1 is a diagram illustrating a screen frame transmission system 10 including a transmission device 11 and a reception device 12 according to an embodiment of the present invention. The transmission device 11 includes a capture module 111, a first communication interface 112 and a first processing unit 113. The capture module 111 captures screen frame data. For example, the capture module 111 obtains lossless screen frame data through a software program by the operating system. The first communication interface 112 converts the screen frame data captured by the capture module 111 into a communication signal and transmits it. The first processing unit 113 is electrically connected to the capture module 111 and the first communication interface 112, processes the screen frame data captured by the capture module 111, and transmits the screen frame data via the first communication interface 112. According to the embodiment, an encoding module 114 is installed between the acquisition module 111 and the first processing unit 113, and the encoding module 114 encodes the screen frame acquired by the acquisition module 111, and the first processing unit. The calculation burden of 113 is reduced.

  The receiving device 12 includes a second communication interface 121, a buffer 123, a second detection module 124, and a second processing unit 122. The second communication interface 121 receives a communication signal from the transmission device 11. According to the embodiment, the first communication interface 112 and the second communication interface 121 comply with a wired or wireless network communication standard. The second processing unit 122 is electrically connected to the second communication interface 121, the buffer 123, and the second detection module 124. The second processing unit 122 receives the demodulated screen frame data from the second communication interface 121, buffers the screen frame data in the buffer 123 as necessary, and the display device 20 stores the screen frame data in the buffer 123. The screen is updated based on the screen frame data.

  The second detection module 124 detects a user operation and outputs a mode switching signal corresponding to the operation mode or the video mode. For example, the second detection module 124 detects an input signal from a human interface device (HID), for example, a keyboard or a mouse, and determines whether the user is performing an operation. When the HID input signal is detected, it indicates that the user is performing an operation, and the second detection module 124 outputs a mode switching signal corresponding to the operation mode. If the input signal is not detected within a predetermined period, it indicates that the user has stopped the operation, and then the second detection module 124 outputs a mode switching signal corresponding to the video mode.

  Continuing the above description, the second processing unit 122 controls the receiving device 12 based on the mode switching signal of the second detection module 124 to switch to the operation mode or the video mode. During the video mode, the second processing unit 122 buffers screen frame data for the first playback period in the buffer 123. During the operation mode, the second processing unit 122 uses the buffer 123 to screen screen frames for the second playback period. Data is buffered and the first playback period is longer than the second playback period. In other words, since the transmission device 11 transmits a screen frame during the video mode, the latency delay until the receiving end 12 displays the screen frame is long. On the other hand, the latency delay is short during the operation mode. Since the transmission apparatus 11 transmits a screen frame as the second playback period is shorter, it can be predicted that the latency delay until the receiving end 12 displays the screen frame is shortened.

  According to the embodiment, the second processing unit 122 reproduces the screen frame data in the buffer 123 from the reproduction module 125. For example, the playback module 125 is implemented by software, although not limited thereto. During the video mode, the playback module 125 waits until a predetermined buffer period, and then plays back the screen frame data buffered in the buffer 123 at the initial playback speed according to the time stamp of each screen frame data. When the receiving device 12 switches from the video mode to the operation mode, the playback module 125 accelerates and outputs the screen frame data buffered during the video mode in order to quickly reduce the latency delay. The playback module 125 ignores the time stamp of each screen frame data and outputs the screen frame data buffered in the buffer 123 at a higher playback speed. The playback module 125 directly abandons the screen frame data buffered in the buffer 123 in the video mode. Then, the screen frame data received by the receiving device 12 is a screen frame for the user's operation period. Because the latency delay is short, the user is unaware and can therefore provide low latency delay control.

  On the other hand, when the receiving device 12 switches from the operation mode to the video mode, the playback module 125 pauses the output of the screen frame data buffered in the buffer 123 or the screen frame buffered in the buffer 123. The buffer 123 is made to buffer more screen frame data by reducing the data output rate, for example, by reducing the frame rate of the video output. After the screen frame data corresponding to the set playback period is accumulated, the playback module 125 outputs the screen frame at the initial playback speed according to the time stamp of each clean frame data.

  According to the embodiment, the receiving device 12 is built in the display device 20. The receiving device 12 also includes a first connection interface 126 that is electrically connected to the second processing unit 122. The display device 20 is electrically connected to the receiving device 12 through the first connection interface 126. For example, the first connection interface 126 is a video connection port. The display device 20 is an electronic device that can display a frame, such as a television, a display, or a projector.

  According to the embodiment, the receiving device 12 includes a second connection interface 127 that is electrically connected to the second processing unit 122. The human interface device (HID) 30 is electrically connected to the receiving device 12 through the second connection interface 127. As a result, the user operates the remote host using the human interface device (HID) 30 connected to the receiving device 12. For example, the second connection interface 127 is a USB interface. According to the above structure, a display device 20 without an input device, for example a projector, is connected to a receiving device 12 including a human interface device (HID) 30 to allow a user to control a remote host, for example, Select the video to play.

  It should be noted that the transmission device 11 includes a first detection module 115 that is electrically connected to the first processing unit 113. Similarly, the first detection module 115 detects a user operation, and outputs a corresponding mode switching signal to the receiving device 12. Based on the mode switching signal output from the transmission device 11, the receiving device 12 switches to the corresponding playback mode. For example, a user does not operate a remote host by a human interface device (HID) connected to the receiving device 11 but uses a wireless keyboard or a wireless mouse to perform an operation. Based on the mode switching signal output from the device 11, it is possible to switch to the corresponding playback mode, and to provide low latency delay control.

  FIG. 2 is a diagram illustrating a screen frame transmission method according to the present embodiment. First, the transmission end captures screen frame data and transmits it to the reception end (S21). The transmission end or the reception end detects a user operation and outputs a mode switching signal corresponding to the operation mode or the video mode (S22). Thereafter, the receiving end receives the screen frame data transmitted by the transmitting end, buffers it, and receives the mode switching signal (S23). The receiving end determines whether to switch to the operation mode or the video mode based on the mode switching signal (S24). When switching to the video mode, the transmission end buffers screen frame data having a long reproduction period (S25), and since the transmission end transmits the screen frame, the latency delay until the reception end displays the screen frame is lengthened. , The video plays smoothly. When switching to the operation mode, the receiving end buffers screen frame data having a short reproduction period (S26), and since the transmission end transmits the screen frame, the latency delay until the receiving end displays the screen frame is shortened. This provides the user with low latency delay control. Finally, the display device updates the screen based on the screen frame data buffered on the receiving end (S27). Since the other detailed description has been described above, the description is omitted here.

  In summary, the receiving device, screen frame transmission system and method can detect the user's operation, control the system, switch between operation mode and video mode, and less screen frame data is buffered during operation mode. Thus, the latency delay for displaying the screen frame is short, and the user is provided with low latency delay control. During the video mode, a lot of screen frame data is buffered and the video playback is smooth.

  In the present invention, preferred embodiments have been disclosed as described above. However, the present invention is not limited to the present invention, and any person who is familiar with the technology can use various methods within the spirit and scope of the present invention. Variations and moist colors can be added, so the protection scope of the present invention is based on what is specified in the claims.

10 Screen Frame Transmission System 11 Transmission Device 111 Capture Module 112 First Communication Interface 113 First Processing Unit 114 Encoding Module 115 First Detection Module 12 Receiving Device 121 Second Communication Interface 122 Second Processing Unit 123 Buffer 124 Second Detection Module 125 Playback module 126 First connection interface 127 Second connection interface 20 Display device 30 Human interface device (HID)
S21 to S27 steps

Claims (14)

A screen frame transmission system,
Capture module that captures screen frame data for screen sharing,
A first communication interface for converting and transmitting the screen frame data to a communication signal; and electrically connecting the acquisition module and the first communication interface to process the screen frame data and via the first communication interface A transmission device including a first processing unit for transmission;
A second communication interface for receiving the communication signal;
A buffer for buffering the screen frame data and causing a display device to update the screen according to the screen frame data;
A second detection module that detects a user operation and outputs a mode switching signal corresponding to an operation mode or a video mode; and electrically connected to the second communication interface, the buffer, and the detection module; A receiver including a second processing unit that accesses the buffer and controls switching to an operation mode or a video mode based on the mode switching signal;
The buffer is, in the video mode, the screen frame data in the first reproduction period to buffering, when switched to the video mode, or pause the output of the screen frame data, or the screen frame When the screen frame data corresponding to the first reproduction period is sufficiently accumulated, the output speed of the data is reduced, and the normal output speed is restored.
When the buffer switches to the operation mode so as to buffer the screen frame data in the second playback period shorter than the first playback period in the operation mode, the buffer buffered during the video mode A screen frame transmission system characterized in that screen frame data is discarded or output at an accelerated rate . The screen frame transmission system according to claim 1 , wherein the second detection module detects an input signal of a human interface device (HID) and determines whether the user is performing an operation. The second detection module, the predetermined time period, does not detect an input signal of a human interface device (HID), according to claim 1 or 2, and outputs the mode switching signal corresponding to the video mode Screen frame transmission system. The receiving device includes a first connection interface electrically connected to the second processing unit, and the display device is electrically connected to the receiving device by the first connection interface. The screen frame transmission system according to any one of claims 1 to 3 . The receiving device includes a second connection interface electrically connected to the second processing unit, and a human interface device (HID) is electrically connected to the receiving device by the second connection interface. screen frame transmission system according to claims 1 to 4 or one paragraph, wherein. The transmission device includes a first detection module electrically connected to the first processing unit, detects the user's operation, and outputs the corresponding mode switching signal to the reception device. The screen frame transmission system according to any one of claims 1 to 5 . The transmission device includes an encoding module for electrically connecting between said capture module the first processing unit, wherein the screen frame from claim 1, characterized in that the coding of 6 any one The screen frame transmission system according to item. A receiver combined with a screen frame transmission system including a transmission device for transmitting a communication signal including screen frame data for screen sharing,
A communication interface for receiving the communication signal;
A buffer for buffering the screen frame data and causing a display device to update the screen based on the screen frame data;
A detection module that detects a user operation and outputs a mode switching signal corresponding to an operation mode or a video mode;
Processing that is electrically connected to the communication interface, the buffer, and the detection module, accesses the buffer, controls the receiving device according to the mode switching signal, and switches to the operation mode or the video mode Unit,
Including
The buffer is, in the video mode, the screen frame data in the first reproduction period to buffering, when switched to the video mode, or pause the output of the screen frame data, or the screen frame When the screen frame data corresponding to the first reproduction period is sufficiently accumulated, the output speed of the data is reduced, and the normal output speed is restored.
When the buffer switches to the operation mode so as to buffer the screen frame data in the second playback period shorter than the first playback period in the operation mode, the buffer buffered during the video mode A receiving apparatus characterized in that screen frame data is discarded or output at an accelerated rate . 9. The apparatus according to claim 8 , further comprising a first connection interface electrically connected to the processing unit, wherein the display device is electrically connected to the receiving device by the first connection interface. The receiving device described. The electronic device further comprises a second connection interface electrically connected to the processing unit, wherein a human interface device (HID) is electrically connected to the receiving device by the second connection interface. Item 10. The receiving device according to Item 8 or 9 . A screen frame transmission method,
A transmission end to capture screen frame data for screen sharing and transmit it to a reception end;
A detection step of detecting a user operation and outputting a mode switching signal corresponding to the operation mode or the video mode;
Receiving, buffering, and causing the display device to update the screen based on the screen frame data at the receiving end; and
In the receiving end, in accordance with the mode switching signal, the operation mode, or is switched to the video mode, in the video mode, to so that the screen frame data in the first reproduction period is buffered, cut in the video mode In other words, the receiving end pauses the output of the screen frame data or reduces the output speed of the screen frame data, and the screen frame data corresponding to the first reproduction period is sufficiently accumulated. was time to recover to the normal output rate, in the operation mode, so that the screen frame data before Symbol second reproduction period shorter than the first regeneration period is buffered, when switched to the operation mode, the The screen frame data buffered while the receiving end is in the video mode Abandoned, or screen frame transmission method characterized by comprising the step of outputting an accelerating rate. The method of claim 11 , wherein the detecting step detects an input signal of a human interface device (HID) to determine whether the user is performing an operation. The detection step, in a predetermined period of time, does not detect an input signal of a human interface device (HID), in claim 11 or 12, characterized in that it comprises a step of outputting the mode switching signal corresponding to the video mode The screen frame transmission method as described. The detection step, the screen frame transmission method according to claims 11 to 13 or one item, characterized in that to be executed by at least one of the receiving end and the transmitting end.

Images