JP7087779B2 - Terminal devices, conference systems, terminal device control methods, and programs - Google Patents

Description

The present invention relates to a terminal device, a conference system, a control method for the terminal device, and a program.

A conference system is known in which an image is shared among a plurality of terminal devices to hold a conference. For example, a conference system is known in which a shared image displayed on an information terminal, an electronic whiteboard, or the like is displayed on the display unit of a terminal device together with a conference video of other participants participating in the conference. ..

Further, in a video conference system, there is known a technique for correcting a time lag between a received audio and a video by delaying and reproducing the received audio (see, for example, Patent Document 1).

When sharing a high-resolution shared image displayed on a PC (Personal Computer), an electronic whiteboard, or the like between a plurality of terminal devices, it takes time to communicate the shared image, so the sharing destination terminal There is a problem that the time required to display the shared image on the device becomes long. As a result, for example, the time difference between the display of the shared image and the conference audio causes the conference to be hindered, for example, the explanation of the shared image starts with the conference audio before the shared image is displayed on the sharing destination terminal device. May be.

Further, as in the technique disclosed in Patent Document 1, if the received voice is only delayed and reproduced, for example, until a response from another participant is returned to the utterance of the participant of the conference. This delay may hinder the progress of the conference, as the delay will be large.

The embodiment of the present invention has been made in view of the above problems, and reduces the time difference between the shared image and the conference voice in the conference system for sharing the shared image and transmitting and receiving the conference voice. While suppressing the increase in the delay time of the conference voice.

In order to solve the above problems, the terminal device according to the embodiment of the present invention is a terminal device that communicates with another terminal device via a network, and data of a shared image shared with the other terminal device. The shared image receiving unit that receives the shared image, the audio data receiving unit that receives the audio data transmitted by the other terminal device, the shared image display unit that displays the shared image on the display unit, and the shared image receiving unit. When the data of the shared image is received, the voice data storage unit that stores the voice data and the voice data stored in the data storage unit are delayed from the time when the voice data is received by the voice data receiving unit. From the output unit to be output and the voice data stored in the data storage unit, the detection unit that detects a period during which there is no speech, and the voice data stored in the voice data storage unit in the detection unit. When a period without speech is detected, a part or all of the period without speech is excluded from the voice data stored in the voice data storage unit, and the voice data is output to the output unit. It has a control unit.

According to the embodiment of the present invention, in the conference system for sharing the shared image and transmitting / receiving the conference audio, the time difference between the shared image and the conference audio is reduced, and the increase in the delay time of the conference audio is suppressed. can do.

It is a figure which shows the example of the system configuration of the conference system which concerns on one Embodiment. It is a figure which shows the image of the display screen displayed by the conference terminal which concerns on one Embodiment. It is a figure which shows the example of the hardware composition of the conference terminal which concerns on one Embodiment. It is a figure which shows the example of the functional structure of the conference terminal which concerns on 1st Embodiment. It is a figure (1) for demonstrating the output process of the conference image which concerns on 1st Embodiment. It is a figure (2) for demonstrating the output process of the conference image which concerns on 1st Embodiment. It is a sequence diagram which shows the example of the processing of the conference system which concerns on 1st Embodiment. It is a figure which shows the example of the functional structure of the conference terminal which concerns on 2nd Embodiment. It is a figure (1) for demonstrating the output process of the conference image which concerns on 2nd Embodiment. It is a figure (2) for demonstrating the output process of the conference image which concerns on 2nd Embodiment. It is a sequence diagram which shows the example of the processing of the conference system which concerns on 2nd Embodiment. It is a flowchart which shows another example of the processing when the shared image which concerns on 2nd Embodiment is received.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
<System configuration>
FIG. 1 is a diagram showing an example of a system configuration of a conference system according to an embodiment. The conference system 100 includes, for example, a plurality of conference terminals 110a, 110b, ..., A call control server 130, an image distribution server 140, and the like that are communicably connected to a network 160 such as the Internet or a LAN (Local Area Network). Have. In the following description, when indicating an arbitrary conference terminal among a plurality of conference terminals 110a, 110b, ..., "Conference terminal 110" is used.

The conference terminal 110 is a terminal device having a computer configuration, may be a dedicated terminal for the conference system 100, or is a general-purpose information terminal that executes an application program (hereinafter referred to as an application) of the conference system 100. There may be. Here, as an example, the following description will be given assuming that the conference terminal 110 is a dedicated terminal for the conference system 100.

The conference terminal 110 can communicate with one or more other conference terminals 110 via the image distribution server 140, and can transmit and receive images and voices of the conference, for example, under the control of the call control server 130. Thereby, for example, the user of the conference terminal 110a can have a video conference with another user of the conference terminal 110b via the image and the sound of the conference transmitted and received in real time. The video conference may be called a video conference, a remote conference, or the like.

The call control server 130 is an information processing device having a computer configuration or a system including a plurality of information processing devices, and controls communication for holding a conference between a plurality of conference terminals 110.

The image distribution server 140 is an information processing device having a computer configuration, or a system including a plurality of information processing devices, and distributes video data including audio and moving images between conference terminals 110 participating in a conference. Relay).

For example, when the conference terminals 110a and 110b are participating in the same conference, the image distribution server 140 receives the video data transmitted from the conference terminal 110a and distributes the received video data to the conference terminal 110b. Further, the image distribution server 140 receives the video data transmitted from the conference terminal 110b and distributes the received video data to the conference terminal 110a.

In this way, the image distribution server 140 receives the video data transmitted from one conference terminal 110 participating in the conference, and distributes the received video data to the other conference terminals 110 participating in the conference. .. As a result, even when the number of conference terminals 110 participating in the conference is large, the conference terminal 110 only needs to transmit the video data to the image distribution server 140, so that the load on the conference terminal 110 can be reduced.

Further, in the conference system 100 according to the present embodiment, for example, the display screen displayed on the information terminal 120 or the like connected to the external I / F (Interface) 111 of the conference terminal 110a is used as a shared screen of another conference terminal. It can be shared with 110b, ....

FIG. 2 is a diagram showing an image of a display screen displayed by the conference terminal according to the embodiment. In the conference system 100 shown in FIG. 1, when a conference is held between the conference terminal 110a and the conference terminal 110b, for example, a display screen 200 as shown in FIG. 2 is displayed on the display unit of the conference terminal 110b. Will be done.

On the display screen 200 shown in FIG. 2, a conference image 210 captured by the conference terminal 110a, a conference video 220 captured by the conference terminal 110b, and a shared image 230 are displayed. The shared image 230 is, for example, a display screen displayed by an information terminal 120 connected to the external I / F 111 of the conference terminal 110a, an electronic whiteboard, or the like, or a still image that captures the display screen.

Here, it is assumed that the shared image 230 is a high-resolution still image such as 4K as an example. Note that 4K represents a resolution of about 4098 x 2160 dots, which is twice the vertical and horizontal resolution (4 times the area ratio) of the full high-definition resolution (1920 x 1080 dots). The 4K still image is an example of a high-resolution still image, and may have another resolution.

For example, the user of the conference terminal 110a can share the still image captured by the display screen to be explained as the shared image 230 when explaining the conference materials and the like. As a result, the user of the conference terminal 110a can freely change the display screen of the information terminal 120 while displaying the display screen to be explained on the shared image 230.

However, when sharing a high-resolution shared image displayed on a PC (Personal Computer), an electronic whiteboard, or the like between a plurality of conference terminals 110, the shared image 230 is displayed on the shared destination conference terminal 110. There is a problem that the time required for the computer becomes long. Thereby, for example, when the user of the conference terminal 110a shares the shared image 230 as shown in FIG. 2 and gives an explanation by the voice of the conference video 210, the shared image 230 is displayed on the conference terminal 110b. Before, the explanation may start with the conference video 210.

Such a time difference between the conference video 210 and the shared image 230 may be, for example, about 4 to 8 seconds depending on the environment, which may hinder the progress of the conference.

Therefore, when the conference terminal 110b according to the present embodiment receives the shared image 230, the conference video 210 (moving image + audio) is output with a delay of a predetermined period. This predetermined period includes, for example, a period from when the conference terminal 110 starts receiving the shared image 230 until the shared image 230 is displayed. As a result, the time difference between the conference video 210 and the shared image 230 is reduced, and for example, the explanation by the conference video 210 is started after the shared image 230 is displayed.

However, with this alone, for example, the delay time until the response from the conference terminal 110b is returned to the utterance of the user of the conference terminal 110a becomes large, so that this delay time facilitates the smooth progress of the conference. It may be an obstacle.

Therefore, after displaying the shared image 230, the conference terminal 110b detects a period without utterance in the conference video 210, and outputs (displays) the conference video 210 by excluding a part or all of the period without utterance. It is desirable to have a function. As a result, when the conference video 210 includes a period in which there is no utterance, it is possible to suppress an increase in the delay time that occurs when the shared image 230 is displayed.

As described above, according to the present embodiment, in the conference system 100 that shares the shared image 230 and transmits / receives the conference video 210, the conference video 210 is reduced in time between the shared image 230 and the conference video 210. It is possible to suppress an increase in the delay time of.

The system configuration of the conference system 100 described with reference to FIG. 1 is an example. For example, the conference system 100 may hold a conference between the conference terminal 110a and the conference terminal 110b without going through the call control server 130, the image distribution server 140, or the like. Further, the call control server 130 and the image distribution server 140 may be realized by one conference server 150.

Further, the conference system 100 may transmit and receive conference audio instead of the conference video (moving image + audio). In this case, when the conference terminal 110b receives the shared image 230, the conference voice is output with a delay of a predetermined period, and the conference voice is output by excluding a part or all of the period in which there is no utterance in the conference voice. Just do it.

Further, the conference terminal 110 may be an information terminal such as a notebook PC or a tablet terminal that executes a conference program corresponding to the conference system 100. Further, the shared image 230 is not limited to the display screen of the information terminal 120 or the whiteboard connected to the external I / F 111, and may be an image taken by a camera included in the information terminal or an external camera.

<Hardware configuration>
(Hardware configuration of conference terminal)
FIG. 3 is a diagram showing an example of the hardware configuration of the conference terminal according to the embodiment. The conference terminal 110 is, for example, a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, a storage 304, an input device 305, a network I / F306, a camera 307, and an image pickup element I /. It has an F308, a speaker 309, a microphone 310, a voice I / F311, a display 312, a display I / F313, an external I / F111, a system bus 314, and the like. At least a part of the input device 305, the camera 307, the speaker 309, the microphone 310, and the display 312 may be connected to the outside of the conference terminal 110.

The CPU 301 is an arithmetic unit that controls the overall operation of the conference terminal 110 by executing a program stored in the storage 304, ROM 302, or the like. The ROM 302 is, for example, a non-volatile memory that stores a program used for starting the CPU 301 such as an IPL (Initial Program Loader). The RAM 303 is a volatile memory used as a work area or the like of the CPU 301. The storage is a large-capacity storage device that stores an OS (Operating System), an application, data, and the like, and is realized by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

The input device 305 is an input device that accepts operations by the user, and is realized by, for example, an operation button, a keyboard, a touch panel, or the like. The network I / F 306 is an interface for connecting the conference terminal 110 to the network 160 and communicating with other devices.

The camera 307 is an imaging device that captures an image of a subject and acquires image data. The image sensor I / F 308 is an interface for connecting the camera 307 to the conference terminal 110. The speaker 309 converts the voice signal output from the voice I / F 311 into voice and outputs it. The microphone 310 acquires peripheral voice, converts it into a voice signal, and outputs it to the voice I / F 311. The voice I / F 311 is an interface for connecting the speaker 309 and the microphone 310 to the conference terminal 110.

The display 312 is a display device that displays the display screen of the conference terminal 110. The display I / F 313 is an interface for connecting the display 312 to the conference terminal 110.

The external I / F 111 is an interface for connecting an external device such as an information terminal 120 to the conference terminal 110. The external I / F 111 is not limited to the wired connection, and may be one that connects an external device to the conference terminal 110 by using wireless communication.

The system bus 314 connects each of the above components as shown in FIG. 2 and transmits an address signal, a data signal, and various control signals.

The call control server 130 and the image distribution server 140 have a general computer hardware configuration, but in the present embodiment, the configuration of the call control server 130 and the image distribution server 140 may be arbitrary. Therefore, the description is omitted here.

[First Embodiment]
Subsequently, the functional configuration of the conference terminal 110 in the conference system 100 according to the first embodiment will be described.

<Functional configuration>
FIG. 4 is a diagram showing an example of the functional configuration of the conference terminal 110a according to the first embodiment. It is a figure which shows the example of the functional structure of a system. In addition, in FIG. 4, it is assumed that the conference terminal 110b has the same functional configuration as the conference terminal 110a.

(Functional configuration of conference terminal)
The conference terminal (terminal device) 110 may include, for example, a communication unit 401, an operation reception unit 402, a conference video transmission unit 403, a shared image transmission unit 404, a shared image reception unit 405, a shared image display unit 406, and a conference video reception unit 407. It has a storage unit 408, a conference video output unit 409, and the like.

The conference terminal 110 realizes each of the above functional configurations by, for example, executing a predetermined program on the CPU 301 of FIG. Further, at least a part of each of the above functional configurations may be realized by hardware.

Further, in the present embodiment, the call control server 130 and the image distribution server 140 in FIG. 1 may have an arbitrary configuration, and thus are described as the conference server 150 in FIG. 3, and the call control is also described below. The explanation will be given without particularly distinguishing between the server 130 and the image distribution server 140.

The communication unit 401 is realized by, for example, a program executed by the CPU 301 of FIG. 3 and a network I / F 306, and communicates with another conference terminal 110, a conference server 150, and the like via the network 160.

For example, the communication unit 401 establishes a control session with the conference server 150, performs login processing to the call control server 130, and performs communication control such as start, end, and participation of the conference. Further, the communication unit 401 establishes a communication session with the conference server 150 under the control of the conference server 150, and shares images and conference images (shared images and conference images) with other conference terminals 110 participating in the same session. For example, moving image + voice) and the like are transmitted and received.

The operation reception unit 402 is realized by, for example, a program executed by the CPU 301 of FIG. 3, and an input operation by a user (for example, an operation such as transmission, response, destination designation, etc.) by using the input device 305 of FIG. 3 or the like. Accept.

The conference video transmission unit (audio data transmission unit, video data transmission unit) 403 is realized by, for example, a program executed by the CPU 301 of FIG. The conference video transmission unit 403 uses, for example, a conference video data including a video of a conference taken by a camera 307 or the like in FIG. 3 and audio data of a conference acquired by a microphone 310 or the like in FIG. It is transmitted to another conference terminal 110 participating in the conference via 150.

The shared image transmission unit 404 is realized by, for example, a program executed by the CPU 301 of FIG. The shared image transmission unit 404 acquires a shared image from, for example, an external device connected by an external I / F 111 or the like, and participates in the conference by using the acquired shared image data, for example, via the conference server 150. It is transmitted to another conference terminal 110.

The shared image receiving unit 405 is realized by, for example, a program executed by the CPU 301 of FIG. When the shared image receiving unit 405 transmits data of the shared image shared by the other conference terminals 110 among the plurality of conference terminals 110 participating in the conference, the shared image receiving unit 405 of the shared image transmitted by the other conference terminal 110. The data is received, for example, via the conference server 150.

The shared image display unit 406 is realized by, for example, a program executed by the CPU 301 of FIG. 3, and uses the data of the shared image received by the shared image receiving unit 405 to display, for example, a display unit such as the display 312 of FIG. Display the shared image. For example, the shared image display unit 406 causes the shared image 230 to be displayed on the conference display screen 200 as shown in FIG.

The conference video receiving unit (audio data receiving unit, video data receiving unit) 407 is realized by, for example, a program executed by the CPU 301 of FIG. The conference video receiving unit 407 receives video data including conference audio data transmitted by another conference terminal 110 participating in the conference, and stores the video data in the storage unit (audio data storage unit, video data storage unit) 408. do.

The storage unit 408 stores, for example, video data including the audio of the conference, which is realized by the program executed by the CPU 301 of FIG. 3, the storage 304, the RAM 303, and the like, and is received by the conference video receiving unit 407. Further, the storage unit 408 can store various information and data such as information such as a terminal ID for identifying the conference terminal and data of the shared image received by the shared image receiving unit 405. The storage unit 408 is an example of an audio data storage unit that stores the audio data received by the conference video receiving unit 407 when the shared image receiving unit 404 receives the shared image data.

The conference video output unit 409 is realized by, for example, a program executed by the CPU 301 of FIG. 3, and uses the video data stored in the storage unit 408 to, for example, transmit the conference video 210 and the conference audio as shown in FIG. Output. Further, when the shared image receiving unit 405 receives the shared image data, the conference video output unit 409 according to the present embodiment uses the video data stored in the storage unit 408 to display the conference video (moving image + audio). It has a function to output with a delay of a predetermined period.

The video data of the conference includes time information (time stamp) for reproducing the video, and the conference video output unit 409 may use the conference video output unit 409, for example, when the shared image receiving unit 405 receives the shared image data, for example, the video of the conference. The time information of the image is output with a delay of a predetermined period. Similarly, the conference video output unit 409 can output the conference audio with a delay by a predetermined period.

5 and 6 are diagrams for explaining the output processing of the conference video according to the first embodiment. For example, at time t1 of FIG. 5, it is assumed that the conference terminal 110a transmits a conference video including a shared image 501 and an utterance 502 for explaining the shared image 501 to the conference terminal 110b.

At this time, it is assumed that the utterance 502 of the conference voice received by the conference terminal 110b does not cause a large delay that hinders the progress of the conference. On the other hand, when the shared image 501 is a high-resolution image such as 4K, a relatively large delay time (for example, 4) is reached by the time t2 when the conference terminal 110b completes the reception of the shared image 501 and displays it. ~ 8 seconds) can occur.

Due to such a time difference between the shared image 230 and the conference video 210 in the conference terminal 110b, for example, when the utterance 502 for explaining the shared image 501 is started in the conference terminal 110b, the shared image 230 is still displayed. It may not have been done. This may hinder the progress of the conference, for example, it may be difficult for the user of the conference terminal 110b to understand the explanation of the shared image 230.

Therefore, in the present embodiment, as shown in FIG. 6, the conference video output unit 409 of the conference terminal 110b outputs the conference video with a delay of Td for a predetermined period when receiving the shared image 501. In the period from time t1 to t2 in FIG. 6, the conference video output unit 409 may display, for example, the frame displayed immediately before the time t1 as a still image, or the timing is being adjusted. It may display a display screen or the like indicating that there is.

As a method for determining the predetermined period Td, for example, the period from the start of reception of the shared image 501 by the conference terminal 110b until the shared image 501 is displayed may be set as the predetermined period Td. Further, as another example, an appropriate delay time Td may be stored in advance in a storage unit or the like according to the resolution of the shared image 501 and the communication speed.

Further, the conference terminal 110b may output the conference video with a delay by, for example, the process shown in FIG. 7.

<Processing flow>
FIG. 7 is a sequence diagram showing an example of the conference system processing according to the first embodiment. This process shows an example of the process executed by the conference system 100 when the conference terminal 110a transmits the shared image 501 while the conference is being held between the conference terminal 110a and the conference terminal 110b. There is.

In steps S701 and S702, transmission of conference video (video + audio) data is started from the conference terminal 110a to the conference terminal 110b via the conference server 150.

In steps S703 and S704, the conference video receiving unit 407 of the conference terminal 110b receives the conference video data transmitted from the conference terminal 110a, and stores the received conference video data in the storage unit 408.

In steps S705 and S706, for example, in response to a screen sharing operation by the user, transmission of shared image data is started from the conference terminal 110a to the conference terminal 110b via the conference server 150.

In steps S707 and S708, the shared image receiving unit 405 of the conference terminal 110b receives the data of the shared image transmitted from the conference terminal 110a, and the shared image display unit 406 performs a display process for displaying the shared image. It should be noted that the reception of the data of the shared image requires, for example, a time corresponding to the delay time Td in FIG.

In step S709, when the shared image receiving unit 405 of the conference terminal 110b completes the reception of the shared image, the conference video output unit 409 is notified that the reception of the shared image is completed.

Preferably, the reception completion notification includes, for example, time information (time stamp) indicating the time when the reception of the shared image data is started (or the time when the transmission of the shared image data is started).

In steps S710 and S711, when the conference video output unit 409 of the conference terminal 110b receives the reception completion notification, it reads out the conference video data stored in the storage unit 408 and starts reproducing the conference video.

Preferably, the conference video data includes time information (time stamp) indicating the time to be reproduced, and the conference video output unit 409 has, for example, the time information included in the reception completion notification and the conference to be reproduced. Play the conference video so that it matches the time information of the video.

Thereby, for example, as shown in FIG. 6, the conference terminal 110b can output the conference video including the utterance 502 explaining the shared image 501 in accordance with the display of the shared image 501 at the time t2. Become.

In the above description, the conference video (moving image + audio) is transmitted and received between the conference terminal 110a and the conference terminal 110b, but in the present embodiment, the conference video is replaced with the conference video. It can also be applied to a conference system 100 that transmits and receives only voice.

As described above, when the shared image receiving unit 405 receives the shared image, the conference video output unit 409 of the conference terminal 110b according to the present embodiment outputs at least the conference sound with a delay of a predetermined period.

Therefore, according to the present embodiment, it is possible to reduce the time difference between the shared image and the conference video in the conference system 100 that shares the shared image and transmits / receives the conference video.

[Second Embodiment]
According to the first embodiment, in a conference system in which a shared image is shared and a conference video is transmitted / received, the time difference between the shared image and the conference video can be reduced. However, with this alone, for example, the delay time until the response from the conference terminal 110b is returned to the utterance of the user of the conference terminal 110a becomes large, so that this delay time facilitates the smooth progress of the conference. It may be an obstacle. In the second embodiment, an example of reducing this delay time will be described.

<Functional configuration>
FIG. 8 is a diagram showing an example of the functional configuration of the conference terminal according to the second embodiment. As shown in FIG. 8, the conference terminal 110 according to the second embodiment has a detection unit 801 and an output control unit 802 in addition to the functional configuration of the conference terminal 110 according to the first embodiment shown in FIG. have.

The detection unit 801 detects, for example, a period during which there is no utterance in the conference voice included in the conference video data stored in the storage unit 408, which is realized by a program executed by the CPU 301 of FIG. For example, the detection unit 801 may detect a period in which the output level (or volume) of the conference voice is less than the threshold value as a period during which there is no utterance, and does not include sound waves in the frequency band corresponding to the human voice. The period may be detected as a period without speech.

The output control unit 802 is realized by, for example, a program executed by the CPU 301 of FIG. 3, and the conference video output unit 409 excludes a part or all of the period in which there is no utterance in the conference video stored in the storage unit 408. To output the conference video.

9 and 10 are diagrams for explaining the output processing of the conference video according to the second embodiment. For example, at time t1 of FIG. 9, the conference terminal 110a shall transmit the shared image 501 and the conference video including the utterance 502 for explaining the shared image 501 to the conference terminal 110b. Further, as an example, it is assumed that the utterance 902 is started again from the time t4 after a period of no utterance 901 between the times t3 and t4. Further, the conference terminal 110b displays the shared image 501 at the time t2 after the delay time Td has elapsed from the time t1 and starts the reproduction of the conference video including the utterance 502 by the process described in the first embodiment. It shall be.

At this time, the detection unit 801 of the conference terminal 110b detects the period Ts of no utterance 901 in the conference video stored in the storage unit 408. Further, the output control unit 802 of the conference terminal 110b excludes the period Ts of no utterance 901 in the conference video, and causes the conference video output unit 409 to output the conference video.

As a result, for example, as shown in FIG. 9, when the period Ts of no utterance 901 is shorter than the delay time Td, the conference video including the utterance 502 is followed by the conference video including the utterance 902.

However, if the period Ts of no utterance 901 is longer than the delay time Td, the output control unit 802 cannot exclude all of the period Ts of 901 without utterance, so that the utterance is within the range of the delay time Td. None Exclude part of the period Ts of 901. As a result, the conference terminal 110b can suppress an increase in the delay time Td due to the reception of the shared image 501.

Here, a specific example of the processing of the output control unit 802 will be described with reference to FIG. Time information (time stamp) indicating the time to be played is added to the video data and the audio data of the conference video.

For example, in FIG. 10, the relative time for the conference terminal 110a to start transmitting the shared image 501 is set to 0 seconds, and after 4.5 seconds, the conference terminal 110b completes the reception of the shared image 501 and displays the shared image 501. It shall be done. Further, in the conference terminal 110a, it is assumed that there is a period of 901 without utterance for 3 seconds after the utterance 502.

When the output control unit 802 of the conference terminal 110b excludes (skips) the period of no utterance 901, the output control unit 802 adds the delay time "4.5 seconds" to the time information "8 seconds" for reproducing the utterance 902, and there is no utterance. The relative time "9.5 seconds" obtained by subtracting the period "3 seconds" of 901 is used as the time information for reproducing the utterance 902. As a result, the conference video output unit 409 of the conference terminal 110b can output the conference video, for example, excluding the period of no utterance 901, as shown in FIG.

<Processing flow>
(Processing of conference system)
FIG. 11 is a sequence diagram showing an example of the conference system processing according to the first embodiment. This process is performed when the conference terminal 110a transmits the shared image 501 while the conference is being held between the conference terminal (first terminal device) 110a and the conference terminal (second terminal device) 110b. , An example of the process executed by the conference system 100 is shown.

In step S1101, the conference video receiving unit 407 of the conference terminal 110b receives, for example, data of the conference video transmitted from the conference terminal 110a via the conference server 150, as shown in steps S701 and 702 of FIG. Shall start.

In steps S1102 and S1103, the conference video receiving unit 407 of the conference terminal 110b stores the received conference video data in the storage unit 408.

In steps S1104 and S1105, the detection unit 801 of the conference terminal 110b acquires the data of the conference video stored in the storage unit 408 and starts the detection of the period when there is no utterance.

In step S1106, the shared image receiving unit 405 of the conference terminal 110b receives the data of the shared image transmitted from the conference terminal 110a via the conference server 150, for example, as shown in steps S705 and S706 of FIG. Shall start.

In steps S1107 and S1108, the shared image display unit 406 of the conference terminal 110b performs a display process of displaying the shared image using the data of the shared image received by the shared image receiving unit 405. It should be noted that the reception of the data of the shared image requires, for example, a time corresponding to the delay time Td in FIG.

In step S1109, when the shared image receiving unit 405 of the conference terminal 110b completes the reception of the shared image data, the conference video output unit 409 is notified of the reception completion notification indicating that the reception of the shared image data is completed. ..

In steps S1110 and S1111, when the conference video output unit 409 of the conference terminal 110b receives the reception completion notification, it reads out the conference video data stored in the storage unit 408 in the same manner as in the first embodiment, and the conference video is read. Starts playing.

In this state, for example, in step S1112, when the detection unit 801 detects a period without utterance, the detection unit 802 notifies the output control unit 802 of a detection notification indicating that the period without utterance is detected.

In step S1114, the conference video output unit 409 of the conference terminal 110b causes the conference video output unit 409 to reproduce the conference video excluding the period when there is no utterance, for example, as described with reference to FIG.

As a result, the conference terminal 110b starts outputting the shared image 501 and the utterance 502 at time t2, for example, as shown in FIG. 9, and the conference video excluding a part or all of the period of no utterance 901. Will be able to be output.

Therefore, by the above processing, the conference terminal 110 starts to output the shared image 501 and the utterance 502 at time t2, for example, as shown in FIG. 9, and part or all of the period of no utterance 901 is started. It will be possible to output the excluded conference video.

The process shown in FIG. 11 is an example of a process of suppressing an increase in the delay time of the conference video while reducing the time difference between the shared image and the conference video. For example, the conference terminal 110b may be one that suppresses an increase in the delay time of the conference video while reducing the time difference between the shared image and the conference video by executing the process shown in FIG.

(Processing of conference terminal)
FIG. 12 is a diagram showing another example of processing when the shared image according to the second embodiment is received. For example, the conference terminal 110 having a conference with another conference terminal repeatedly executes the process shown in FIG. The process shown in FIG. 12 is an example of a control method for the conference terminal 110.

In step S1201, the shared image receiving unit 405 of the conference terminal 110 determines whether or not the reception of the shared image data has started from another conference terminal having a conference.

When the reception of the shared image data is started, the conference terminal 110 executes the processes after step S1202. On the other hand, when the reception of the shared image data has not started, the conference terminal 110 ends the process shown in FIG.

When the process proceeds to step S1202, the conference video output unit 409 of the conference terminal 110 temporarily stops the reproduction of the conference video.

In step S1203, the conference video output unit 409 determines whether or not the reception of the shared image data is completed, and when the reception of the shared image data is completed, the conference video reproduction is restarted in step S1204.

As a result, when the shared image receiving unit 405 receives the shared image data, the conference video output unit 409 delays the conference voice and the moving image for a predetermined period based on the conference video data stored in the storage unit 408. It can be output.

In step S1205, when the detection unit 801 of the conference terminal 110 detects a period during which there is no utterance, the process of step S1206 is executed.

In step S1206, the output control unit 802 of the conference terminal 110 causes the conference video output unit 409 to reproduce the conference video excluding the period when there is no utterance, for example, as described with reference to FIG.

By the above processing, for example, as shown in FIG. 9, the conference terminal 110 starts to output the shared image 501 and the utterance 502 at time t2, and excludes a part or all of the period of no utterance 901. Conference video can be output.

As described above, according to each process of the second embodiment, in the conference system 100 for sharing the shared image and transmitting / receiving the conference video, the delay of the conference video is reduced while reducing the time difference between the shared image and the conference video. It becomes possible to suppress the increase in time.

The conference video receiving unit 407 shown in FIGS. 4 and 8 is an example of a video data receiving unit that receives video data including conference audio or an audio data receiving unit that receives audio data. Further, the conference video transmission unit 403 is an example of a video data transmission unit that transmits video data including conference audio, or an audio data transmission unit that transmits audio data. As described above, the video data may be, for example, audio data of the conference that does not include the video data of the conference.

Similarly, the conference video output unit 409 is an example of an output unit that outputs the audio data stored in the storage unit 408 later than the time when the audio data is received by the conference video reception unit 407. As described above, the output unit can output the moving image and the sound of the conference with a delay of a predetermined period.

100 conference system 100
110 Conference terminal (terminal device)
110a conference terminal (first terminal device)
110b conference terminal (second terminal device)
403 Conference video transmission unit (audio data transmission unit, video data transmission unit)
404 Shared image transmitter 405 Shared image receiver 406 Shared image display 407 Conference video receiver (audio data receiver, video data receiver)
408 Storage unit (audio data storage unit, video data storage unit)
409 Conference video output section (output section)
801 Detection unit 802 Output control unit

Japanese Unexamined Patent Publication No. 07-202887

Claims (10)

A terminal device that communicates with other terminal devices via a network.
A shared image receiving unit that receives data of a shared image shared with the other terminal device, and
An audio data receiving unit that receives audio data transmitted by the other terminal device, and
A shared image display unit that displays the shared image on the display unit,
When the shared image receiving unit receives the data of the shared image, the audio data storage unit that stores the audio data and the audio data storage unit
An output unit that outputs the voice data stored in the voice data storage unit later than the time when the voice data is received by the voice data receiving unit, and an output unit.
A detection unit that detects a period during which there is no utterance from the voice data stored in the voice data storage unit, and a detection unit.
When the detection unit detects a period in which the voice data stored in the voice data storage unit has no utterance, one of the periods in which the voice data stored in the voice data storage unit has no utterance. An output control unit that outputs the audio data to the output unit by excluding the unit or all of the units.
Has a terminal device. When the shared image display unit displays the shared image on the display unit and then a period in which no utterance is detected is detected in the voice data, the output control unit is one of the periods in which the voice data has no utterance. The terminal device according to claim 1, which excludes parts or all. The output unit outputs the voice data stored in the voice data storage unit with a delay of a predetermined period.
The terminal device according to claim 1 or 2, wherein the output control unit excludes a period during which there is no utterance from the voice data within the range of the predetermined period. The terminal device according to claim 3, wherein the predetermined period includes a period from the start of receiving the data of the shared image to the display of the shared image. The audio data is included in video data including moving images taken by the other terminal.
The terminal device according to claim 3 or 4, wherein the output unit outputs the moving image with a delay of the predetermined period using the audio data. The moving image includes time information for reproducing the moving image.
The terminal device according to claim 5, wherein the output unit outputs the time information of the moving image with a delay of the predetermined period when the shared image receiving unit receives the data of the shared image. When the shared image display unit displays the shared image on the display unit and then a period in which there is no utterance is detected in the voice data, the output control unit uses the time information of the moving image to utter the utterance. The terminal device according to claim 6, wherein the moving image is output to the output unit by advancing only a part or all of the period in which there is no information. A conference system that communicates using a first terminal device and a second terminal device that can communicate with the first terminal device via a network.
The first terminal device is
A shared image transmission unit that transmits data of a shared image shared with the second terminal device to the second terminal device, and a shared image transmission unit.
A voice data transmission unit that transmits voice data acquired by the first terminal device to the second terminal device, and a voice data transmission unit.
Have,
The second terminal device is
A shared image receiving unit that receives the data of the shared image transmitted by the first terminal device, and
A voice data receiving unit that receives the voice data transmitted by the first terminal device, and a voice data receiving unit.
A shared image display unit that displays the shared image on the display unit,
When the shared image receiving unit receives the data of the shared image, the audio data storage unit that stores the audio data and the audio data storage unit
An output unit that outputs the voice data stored in the voice data storage unit later than the time when the voice data is received by the voice data receiving unit, and an output unit.
A detection unit that detects a period during which there is no utterance from the voice data stored in the voice data storage unit, and a detection unit.
When the detection unit detects a period in which the voice data stored in the voice data storage unit has no utterance, one of the periods in which the voice data stored in the voice data storage unit has no utterance. An output control unit that outputs the audio data to the output unit by excluding the unit or all of the units.
Has a conference system. It is a control method of a terminal device that communicates with other terminal devices via a network.
The terminal device
The step of receiving the data of the shared image to be shared with the other terminal device,
The step of receiving the voice data transmitted by the other terminal device, and
The step of displaying the shared image on the display unit and
When the data of the shared image is received, the step of storing the audio data in the storage unit and
A step of outputting the voice data stored in the storage unit later than the time of receiving the voice data, and a step of outputting the voice data.
A step of detecting a period during which there is no utterance from the voice data stored in the storage unit, and
When a period during which there is no utterance is detected in the voice data stored in the storage unit, a part or all of the period during which there is no utterance is excluded from the voice data stored in the storage unit. Steps to output audio data and
How to control the terminal device. It is a program to be executed by the computer of the terminal device.
To the computer
The step of receiving shared image data to be shared with other terminal devices that communicate over the network,
The step of receiving the voice data transmitted by the other terminal device, and
The step of displaying the shared image on the display unit and
When the data of the shared image is received, the step of storing the audio data in the storage unit and
A step of outputting the voice data stored in the storage unit later than the time of receiving the voice data, and a step of outputting the voice data.
A step of detecting a period during which there is no utterance from the voice data stored in the storage unit, and
When a period during which there is no utterance is detected in the voice data stored in the storage unit, a part or all of the period during which there is no utterance is excluded from the voice data stored in the storage unit. Steps to output audio data and
A program that runs.

Images