JP7625901B2 - CONTROL DEVICE, CONTROL SYSTEM AND METHOD - Google Patents

Description

The present invention relates to a control device, a terminal device, a movable device, a control system and a method.

In recent years, with the spread of broadband and cloud environments, and the development of communication tools accompanying advances in information and communications technology, remote conferences, which allow remote participants to share video and audio, are becoming more common.

For example, Patent Document 1 is known as a document disclosing a system for realizing remote conferencing. Specifically, the system disclosed in Patent Document 1 comprises a recording means for recording data regarding speech by conference participants for each participant, and a control means for displaying the data recorded by the recording means at each location on the display device of the conference facilitator, a specific participant, or all participants. The speech data includes the number of opportunities for each participant to speak and the length of time each participant spent speaking. By viewing the display of the speech data, conference participants can recognize the presence of participants who rarely speak and encourage them to speak.

JP 2007-325063 A

However, even if the number of speaking opportunities and speaking time for each participant are displayed, it is difficult for meeting participants to immediately understand the presence of participants who hardly speak, and this does not necessarily lead to the appropriate progress of the meeting.

The present invention has been made in consideration of the above problems, and the object of the present invention is to provide new and improved control devices, terminal devices, movable devices, control systems and methods that can support the appropriate progress of communication.

In order to solve the above problem, according to one aspect of the present invention, there is provided a control device comprising: a data communication unit that receives utterance data for communication between users from each terminal device among a plurality of terminal devices; a tendency analysis unit that analyzes trends in communication between the users based on the utterance data received by the data communication unit; a transmission unit that transmits a guidance message regarding the progress of communication between the users to at least one terminal device among the plurality of terminal devices based on a predetermined analysis result obtained by the tendency analysis unit; and an identification unit that identifies the at least one terminal device to which the guidance message is to be sent based on the predetermined analysis result obtained by the tendency analysis unit, wherein the predetermined analysis result is an analysis result indicating the presence of a user whose bias in the amount of utterances is equal to or greater than a reference value, and the identification unit identifies a terminal device used by a user with the second largest amount of utterances among the plurality of terminal devices as the at least one terminal device .

When an analysis result is obtained indicating that there is a user whose bias toward a larger volume of comments is equal to or greater than the reference value, the transmission unit may transmit, as the prompting message, a message that prompts comments from users other than the user in question.

When an analysis result is obtained indicating that there is a user whose bias toward low volume of comments is equal to or greater than the reference value, the transmission unit may transmit, as the prompting message, a message that prompts the user to make a comment.

The utterance data may be voice data, and the control device may further include a conversion unit that converts the voice data received by the data communication unit into text data, and a text transmission unit that transmits the text data obtained by the conversion unit to the multiple terminal devices.

In order to solve the above-mentioned problems, according to another aspect of the present invention, there is provided a control system having a plurality of terminal devices and a control device communicating with the plurality of terminal devices, the control device including a data communication unit that receives utterance data for communication between users from each of the plurality of terminal devices and transmits text data indicating the contents of utterances made by the users of each of the terminal devices to the plurality of terminal devices, a tendency analysis unit that analyzes the tendency of communication between the users based on the utterance data received by the data communication unit, and a guidance message regarding the progress of communication between the users to at least one of the plurality of terminal devices based on a predetermined analysis result obtained by the tendency analysis unit. a transmitting unit that transmits a message to the user and an identifying unit that identifies the at least one terminal device to which the guidance message is to be sent based on a predetermined analysis result obtained by the tendency analysis unit, the predetermined analysis result being an analysis result indicating the presence of a user whose degree of bias in the amount of speech is equal to or greater than a reference value, the identifying unit identifying a terminal device used by a user with the second highest amount of speech among the plurality of terminal devices as the at least one terminal device, the terminal device having a receiving unit that receives the text data and the guidance message from the control device, and a display control unit that generates a display screen on which the text data and the guidance message are arranged side by side.

In addition, in order to solve the above problem, according to another aspect of the present invention, a method executed by a control device is provided, including: receiving utterance data for communication between users from each terminal device among a plurality of terminal devices; analyzing trends in communication between the users based on the received utterance data; sending a guidance message regarding the progress of communication between the users to at least one terminal device among the plurality of terminal devices based on the result of a predetermined analysis; and identifying a terminal device used by a user with the second-highest volume of utterances as the at least one terminal device to which the guidance message is to be sent based on the result of the predetermined analysis, wherein the predetermined analysis result is an analysis result indicating the presence of a user whose bias in volume of utterances is equal to or greater than a reference value .

The present invention as described above makes it possible to support the appropriate progress of communication.

1 is an explanatory diagram showing a configuration of a remote conference system according to an embodiment of the present invention; FIG. 11 is an explanatory diagram showing a specific example of a conference screen. FIG. 2 is an explanatory diagram showing the configuration of a control server 30 according to an embodiment of the present invention. 10 is an explanatory diagram showing a specific example of the correspondence between each terminal device 20 and participant names. FIG. 11 is an explanatory diagram showing a specific example of a conversation message supplied from a text conversion unit 320 to a chat control unit 330. FIG. FIG. 2 is an explanatory diagram showing a configuration of a terminal device 20 according to an embodiment of the present invention. FIG. 13 is an explanatory diagram showing a specific example of a conference screen including a guidance message. FIG. 2 is an explanatory diagram showing the operation of the control server 30 according to one embodiment of the present invention. FIG. 2 is an explanatory diagram showing the operation of a terminal device 20 according to an embodiment of the present invention. FIG. 13 is an explanatory diagram showing a configuration according to a modified example. FIG. 2 is a block diagram showing the hardware configuration of a terminal device 20.

The following describes in detail an embodiment of the present invention with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

In addition, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different letters after the same reference numeral. For example, multiple components having substantially the same functional configuration or logical meaning may be distinguished as necessary, such as terminal devices 20A, 20B, and 20C. However, if there is no need to particularly distinguish between multiple components having substantially the same functional configuration, only the same reference numeral is assigned to each of the multiple components. For example, if there is no need to particularly distinguish between terminal devices 20A, 20B, and 20C, each terminal device will be simply referred to as terminal device 20.

<Outline of remote conferencing system>
An embodiment of the present invention relates to a control system, and more particularly to a remote conference system for controlling a remote conference using a plurality of remote terminal devices. Hereinafter, an overview of the remote conference system according to an embodiment of the present invention will be described with reference to FIG.

Figure 1 is an explanatory diagram showing the configuration of a remote conference system according to one embodiment of the present invention. As shown in Figure 1, the remote conference system according to one embodiment of the present invention has multiple terminal devices 20A to 20E and a control server 30.

These terminal devices 20A-20E and the control server 30 are connected via a network 12. The network 12 is a wired or wireless transmission path for information transmitted from devices connected to the network 12. For example, the network 12 may include public line networks such as the Internet, telephone line networks, and satellite communication networks, as well as various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks). The network 12 may also include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

In the example shown in FIG. 1, participant A uses terminal device 20A, participant B uses terminal device 20B, participant C uses terminal device 20C, participant D uses terminal device 20D, and participant E uses terminal device 20E. However, the number of terminal devices 20 constituting the remote conference system and the number of participants (users) using the remote conference system may be fewer or greater.

(Terminal device 20)
The terminal device 20 transmits utterance data for a remote conference, which is communication between participants, and video data of a participant using the terminal device 20 to the control server 30. The utterance data may be voice data indicating a voice uttered by a participant using the terminal device 20, or may be text data obtained by converting a voice uttered by a participant using the terminal device 20 into text. The following mainly describes an example in which the terminal device 20 transmits voice data as utterance data.

The terminal device 20 also receives audio data indicating the voices made by the other participants and video data of the other participants from the control server 30. The terminal device 20 outputs the audio data received from the control server 30 as audio and displays the video data. This makes it possible to hold a remote conference between multiple participants via video and audio.

The terminal device 20 also receives a conversation message from the control server 30, the conversation message including text data obtained by converting the voice data of each participant into text. The terminal device 20 displays the conversation message in addition to the video data of each participant described above. Here, a specific example of a conference screen displayed by the terminal device 20 will be described with reference to FIG. 2.

Figure 2 is an explanatory diagram showing a specific example of a conference screen. As shown in Figure 2, the conference screen includes a chat display area 50 and a video display area 60. The chat display area 50 displays a plurality of conversation messages, including text data indicating the speaker's name and the content of the comment. The video display area 60 displays the video data and participant name of each participant.

Note that while FIG. 1 shows a notebook PC (Personal Computer) as an example of the terminal device 20, the terminal device 20 may be other information processing devices such as a desktop PC, a smartphone, a mobile phone, or a PHS (Personal Handyphone System).

(Control Server 30)
The control server 30 is a control device for realizing a remote conference using a plurality of terminal devices 20. The control server 30 receives audio data and video data from each terminal device 20, and transmits the audio data and video data to each terminal device 20. The control server 30 also converts the audio data received from each terminal device 20 into text to obtain text data, and transmits a conversation message including the text data to each terminal device 20.

Furthermore, the control server 30 according to one embodiment of the present invention transmits guidance messages regarding the progress of the remote conference according to the communication trends between the participants, as will be described in detail below. This makes it possible to support the appropriate progress of the remote conference.

Above, we have explained the overview of the remote conference system according to one embodiment of the present invention. Next, we will explain in detail the configuration and operation of the remote conference system according to one embodiment of the present invention.

<Configuration of control server 30>
3 is an explanatory diagram showing a configuration of a control server 30 according to an embodiment of the present invention. As shown in FIG. 3, the control server 30 according to an embodiment of the present invention includes a conference management unit 310, a text conversion unit 320, a chat control unit 330, a conversation recording unit 340, a control unit 350, and a conversation guidance unit 360.

(Conference management unit 310)
The conference management unit 310 functions as a data communication unit that communicates video data and audio data with each terminal device 20. For example, the conference management unit 310 receives audio data and video data from each terminal device 20 and transmits the audio data and video data to each terminal device 20. The conference management unit 310 also supplies the audio data received from each terminal device 20 to the text conversion unit 320.

(Text conversion unit 320)
The text conversion unit 320 recognizes the voice data supplied from the conference management unit 310 and converts the voice data into text data. The text conversion unit 320 also supplies a conversation message including the speaker's name and the text data to the chat control unit 330 and the conversation recording unit 340. Regarding the speaker's name, the control server 30 receives information indicating the participant's name of each terminal device 20 from each terminal device 20 in advance, and stores the correspondence between each terminal device 20 and the participant's name.

Figure 4 is an explanatory diagram showing a specific example of the correspondence between each terminal device 20 and the participant name. In the example shown in Figure 4, the participant name "Mr. A" is associated with terminal device 20A, and the participant name "Mr. B" is associated with terminal device 20B. Note that in practice, an ID for identifying each terminal device 20 may be associated with a participant name.

The text conversion unit 320 can identify the name of the speaker based on the correspondence and the information indicating from which terminal device 20 the text data provided by the conference management unit 310 was obtained from the voice data.

Figure 5 is an explanatory diagram showing a specific example of a conversation message supplied from the text conversion unit 320 to the chat control unit 330. In the conversation message shown in Figure 5, the speaker's name is "Mr. A" and the message body (text data) is "Please let me know your opinions on what kind of exhibition would be possible."

(Chat control unit 330)
The chat control unit 330 functions as a text sending unit that sends the conversation message supplied from the text conversion unit 320 to each terminal device 20. The chat control unit 330 also functions as a sending unit that sends a prompting message, which is supplied from the conversation prompting unit 360 and will be described later.

(Conversation Recording Unit 340)
The conversation recording unit 340 records the conversation message provided by the text conversion unit 320 .

(Control unit 350)
The control unit 350 controls the overall operation of the control server 30. For example, the control unit 350 may hold the above-mentioned correspondence between each of the terminal devices 20 and the participant names.

(Conversation Guiding Unit 360)
Conversation guiding unit 360 generates a guidance message regarding the progress of the remote conference as necessary. The functions of conversation guiding unit 360 are divided into a tendency analysis unit 362, a guidance partner specification unit 364, and a guidance message generation unit 366, as shown in Fig. 3. The functions of each of tendency analysis unit 362, guidance partner specification unit 364, and guidance message generation unit 366 will be described below.

The trend analysis unit 362 analyzes the trend of the remote conference, i.e., the trend of communication between the participants. For example, the trend analysis unit 362 refers to the conversation messages recorded by the conversation recording unit 340, and calculates the amount of speech of each participant each time a new conversation message is recorded. The amount of speech may be the number of conversation messages, the total length of the message bodies in the conversation messages, or the total number of words included in the message bodies in the conversation messages.

Then, when the number of recorded conversation messages exceeds a predetermined number, the tendency analysis unit 362 starts analyzing the speech volume of each participant. Specifically, the tendency analysis unit 362 calculates the standard deviation of the speech volume of each participant and the deviation of each participant, and further calculates the deviation value of the speech volume of each participant, "((speech volume-deviation)/standard deviation)) x 10 + 50".

The tendency analysis unit 362 determines whether or not there is a participant whose bias in speech volume is equal to or greater than a reference value based on the calculated deviation value of the speech volume of each participant. For example, the tendency analysis unit 362 may determine whether or not there is a participant whose bias toward high speech volume is equal to or greater than a reference value and whose deviation value is "60" or greater. Similarly, the tendency analysis unit 362 may determine whether or not there is a participant whose bias toward low speech volume is equal to or greater than a reference value and whose deviation value is "40" or less. If such a participant is present, the guidance partner identification unit 364 and the guidance message generation unit 366 perform processing for sending a guidance message.

However, when a prompting message is sent, the tendency analysis unit 362 does not need to make the above judgment until a predetermined time has elapsed since the prompting message was sent, or until a predetermined number of conversation messages have been received since the prompting message was sent. In this case, it is possible to avoid sending prompting messages frequently.

When the tendency analysis unit 362 obtains an analysis result indicating that there is a participant whose bias in the amount of speech is equal to or greater than a reference value as a predetermined analysis result, the guidance partner identification unit 364 identifies at least one terminal device 20 to which the guidance message is to be sent. For example, the guidance partner identification unit 364 may identify a terminal device 20 to which the guidance message is to be sent from among terminal devices 20 other than the terminal device 20 used by the participant with the largest amount of speech. As a more specific example, the guidance partner identification unit 364 may identify the terminal device 20 used by the participant with the second largest amount of speech as the terminal device 20 to which the guidance message is to be sent.

The prompting message generating unit 366 generates a prompting message when the tendency analyzing unit 362 obtains an analysis result indicating the presence of a participant whose bias in the amount of speech is equal to or greater than a reference value as a predetermined analysis result. For example, when a participant whose bias toward less speech is equal to or greater than a reference value is present, the prompting message generating unit 366 generates a prompting message that prompts the participant to make a comment. When the participant whose bias toward less speech is equal to or greater than the reference value is E, the prompting message generating unit 366 generates a prompting message including the message text "Why don't you ask E for his opinion?".

On the other hand, if there is a participant whose bias toward more speech volume is equal to or greater than the reference value, the prompting message generating unit 366 generates a prompting message that prompts speech from participants other than the participant in question. For example, if the participant whose bias toward less speech volume is equal to or greater than the reference value is person B, the prompting message generating unit 366 generates a prompting message that includes the message text "Why not listen to comments by people other than person B as well?".

Furthermore, the guidance message generating unit 366 generates a guidance message by adding the name "AI" to the beginning of the message body so that the guidance message can be identified as a guidance message from the control server 30. As a result, the guidance message can be, for example, a message such as "AI: Why don't you listen to the opinion of Mr. E?" or "AI: Why don't you listen to comments other than those of Mr. B?".

The guidance message generating unit 366 supplies the generated guidance message to the chat control unit 330. The guidance message generating unit 366 transmits the guidance message supplied from the guidance message generating unit 366 to the terminal device 20 identified by the guidance partner identifying unit 364. As a result, the guidance message is displayed on the terminal device 20 identified by the guidance partner identifying unit 364.

<Configuration of terminal device 20>
The configuration of the control server 30 according to an embodiment of the present invention has been described above. Next, the configuration of the terminal device 20 according to an embodiment of the present invention will be described with reference to FIG.

Figure 6 is an explanatory diagram showing the configuration of a terminal device 20 according to one embodiment of the present invention. As shown in Figure 6, the terminal device 20 according to one embodiment of the present invention has an imaging unit 220, an audio input unit 230, a communication unit 240, a display unit 250, an audio output unit 260, and a control unit 270.

(Imaging unit 220)
The imaging unit 220 captures video of the participants using the terminal devices 20 and acquires video data of the participants. The imaging unit 220 provides the video data of the participants to the communication unit 240.

(Voice input unit 230)
The voice input unit 230 converts voices uttered by participants using the terminal devices 20 into electrical voice data. The voice input unit 230 supplies the voice data of the participants to the communication unit 240.

(Communication unit 240)
The communication unit 240 functions as a transmitting unit that transmits various data to the control server 30 and as a receiving unit that receives various data from the control server 30. For example, the communication unit 240 transmits video data supplied from the imaging unit 220 and audio data supplied from the audio input unit 230 to the control server 30. The communication unit 240 also receives video data and audio data of each participant from the control server 30. Furthermore, the communication unit 240 receives conversation messages and guidance messages from the control server 30.

(Display unit 250)
The display unit 250 displays a conference screen including the video data and conversation messages received from the control server 30 by the communication unit 240 .

(Audio output unit 260)
The audio output unit 260 outputs the audio data of each participant received from the control server 30 by the communication unit 240 .

(Control unit 270)
The control unit 270 controls the overall operation of the terminal device 20. For example, the control unit 270 generates a conference screen including the video data and conversation messages received by the communication unit 240, and displays the conference screen on the display unit 250. When the communication unit 240 receives a guidance message from the control server 30, the guidance message is also included in the conference screen.

Figure 7 is an explanatory diagram showing a specific example of a conference screen including a prompting message. As shown in Figure 7, the prompting message 52 is placed alongside other conversation messages in the chat display area 50.

The guidance message 52 is displayed with more emphasis than the other conversation messages. In the example shown in FIG. 7, the emphasis is achieved by underlining, but the emphasis may be achieved by using a color different from that of the other conversation messages for displaying the guidance message 52, or by using a font different from that of the other conversation messages for displaying the guidance message 52, or the like. Alternatively, the guidance message 52 may be distinguished from the other conversation messages by being given the name of a person other than the participant (e.g., AI) as the speaker name. Furthermore, the guidance message 52 may be displayed alongside an avatar image of the AI.

The control unit 270 may also store the participant name entered by the participant into the terminal device 20, for example, when the terminal device 20 is started, and cause the communication unit 240 to transmit the participant name to the control server 30 when the remote conference begins.

<Operation>
The configurations of the control server 30 and the terminal device 20 according to one embodiment of the present invention have been described above. Next, the operations of the control server 30 and the terminal device 20 according to one embodiment of the present invention will be summarized with reference to Figs. 8 and 9.

(Operation of control server 30)
8 is an explanatory diagram showing the operation of the control server 30 according to one embodiment of the present invention. As shown in FIG. 8, when the control server 30 receives voice data and video data from each terminal device 20, the control server 30 transmits the voice data and video data to each terminal device 20 (S404). In addition, the text conversion unit 320 of the control server 30 converts the voice data received from each terminal device 20 into text data (S408), and the chat control unit 330 transmits a conversation message including the text data to each terminal device 20 (S412).

When the number of conversation messages exceeds a predetermined number, the tendency analysis unit 362 analyzes the tendency of communication between the participants (S416). If the analysis results in a participant who speaks a lot or a participant who speaks a little (S420/Yes), the guidance message generation unit 366 generates a guidance message regarding the progress of the remote conference (S424). In addition, the guidance partner identification unit 364 identifies at least one terminal device 20 to which the guidance message will be sent (S428). For example, the guidance partner identification unit 364 may identify the terminal device 20 used by the participant who speaks the second most as the terminal device 20 to which the guidance message will be sent.

Then, the chat control unit 330 transmits the prompting message generated by the prompting message generation unit 366 to the terminal device 20 identified by the prompting partner identification unit 364 (S432).

After the prompting message is sent, or if the analysis results obtained by the trend analysis unit 362 show that there are no participants who speak a lot or who speak little (S420/No), the process from S404 is repeated until the remote conference ends (S436).

(Operation of Terminal Device 20)
Next, the operation of the terminal device 20 will be described with reference to FIG.

Figure 9 is an explanatory diagram showing the operation of the terminal device 20 according to one embodiment of the present invention. As shown in Figure 9, the terminal device 20 transmits the voice data of the participants obtained by the voice input unit 230 and the video data of the participants obtained by the imaging unit 220 to the control server 30 (S504). In addition, the terminal device 20 receives the voice data, video data and conversation messages of each participant from the control server 30 (S508).

The terminal device 20 outputs the audio data of each participant received from the control server 30 from the audio output unit 260, and displays a conference screen including the video data and conversation messages received from the control server 30 on the display unit 250 (S512).

After that, when the terminal device 20 receives a prompting message from the control server 30 (S516/Yes), the terminal device 20 displays the prompting message in the chat display area on the conference screen (S520). After that, the process from S504 is repeated until the remote conference ends (S524).

<Action and effect>
According to the embodiment of the present invention described above, various operational effects can be obtained. For example, according to the embodiment of the present invention, since the guidance message regarding the progress of the remote conference is displayed in the same chat display area as the conversation message, the participants can easily notice the guidance message. Therefore, it is expected that the participants will encourage other participants to speak at an appropriate time.

In one embodiment of the present invention, a prompting message is sent to the participant who speaks the second most. The participant who speaks the second most is likely not the organizer or moderator of the conference, and having such a participant facilitate the conference may lead to many participants actively participating in the conference.

In one embodiment of the present invention, a prompting message is generated to prompt participants who speak less to speak, or to prompt participants other than those who speak more to speak. Therefore, participants who see the prompting message can easily proceed with the conference in a way that smooths out speaking.

<Modification>
An embodiment of the present invention has been described above. In the following, some modified examples of the above-described embodiment will be described. Note that each modified example described below may be applied alone to the above-described embodiment, or may be applied in combination to the above-described embodiment. Also, each modified example may be applied in place of the configuration of the above-described embodiment, or may be applied in addition to the configuration of the above-described embodiment.

(First Modification)
For example, an embodiment of the present invention may be combined with a moving device such as a robot.

Figure 10 is an explanatory diagram showing a configuration according to a modified example. As shown in Figure 10, in the modified example, a movable device 70 is connected to a terminal device 20. The terminal device 20 and the movable device 70 may be connected by wire or wirelessly.

The movable device 70 has the functions of a receiver that receives a control signal from the terminal device 20, a movable unit, and a control unit that moves the movable unit based on the control signal received by the receiver. For example, the movable device 70 receives a control signal transmitted from the terminal device 20 in response to reception of a guidance message at the terminal device 20. Then, based on the reception of the control signal, the movable device 70 moves the head of the movable device 70 up and down or left and right, for example by driving a joint that connects the body of the movable device 70 to the head, which is an example of a movable unit. Furthermore, if the control signal includes a guidance message, the movable device 70 may output a sound that reads out the guidance message.

According to this modified example, the participants are notified of the receipt of the guidance message by the movement of the movable part of the movable device 70. Therefore, the participants are more likely to notice the guidance message.

In the above, an example has been described in which the movable device 70 receives a control signal via the terminal device 20, but the movable device 70 may receive a control signal directly from the control server 30. This operation can be realized by storing in advance the correspondence between the terminal device 20 and the movable device 70 in the control server 30.

(Second Modification)
The above describes an example in which the guidance recipient identification unit 364 identifies the terminal device 20 to which the guidance message is to be sent based on the relationship between the volume of speech of each participant, but the guidance recipient identification unit 364 may also identify the terminal device 20 to which the guidance message is to be sent by other methods.

Specifically, the guidance partner identification unit 364 may estimate the emotion of each participant from at least one of the audio data and video data of each participant, and identify the terminal device 20 to which the guidance message is to be sent based on the emotion estimation result. For example, the guidance partner identification unit 364 may identify the terminal device 20 to which the guidance message is to be sent from the terminal device 20 used by a participant who is not in an excited state.

The emotion estimation result can also be used to determine whether or not to send a prompting message. For example, the tendency analysis unit 362 may determine that a prompting message should be sent when a confused participant is present, and the prompting message generation unit 366 may generate a prompting message that encourages the confused participant to speak.

Using such emotion estimation results can also help ensure that remote conferences proceed smoothly.

(Third Modification)
There are various other methods for identifying the terminal device 20 to which the guidance message is to be sent. For example, the guidance partner identification unit 364 may randomly identify the terminal device 20 to which the guidance message is to be sent, or may identify a different terminal device 20 in descending order of the amount of speech each time a guidance message is sent. Alternatively, the terminal device 20 to which the guidance message is to be sent may be fixed. For example, it is possible to fix the terminal device 20 used by a participant who will be the organizer of the conference or a leader in an organization as the terminal device 20 to which the guidance message is to be sent.

(Fourth Modification)
Although an example has been described above in which the voice data and video data of each participant are transmitted and received via the control server 30, the voice data and video data of each participant may be transmitted and received directly between the terminal devices 20. However, in order to analyze communication trends in the control server 30, it is desirable to also transmit the voice data of each participant to the control server 30.

<Hardware Configuration>
An embodiment of the present invention has been described above. The above-mentioned information processing such as analysis of communication trends, transmission of prompting messages, and display of a conference screen is realized by cooperation between software and the hardware of the terminal device 20 and the control server 30. An example of the hardware configuration of the terminal device 20 will be described below. Note that the hardware configuration of the terminal device 20 can also be applied to the control server 30.

Figure 11 is a block diagram showing the hardware configuration of the terminal device 20. The terminal device 20 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and a host bus 204. The terminal device 20 also includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, and a network interface 215.

The CPU 201 functions as an arithmetic processing device and control device, and controls the overall operation of the terminal device 20 in accordance with various programs. The CPU 201 may also be a microprocessor. The ROM 202 stores programs and arithmetic parameters used by the CPU 201. The RAM 203 temporarily stores programs used in the execution of the CPU 201 and parameters that change appropriately during the execution. These are interconnected by a host bus 204 consisting of a CPU bus and the like. The functions of the control unit 270 can be realized by the cooperation of the CPU 201, ROM 202, RAM 203, and software.

The host bus 204 is connected to an external bus 206, such as a PCI (Peripheral Component Interconnect/Interface) bus, via a bridge 205. Note that the host bus 204, bridge 205, and external bus 206 do not necessarily need to be configured separately, and these functions may be implemented on a single bus.

The input device 208 is composed of input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, sensor, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 201. The user of the terminal device 20 can input various data and instruct processing operations to the terminal device 20 by operating the input device 208.

The display device 209 includes display devices such as a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, and a lamp. The audio output device 210 includes audio output devices such as a speaker and headphones.

The storage device 211 is a data storage device configured as an example of a storage unit of the terminal device 20 according to this embodiment. The storage device 211 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, and a deleting device that deletes data recorded on the storage medium. The storage device 211 is configured, for example, by a HDD (Hard Disk Drive) or an SSD (Solid Storage Drive), or a memory having equivalent functions. This storage device 211 drives the storage and stores the programs executed by the CPU 201 and various data.

The drive 212 is a reader/writer for storage media, and is built into the terminal device 20 or attached externally. The drive 212 reads information recorded on a removable storage medium 24, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203 or the storage device 211. The drive 212 can also write information to the removable storage medium 24.

The network interface 215 is, for example, a communication interface configured with a communication device for connecting to the network 12. The network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wired communication device that performs wired communication.

<Additional Information>
Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

For example, each step in the processing of the terminal device 20 and the control server 30 in this specification does not necessarily have to be processed chronologically in the order described in the sequence diagram or flowchart. For example, each step in the processing of the terminal device 20 and the control server 30 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

It is also possible to create a computer program for causing the hardware, such as the CPU, ROM, and RAM, built into the terminal device 20 and the control server 30 to perform functions equivalent to those of the respective components of the terminal device 20 and the control server 30 described above. A storage medium on which the computer program is stored is also provided.

20 Terminal device 220 Imaging unit 230 Audio input unit 240 Communication unit 250 Display unit 260 Audio output unit 270 Control unit 30 Control server 310 Conference management unit 320 Text conversion unit 330 Chat control unit 340 Conversation recording unit 350 Control unit 360 Conversation guidance unit 362 Tendency analysis unit 364 Guidance partner identification unit 366 Guidance message generation unit

Claims (6)

a data communication unit that receives utterance data for communication between users from each of the plurality of terminal devices;
a trend analysis unit that analyzes a trend of communication between the users based on the utterance data received by the data communication unit;
a transmission unit that transmits a guidance message regarding a progress of communication between the users to at least one terminal device among the plurality of terminal devices based on a predetermined analysis result obtained by the tendency analysis unit;
a determination unit that determines the at least one terminal device to which the prompting message is to be sent based on a predetermined analysis result obtained by the tendency analysis unit;
Equipped with
the predetermined analysis result is an analysis result indicating the presence of a user whose degree of bias in message volume is equal to or greater than a reference value;
The identification unit is configured to identify, from among the plurality of terminal devices, a terminal device used by a user with the second largest amount of messages as the at least one terminal device . The control device according to claim 1, wherein when an analysis result is obtained indicating that there is a user whose bias toward a larger volume of comments is equal to or greater than the reference value, the transmission unit transmits as the prompting message a message that prompts comments from users other than the user. The control device according to claim 1 or 2, wherein when an analysis result is obtained indicating that there is a user whose bias toward low volume of comments is equal to or greater than the reference value, the transmission unit transmits a message as the prompting message to prompt the user to make a comment. The utterance data is voice data,
The control device includes:
a conversion unit that converts the voice data received by the data communication unit into text data;
a text sending unit that sends the text data obtained by the conversion unit to the plurality of terminal devices;
The control device according to any one of claims 1 to 3 , further comprising: A control system having a plurality of terminal devices and a control device communicating with the plurality of terminal devices,
The control device includes:
a data communication unit that receives utterance data for communication between users from each of the plurality of terminal devices and transmits text data indicating the contents of the utterances of the users of each of the plurality of terminal devices to the plurality of terminal devices;
a trend analysis unit that analyzes a trend of communication between the users based on the utterance data received by the data communication unit;
a transmission unit that transmits a guidance message regarding a progress of communication between the users to at least one terminal device among the plurality of terminal devices based on a predetermined analysis result obtained by the tendency analysis unit;
an identification unit that identifies the at least one terminal device to which the prompting message is to be sent based on a predetermined analysis result obtained by the tendency analysis unit;
having
the predetermined analysis result is an analysis result indicating the presence of a user whose degree of bias in message volume is equal to or greater than a reference value;
the identification unit identifies a terminal device used by a user with a second largest message volume among the plurality of terminal devices as the at least one terminal device;
The terminal device
a receiving unit for receiving the text data and the guidance message from the control device; and
a display control unit that generates a display screen on which the text data and the guidance message are arranged side by side;
A control system having receiving utterance data for communication between users from each of the plurality of terminal devices;
Analyzing a trend of communication between the users based on the received utterance data;
sending a guidance message regarding a progress of communication between the users to at least one terminal device among the plurality of terminal devices based on the result of the predetermined analysis;
identifying a terminal device used by a user with a second largest message volume as the at least one terminal device to which the prompting message is to be sent based on the predetermined analysis result;
Including,
A method executed by a control device , wherein the predetermined analysis result is an analysis result indicating the presence of a user whose bias in message volume is equal to or greater than a reference value .

Images