JP7619094B2 - Speech control program, speech control method, and speech control device - Google Patents

Description

The present invention relates to a speech control program, a speech control method, and a speech control device.

With the advancement of remote work, many audio conferences are being held remotely over networks. In audio conferences, because the devices of the participants are far from the server hosting the conference, there can be a delay before the audio is transmitted to others.

As a result, even if a participant speaks when the speaker has finished, another participant may also begin speaking, or the speaker may continue speaking, causing the participants' voices to overlap and disrupting the progress of the meeting.

Japanese Patent Application Publication No. 8-9063 JP 2006-60364 A Special Publication No. 2002-522998 Japanese Unexamined Patent Publication No. 6-1890052

However, unlike face-to-face meetings, audio conferences make it difficult for participants to observe each other's postures or facial expressions, making it difficult to time their speech. For this reason, there is a demand for technology that can prevent voices from overlapping in audio conferences.

One method for preventing voice overlap is to determine the priority of each speaker's speech based on their relevance to the meeting agenda. However, this method determines the priority before the meeting starts based on static information, and cannot handle cases that occur in real meetings where the speaker with the highest priority changes as the agenda progresses and changes.

In one aspect, the objective is to provide a speech control program, a speech control method, and a speech control device that can prevent overlapping of voices between participants in an audio conference.

In the first proposal, the speech control device calculates speech priority for multiple participants who speak within a specified time period based on characteristic words in meeting-related email sentences and keywords acquired sequentially from the meeting audio, and performs a process to mute speech from all participants except for specific ones based on the speech priority.

On one hand, it can prevent voice overlap between participants in an audio conference.

FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system according to a first embodiment. FIG. 2 is a functional block diagram of the speech control device 100 according to the first embodiment. FIG. 3 is a diagram illustrating an overall flow of the speech control process according to the first embodiment. FIG. 4 is a diagram illustrating an example of word extraction from past conference voice according to the first embodiment. FIG. 5 is a diagram illustrating an example of how topic relevance is determined according to the first embodiment. FIG. 6 is a diagram illustrating an example of extracted characteristic words from an email text according to the first embodiment. FIG. 7 is a diagram illustrating an example of determining position points according to the first embodiment. FIG. 8 is a diagram illustrating an example of calculation of a reference statement priority level according to the first embodiment. FIG. 9 is a diagram illustrating an example of keyword extraction from the conference voice according to the first embodiment. FIG. 10 is a diagram illustrating an example of points added to a message priority level according to the first embodiment. FIG. 11 is a diagram illustrating an example of speech control according to the first embodiment. FIG. 12 is a diagram illustrating an example of an output of a speech control display according to the first embodiment. FIG. 13 is a flowchart of the statement priority calculation process according to the first embodiment. FIG. 14 is a flowchart of the speech control process according to the first embodiment. FIG. 15 is a diagram illustrating an example of a hardware configuration.

Below, examples of the speech control program, speech control method, and speech control device disclosed in the present application are described in detail with reference to the drawings. Note that the present invention is not limited to these examples. Furthermore, the respective examples can be appropriately combined within a range that does not cause inconsistencies.

[Overall configuration example]
FIG. 1 is a diagram showing an example of the overall configuration of an information processing system according to a first embodiment. As shown in FIG. 1, the information processing system according to the first embodiment is a system in which a speech control device 100 and participant terminals 200-1 to 200-n (n is any integer; hereinafter, collectively referred to as "participant terminals 200") are connected to each other via a network N so as to be able to communicate with each other. Note that the network N can be any of a variety of communication networks, such as an intranet, whether wired or wireless. Furthermore, the network N is not a single network, and may be configured, for example, by connecting an intranet and the Internet via a network device such as a gateway or other device (not shown).

The speech control device 100 is the executing entity of this embodiment including Example 1, and is, for example, an information processing device such as a server computer managed by a company to which the participants of the audio conference belong. Alternatively, the speech control device 100 may be a cloud server device managed by a cloud service provider that provides cloud computing services. Furthermore, the speech control device 100 may be a distributed computing system composed of multiple computers.

The speech control device 100 provides an audio conference service to the participant terminals 200, receives the voice of each participant from each participant terminal 200, and distributes the voice of each participant to the participant terminals 200 used by the other participants. In addition to the voice, the speech control device 100 may also receive and distribute video captured by the camera function of the participant terminal 200 or a web camera device connected to the participant terminal 200.

In addition, the speech control device 100 calculates speech priority based on characteristic words in the meeting-related email text and keywords sequentially acquired from the meeting audio, and when the voices of participants overlap, mutes the speech of participants other than the one with the highest speech priority.

The participant terminal 200 is an information processing terminal used by the participants of the audio conference. The participant terminal 200 may be a desktop PC or a notebook PC installed in the participant's company or home, or may be a mobile terminal such as a smartphone or a tablet PC carried by the participant.

The participant terminal 200 is equipped with a microphone and speaker function for participants to participate in an audio conference, or is connected to an external device that provides these functions. In addition, an application for participating in an audio conference is pre-installed in the participant terminal 200. Alternatively, if the participant terminal 200 uses a web application on the speech control device 100 that has the same function as such an application, the application does not need to be installed in the participant terminal 200.

Participants can hold an audio conference by sending and receiving audio between each other via the participant terminal 200.

[Functional configuration of the speech control device 100]
Next, a description will be given of the functional configuration of the remark control device 100 shown in Fig. 1. Fig. 2 is a functional block diagram showing the functional configuration of the remark control device 100 according to the first embodiment. As shown in Fig. 2, the remark control device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

The communication unit 110 is a processing unit that controls communication with other devices, such as the participant terminal 200, and is, for example, a communication interface such as a network interface card.

The storage unit 120 is an example of a storage device that stores various data and programs executed by the control unit 130, such as a memory or a hard disk. The storage unit 120 stores email information 121, schedule information 122, organizational information 123, and meeting information 124.

The email information 121 stores information such as reply emails from each participant to the invitation email for the audio conference and characteristic words for each participant extracted from the text of the reply email. The email information 121 may be obtained from an information processing device such as a mail server other than the speech control device 100.

The schedule information 122 stores schedule information such as conferences of participants in an audio conference. The schedule information 122 may be obtained from an information processing device such as a schedule management server other than the speech control device 100.

The organizational information 123 stores information such as the participants' job titles and the job title points set for each job title. The organizational information 123 may be obtained from an information processing device other than the speech control device 100.

The conference information 124 stores information such as the conference audio received from the participant terminal 200, words extracted from the conference audio, and the number of times each word occurs.

Note that the storage unit 120 can store various information other than the above information. Also, the data structure of each piece of information is not limited to the above content.

The control unit 130 is a processing unit that controls the entire speech control device 100, and is, for example, a processor. The control unit 130 includes an extraction unit 131, a determination unit 132, a calculation unit 133, a mute unit 134, and an output unit 135. Each processing unit is an example of an electronic circuit that the processor has, or an example of a process that the processor executes.

The extraction unit 131 performs morphological analysis on the email text for each participant and extracts characteristic words. Note that the extraction unit 131 can use a statistical method such as the tf-idf method to extract characteristic words.

The extraction unit 131 also converts the conference voice into text, performs morphological analysis, and extracts keywords. More specifically, the extraction unit 131 converts the conference voice of a certain number of past conferences, for example 20 conferences, into text, performs morphological analysis, and pre-extracts words and the number of occurrences of the words for each conference. The extraction unit 131 also converts the voice of the current conference received from the participant terminal 200 into text in real time, performs morphological analysis, and extracts words and the number of occurrences of the words. Then, the extraction unit 131 extracts important words, i.e., keywords, in the current conference based on the words and the number of occurrences of the past conferences and the words and the number of occurrences of the current conference. The extraction unit 131 can also use statistical methods such as the tf-idf method to extract keywords.

The determination unit 132 determines the agenda relevance, which is the degree of relevance to the meeting, for each participant based on the number of characters in the email text. For example, the determination unit 132 may determine the agenda relevance in proportion to the number of characters in the email text.

In addition, when the determination unit 132 receives the voices of at least two or more of the participants who have started speaking within a specified time, it determines the participant with the highest speech priority calculated by the calculation unit 133 as the specific person. Any speech other than that of the specific person determined here will be muted, but the reason why a certain amount of time is allowed for the reception of the voice within the specified time is to take into account the time lag between the participant terminals 200 in receiving the voice.

The calculation unit 133 calculates a comment priority for multiple participants who spoke within a specified time based on the characteristic words of the conference-related email text and the keywords successively acquired from the conference audio. More specifically, for example, a participant with a greater number of matches between the characteristic words extracted from the email text and the keywords extracted from the conference audio is determined to be more relevant to the current agenda, and a higher comment priority is calculated. Note that since the keywords extracted from the conference audio change successively in response to changes in the agenda of the conference, the comment priority for each participant also changes dynamically during the conference.

The calculation unit 133 can also calculate the comment priority based on the subject relevance and job title of the participant, which are determined in advance by the determination unit 132 from the number of characters in the email text. The comment priority determined here is a comment priority that can be determined before the conference and serves as a standard for the comment priority that changes dynamically during the conference as described above. In this way, the comment control device 100 calculates a standard comment priority for each participant before the conference, and dynamically changes the comment priority in response to changes in the subject and other factors during the conference. This allows the comment control device 100 to handle cases where participants with high comment priority change as the proceedings progress, as occurs in actual conferences.

The muting unit 134 mutes comments from participants other than the specific participant with the highest speech priority determined by the determination unit 132 based on the speech priority calculated by the calculation unit 133. In addition, when any participant is speaking, the muting unit 134 prioritizes the speech of the speaker who is already speaking, and therefore mutes comments from participants other than the speaker, regardless of the speech priority of each participant.

When a participant's speech is muted, the output unit 135 causes the participant terminal 200 to output a display indicating that the speech is muted and a display indicating that the speech has priority.

[Function details]
Next, the utterance control process according to the first embodiment will be described in detail. Fig. 3 is a diagram showing an overall flow of the utterance control process according to the first embodiment. In the example of Fig. 3, the utterance control process is executed by the utterance control device 100. In the example of Fig. 3, steps S1 to S4 of the utterance control process are processing steps executed before the conference, and steps S5 to S9 are processing steps executed during the conference.

First, the speech control device 100 extracts words from past conference audio (step S1). This is a process of extracting reference words from the conference audio of a certain number of past conferences, for example 20 conferences, in order to extract characteristic keywords from the conference audio of the conference in which the speech of participants is to be controlled.

Next, the speech control device 100 determines the topic relevance for each participant from the number of characters in the conference-related email (step S2). Here, the topic relevance is determined so that, for example, the topic relevance is higher for participants who write more characters in their emails, based on the idea that participants who write more characters in their conference-related emails will be more relevant to the conference.

Next, the speech control device 100 extracts characteristic words from the conference-related email text for each participant (step S3). The characteristic words extracted here for each participant are compared with keywords extracted from the conference audio of the conference for which the participant's speech is controlled, and the speech priority of the participant is controlled to be higher, for example, as the characteristic words and keywords match.

Note that steps S2 and S3 may be executed, for example, by searching schedule information 122 every 10 minutes, and if there is a meeting that is to start within 10 minutes, searching email information 121 for a reply email to the invitation email for that meeting. Note that the search interval for schedule information 122 is not limited to 10 minutes.

Next, the speech control device 100 determines position points for each participant (step S4). Here, the position points are determined so that the higher the position of the participant, the higher the position. This is based on the idea that, in the speech control process of Example 1, the higher the position points, the higher the speech priority, and therefore, particularly in Japan, priority is given to the speech of participants with higher positions. Therefore, when there is no need to control speech priority based on position, it is not necessary to determine position points.

In the example of FIG. 3, steps up to S4 are shown to be executed before the meeting, but the execution timing of each processing step can be changed, for example, so that step S4 is executed during the meeting.

Next, the speech control device 100 determines a standard speech priority for each participant (step S5). This is a process that is executed, for example, at the start of the conference, to determine the speech priority for each participant based on the topic relevance determined in step S2 and the position points determined in step S4.

Next, the speech control device 100 extracts keywords from the audio of the ongoing conference (step S6). This is a process in which words are sequentially extracted in real time from the audio of the ongoing conference, and characteristic keywords are extracted by comparing the words with the reference words from past conference audio extracted in step S1.

Next, the speech control device 100 adds points to the speech priority determined in step S5 (step S7). This is a process of dynamically adding points to the speech priority so that the more the characteristic words for each participant extracted in step S3 match the keywords extracted in step S6, the higher the speech priority of that participant becomes.

Next, the speech control device 100 controls the speech of participants when their voices overlap (step S8). For example, when multiple participants start speaking and their voices overlap, this is a process of muting the speech of all participants except the one with the highest speech priority. Or, when a specific participant is already speaking, this is a process of muting the voice of a participant who starts speaking later.

Next, when the speech control device 100 mutes speech other than that of a specific participant, it outputs a speech control display (step S9). This is a process in which, when speech is muted in step S8, the participant terminal 200 outputs, for example, a display indicating that the speech has been muted or a display indicating that the speech has been given priority.

The overall flow of the speech control process in Example 1 has been described above using Figure 3. Next, each of the processing steps S1 to S9 of the speech control process described using Figure 3 will be described in more detail using Figures 4 to 12.

Figure 4 is a diagram showing an example of word extraction from past conference audio according to the first embodiment. The extraction process described using Figure 4 corresponds to step S1 in Figure 3.

As shown in FIG. 4, the speech control device 100 converts the conference audio of past conferences A to C into text, performs morphological analysis, and extracts words and their frequency of occurrence for each conference. Note that the example in FIG. 4 shows three past conferences, A to C, but the number of conferences to be extracted is not limited to three. A speech-to-text conversion tool such as Azure Speech API (Application Programming Interface) may be used to convert the conference audio into text.

Next, FIG. 5 is a diagram showing an example of determining the topic relevance in Example 1. The determination process described using FIG. 5 corresponds to step S2 in FIG. 3.

As shown in FIG. 5, the speech control device 100 determines the topic relevance for each participant from the number of characters in the reply email to the conference-related invitation email for which the participant's speech is being controlled. The topic relevance may be determined as 0, 1, 2, etc., for each range of reply email character counts: 0-99, 100-199, 200-299, etc. In addition, if one participant has multiple emails, the number of characters in the reply emails may be the total number of characters in the multiple emails, but quoted portions in the reply emails are not counted.

Next, FIG. 6 is a diagram showing an example of extracting characteristic words from an email text according to the first embodiment. The extraction process described with reference to FIG. 6 corresponds to step S3 in FIG. 3.

As shown in FIG. 6, the speech control device 100 first extracts, for each participant, words and their frequency of occurrence from the conference-related email text that controls the participant's speech. Note that if one participant has multiple emails, the frequency of occurrence of a word may be the total frequency of occurrence in the multiple emails, but quoted portions in reply emails are excluded from the count. The speech control device 100 then extracts words with high tf-idf values, for example, the top 10%, as characteristic words for the participant that characterize the participant's interests.

Next, FIG. 7 is a diagram showing an example of determining position points in Example 1. The determination process described using FIG. 7 corresponds to step S4 in FIG. 3.

As shown in FIG. 7, the speech control device 100 determines the position points for each participant by, for example, awarding higher pre-set points to participants according to their higher position. Note that the numerical values of the position points in FIG. 7 are merely examples and are not limited to these. Furthermore, the positions may be set more finely or coarsely than in the example shown in FIG. 7.

Next, FIG. 8 is a diagram showing an example of calculation of a reference comment priority according to the first embodiment. The calculation process described using FIG. 8 corresponds to step S5 in FIG. 3.

As shown in FIG. 8, the speech control device 100 determines a standard speech priority for each participant by, for example, adding up the topic relevance determined in step S2 and the position points determined in step S4. Note that the speech priority determined here is merely a standard, and the speech priority can change dynamically based on keywords successively extracted from the conference audio.

Next, FIG. 9 is a diagram showing an example of keyword extraction from the conference audio according to the first embodiment. The extraction process described with reference to FIG. 9 corresponds to step S6 in FIG. 3.

As shown in FIG. 9, the speech control device 100 first converts the audio of the ongoing conference into text in predetermined time units, such as one minute, and performs morphological analysis to extract words and their frequency of occurrence. A speech-to-text conversion tool, such as Azure Speech API, may also be used to convert the audio of the ongoing conference into text.

Next, the speech control device 100 compares the words extracted from the audio of the ongoing conference and their frequency of occurrence with the words extracted from the audio of past conferences extracted in step S1 of FIG. 3 and their frequency of occurrence. The speech control device 100 then extracts characteristic keywords that appear in the audio of the ongoing conference from the comparison results. Keywords are extracted using a statistical method such as the tf-idf method, and words with high tf-idf values, for example, the top 10%, are extracted as keywords.

The reason why words and their occurrence counts are extracted from the audio of an ongoing meeting in specified time units, such as one minute, is to accommodate cases where important words change as the agenda progresses.

Next, FIG. 10 is a diagram showing an example of adding points to a comment priority level in Example 1. The adding point process described using FIG. 10 corresponds to step S7 in FIG. 3.

As shown in FIG. 10, the speech control device 100 adds, for example, the number of matches between the characteristic words for each participant extracted in step S3 of FIG. 3 and the keywords for the ongoing conference extracted in step S6 to the speech priority determined in step S5.

As described above, the keywords of the ongoing conference extracted in step S6 are extracted sequentially in predetermined time units, such as one minute, so the addition of the comment priority in step S7 is also performed each time a keyword is extracted. At that time, the previous addition to the comment priority is reset, and the number of matches between the newly extracted keyword and the characteristic words of each participant is added to the comment priority.

Next, FIG. 11 is a diagram showing an example of speech control according to the first embodiment. The control process described using FIG. 11 corresponds to step S8 in FIG. 3.

As shown in FIG. 11, the speech control device 100 controls the speech of conference participants when they speak at the same time and their voices overlap. In the example of FIG. 11, the speech control device 100 receives the voices of A and B as they begin to speak at the same time, so the speech control device 100 compares the speech priorities of A and B, prioritizes the voice of A, who has a higher speech priority, and mutes the voice of B. Note that the voices received at the same time here refer to voices received within, for example, 0.5 seconds or 1 second, taking into account the time lag between the participant terminals 200 relative to the reception of the voices.

As shown in FIG. 11, when participants speak at different times and the voices do not overlap, the speech control device 100 distributes each voice to the participant terminal 200 without muting them. Also, when another participant, C, is already speaking, as in the case of D's speech in the example of FIG. 11, the speech control device 100 mutes the speech of D, who started speaking later, regardless of speech priority.

Also, as shown in FIG. 11, the speech control device 100 converts the audio that is not muted and is delivered to the participant terminal 200 into text in predetermined time units, such as one minute units, and extracts keywords.

Next, FIG. 12 is a diagram showing an example of the speech control display output according to the first embodiment. The output process described using FIG. 12 corresponds to step S9 in FIG. 3.

As shown in FIG. 12, when the speech control device 100 mutes speech other than that of a specific participant, it causes each participant terminal 200 to output a speech control display indicating that speech is being controlled. One example of a speech control display is a muted display 250 indicating that speech is being muted, which is output to the participant terminal 200-1 of the participant whose speech has been muted, as shown in FIG. 12. On the other hand, a priority speech display 260 indicating that speech is being distributed with priority is output to the participant terminal 200-2 of the participant making a priority speech to the muted participant.

Note that the speech control display shown in FIG. 12 is merely an example. For example, a display indicating that a participant is muted or making a priority speech may be output near the icon or name of each participant in the participant list displayed on the participant terminal 200.

[Process flow]
The overall flow of the speech control process and each processing step have been described in detail above using Fig. 3. The speech control process can be divided into a speech priority calculation process that calculates speech priorities for controlling speech of participants when their voices overlap, and a speech control process that controls speech based on speech priorities when their voices overlap. Each process will be described using the flowcharts in Figs. 13 and 14.

First, the flow of the speech priority calculation process according to the first embodiment executed by the speech control device 100 will be described. FIG. 13 is a flowchart showing the flow of the speech priority calculation process according to the first embodiment. The speech priority calculation process shown in FIG. 13 is a process for calculating a speech priority for controlling the speech of a participant when the voices of the participants overlap.

The speech control device 100 searches the schedule information 122, for example, every 10 minutes, and if there is a meeting that is starting within 10 minutes, it obtains a reply email to the invitation email for that meeting from the email information 121 (step S101).

Next, the speech control device 100 determines, for each participant, the topic relevance, which is the degree of relevance to the meeting, in proportion to the number of characters in the email text obtained in step S101 (step S102).

Next, the speech control device 100 determines a standard speech priority for each participant based on the topic relevance determined in step S102 (step S103). Note that at this time, the speech control device 100 may obtain, for example, position points from the organizational information 123, which are set higher for participants with higher positions, and determine the standard speech priority further based on the position points.

Next, the speech control device 100 performs morphological analysis on the email text obtained in step S101 for each participant, and extracts the top 10% of characteristic words using the tf-idf method (step S104).

Next, if the speech control device 100 has not received the participant's voice from the participant terminal 200 (step S105: No), it waits for the voice to be received. On the other hand, if the speech has been received (step S105: Yes), the speech control device 100 converts the received speech into text, performs morphological analysis, and extracts keywords (step S106).

For keyword extraction in step S106, the utterance control device 100 extracts words and their frequency of occurrence from each conference audio received in one-minute increments, for example. The utterance control device 100 then compares the words and their frequency of occurrence extracted from each conference audio with the words and their frequency of occurrence in audio from past conferences that have been extracted in advance, and extracts the top 10% of keywords using the tf-idf method.

Next, the speech control device 100, for example, for each participant, compares the keywords extracted in step S106 with the characteristic words extracted in step S104 and calculates the number of matches (step S107).

Next, the speech control device 100, for example, uses the number of matches calculated in step S107 for each participant to add points to the standard speech priority determined in step S103 (step S108). Note that for participants with zero matches, the added points are also zero, so no points are actually added.

Next, if a predetermined time, such as one minute, has not elapsed from a predetermined interval, such as one minute (step S109: No), the process returns to step S105, and the utterance control device 100 waits to receive further audio until the end of the conference. On the other hand, if a predetermined time, such as one minute, has elapsed (step S109: Yes), the utterance control device 100 resets the added points for the utterance priority in step S108, and returns to the standard utterance priority (step S110). After executing step S110, the utterance control device 100 returns to step S105, and waits to receive further audio until the end of the conference.

Next, the speech control process according to the first embodiment, which is executed by the speech control device 100, will be described in sequence. FIG. 14 is a flowchart showing the flow of the speech control process according to the first embodiment. The speech control process shown in FIG. 14 is a process for controlling speech based on speech priority when the voices of participants overlap.

First, if the speech control device 100 has not received the participant's voice from the participant terminal 200 (step S201: No), it waits for the reception of the voice. On the other hand, if the speech control device 100 has received the voice of another participant (step S201: Yes), it determines whether the other participant is speaking or not (step S202).

If another participant is already speaking (step S202: Yes), the speech control device 100 mutes the audio received in step S201 (step S205).

On the other hand, if no participant is speaking (step S202: No), the speech control device 100 determines whether or not there is conference audio received simultaneously in addition to the audio received in step S201 (step S203). Note that simultaneous reception here may be reception within, for example, 0.5 seconds or 1 second, taking into account the time lag between the participant terminals 200 for receiving the audio.

If there is no other conference audio received simultaneously (step S203: No), the speech control device 100 distributes the audio received in step S201 to each participant terminal 200 without muting it, and the speech control process shown in FIG. 14 ends.

On the other hand, if there is other conference audio received simultaneously (step S203: Yes), the speech control device 100 determines the speaker based on the speech priority calculated by the speech priority calculation process shown in FIG. 13 (step S204).

Next, the speech control device 100 distributes the voice of the speaker determined in step S204 to each participant terminal 200 without muting it, and mutes the voices of all speakers other than that speaker (step S205).

Next, the speech control device 100 outputs a speech control display to the participant terminal 200 (step S206). The speech control display is, for example, a display indicating that a speech has been muted or a display indicating that a speech has been prioritized, and can be output to the participant terminal 200 of the participant whose speech has been muted or prioritized. Alternatively, the speech control device 100 may output a speech control display indicating that a speech is being muted or prioritized near the icon or name of each participant in the participant list displayed on the participant terminal 200. After step S206 is executed, the speech control process shown in FIG. 14 ends.

[effect]
As described above, the speech control program causes the speech control device 100 to calculate speech priority for multiple participants who speak within a specified period of time based on characteristic words in conference-related email sentences and keywords acquired successively from the conference audio, and to execute a process of muting speech from all participants other than specific ones based on the speech priority.

As a result, the speech control device 100 mutes the speech of all participants in an audio conference except for the one with the highest priority, thereby preventing overlapping speech between participants in the audio conference.

The speech control program also causes the speech control device 100 to execute a process for determining the relevance of each participant to the conference based on the number of characters in the email text, and the speech control device 100 calculates the speech priority based on the relevance.

This allows the speech control device 100 to prevent overlapping voices between participants in an audio conference by using speech priorities that are more appropriate for each audio conference.

The speech control device 100 also calculates speech priority based on the participants' job titles.

This allows the speech control device 100 to prevent voice overlap between participants in an audio conference by using speech priorities that are more appropriate for the participants in the audio conference.

The speech control program also causes the speech control device 100 to perform morphological analysis on the email text for each participant, extract characteristic words, convert the conference audio into text, perform morphological analysis, and further execute the process of extracting keywords, and the speech control device 100 calculates speech priority based on the relevance and the number of matches between the characteristic words and the keywords.

As a result, the speech control device 100 can dynamically change the speech priority in response to changes in the agenda during the audio conference, and use the speech priority that is more appropriate for the conference content, thereby preventing overlapping of voices between participants in the audio conference.

The keyword extraction process that the speech control program causes the speech control device 100 to execute includes converting the conference audio into text, performing morphological analysis, extracting a first word and a first number of occurrences of the first word, converting a second conference audio other than the conference audio into text, performing morphological analysis, extracting a second word and a second number of occurrences of the second word for each conference, and extracting keywords based on the first word, the first number of occurrences, the second word, and the second number of occurrences.

As a result, the speech control device 100 can dynamically change the speech priority in response to changes in the agenda during the audio conference, and use the speech priority that is more appropriate for the conference content, thereby preventing overlapping of voices between participants in the audio conference.

The speech control device 100 also uses the tf-idf method to extract feature words and keywords.

This allows the speech control device 100 to extract characteristic words for each participant and keywords from the conference audio that are more suited to the participants in the audio conference and the content of the conference.

The speech control program also causes the speech control device 100 to execute a process of determining the participant with the highest speech priority as the specified person when the speech of at least two or more of the participants who have started speaking is received within a second predetermined time.

This allows the speech control device 100 to prevent voice overlap between participants in an audio conference.

In addition, the speech control program further causes the speech control device 100 to execute a process of muting speech from anyone other than the speaker when any of the participants is speaking.

This allows the speech control device 100 to prevent voice overlap between participants in an audio conference.

[system]
The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. In addition, the specific examples, distributions, values, etc. described in the embodiments are merely examples and can be changed as desired.

In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. In other words, all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. For example, the extraction unit 131 and the determination unit 132 of the speech control device 100 can be integrated.

Furthermore, each processing function performed by each device may be realized, in whole or in part, by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[Hardware]
The hardware configuration of the above-mentioned speech control device 100 will be described. Fig. 15 is a diagram showing an example of the hardware configuration. As shown in Fig. 15, the speech control device 100 has a communication unit 100a, a HDD (Hard Disk Drive) 100b, a memory 100c, and a processor 100d. The units shown in Fig. 15 are connected to each other via a bus or the like.

The communication unit 100a is a network interface card or the like, and communicates with other servers. The HDD 100b stores the programs and DBs that operate the functions shown in FIG. 2.

The processor 100d reads out a program that executes the same processes as the processing units shown in FIG. 2 from the HDD 100b or the like and loads it into the memory 100c, thereby operating a process that executes the functions described in FIG. 2. For example, this process executes the same functions as the processing units of the speech control device 100. Specifically, for example, the processor 100d reads out a program that has the same functions as the determination unit 132, the calculation unit 133, and the like from the HDD 100b or the like. Then, the processor 100d executes a process that executes the same processes as the determination unit 132, the calculation unit 133, and the like.

In this way, the utterance control device 100 operates as an information processing device that executes each process by reading and executing a program. The utterance control device 100 can also realize functions similar to those of the above-mentioned embodiment by reading the program from a recording medium using a media reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the utterance control device 100. For example, the present invention can be similarly applied to cases where another computer or server executes a program, or where these cooperate to execute a program.

This program can be distributed via a network such as the Internet. This program can also be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and can be executed by being read from the recording medium by a computer.

The following notes are further provided with respect to the embodiments including Example 1 above.

(Appendix 1) A speech control device,
Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio;
A speech control program that executes a process of muting speeches of participants other than those of specific participants based on the speech priority.

(Additional Note 2) The speech control device includes:
determining a degree of relevance to the conference for each of the participants based on the number of characters in the email message;
2. The speech control program according to claim 1, wherein the speech control device calculates the speech priority based on the relevance.

(Appendix 3) The speech control program described in Appendix 2 is characterized in that the speech control device calculates the speech priority based on the role of the participant.

(Additional Note 4) The speech control device includes:
performing a morphological analysis on the email text for each of the participants to extract the characteristic words;
The conference voice is converted into text, a morphological analysis is performed, and the keywords are extracted.
The comment control program according to claim 2 or 3, wherein the comment control device calculates the comment priority based on the relevance and the number of matches between the feature word and the keyword.

(Note 5) The process of extracting the keywords executed by the speech control device includes:
converting the conference voice into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word;
A second conference voice other than the conference voice is converted into text, a morphological analysis is performed, and a second word and a second number of occurrences of the second word are extracted for each conference;
5. The speech control program according to claim 4, further comprising a process of extracting the keyword based on the first word, the first number of occurrences, the second word, and the second number of occurrences.

(Appendix 6) The speech control program described in any one of appendices 1 to 5, characterized in that the speech control device uses the tf-idf method to extract the characteristic words and the keywords.

(Supplementary Note 7) The speech control device includes:
The speech control program according to any one of claims 1 to 6, further comprising a process of determining the participant with the highest speech priority as the specific person when speech start voices of at least two or more of the participants are received within a second predetermined time.

(Supplementary Note 8) The speech control device includes:
8. The speech control program according to claim 1, further comprising a process of muting speeches made by participants other than the speaker when any of the participants is speaking.

(Supplementary Note 9) A speech control device,
Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio;
A speech control method comprising: executing a process of muting speech from other than that of specific participants based on the speech priority.

(Appendix 10) For multiple participants who make comments within a specified time, a comment priority is calculated based on characteristic words in emails related to the conference and keywords sequentially acquired from the conference audio,
A speech control device comprising: a control unit that executes a process of muting speech from any of the participants other than specific participants based on the speech priority.

(Supplementary Note 11) The control unit
determining a relevance of each of the participants to the conference based on the number of characters in the email message;
11. The speech control device according to claim 10, wherein the control unit calculates the speech priority based on the relevance.

(Appendix 12) The speech control device described in Appendix 11, characterized in that the control unit calculates the speech priority based on the role of the participant.

(Supplementary Note 13) The control unit
performing a morphological analysis on the email text for each of the participants to extract the characteristic words;
converting the conference voice into text, performing morphological analysis, and extracting the keywords;
13. The speech control device according to claim 11, wherein the control unit calculates the speech priority based on the relevance and the number of matches between the feature word and the keyword.

(Additional Note 14) The process of extracting the keywords executed by the control unit includes:
converting the conference voice into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word;
A second conference voice other than the conference voice is converted into text, a morphological analysis is performed, and a second word and a second number of occurrences of the second word are extracted for each conference;
The speech control device according to claim 13, further comprising a process of extracting the keyword based on the first word, the first number of occurrences, the second word, and the second number of occurrences.

(Appendix 15) The speech control device described in any one of appendices 10 to 14, characterized in that the control unit uses the tf-idf method to extract the characteristic words and the keywords.

(Supplementary Note 16) The control unit
The speech control device according to any one of claims 10 to 15, further comprising a process of determining the participant with the highest speech priority as the specific person when speech start voices of at least two or more of the participants are received within a second predetermined time.

(Supplementary Note 17) The control unit,
The speech control device according to any one of claims 10 to 16, further comprising a process of muting speeches made by any participant other than the participant when the participant is speaking.

(Supplementary Note 18) A processor;
A speech control device comprising: a memory operably connected to the processor, the processor comprising:
Calculate comment priorities for multiple participants who comment within a specified time period based on characteristic words in emails related to the meeting and keywords sequentially acquired from the meeting audio;
A speech control device which executes a process of muting speech from any of the participants other than specific participants based on the speech priority.

Though the present invention has been described above with reference to examples, the present invention may be implemented in a variety of different forms other than the examples described above.

100: speech control device 100a: communication unit 100b: HDD
100c Memory 100d Processor 110 Communication unit 120 Storage unit 121 Mail information 122 Schedule information 123 Organization information 124 Conference information 130 Control unit 131 Extraction unit 132 Determination unit 133 Calculation unit 134 Muting unit 135 Output unit 200 Participant terminal 250 Muted display 260 Priority speech display

Claims (10)

A system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, in which a plurality of participants using the plurality of participant terminals hold a conference by voice,
The speech control device includes :
calculating , for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice;
A speech control program that executes a process of muting speeches of participants other than a specific person having the highest speech priority based on the speech priority . The speech control device includes:
determining a degree of relevance to the conference for each of the participants based on the number of characters in the email message;
2. The program according to claim 1, wherein the speech control device calculates the speech priority based on the relevance. The speech control program according to claim 2, characterized in that the speech control device calculates the speech priority based on the role of the participant. The speech control device includes:
performing a morphological analysis on the email text for each of the participants to extract the characteristic words;
The conference voice is converted into text, a morphological analysis is performed, and the keywords are extracted.
4. The comment control program according to claim 2, wherein the comment control device calculates the comment priority based on the relevance and the number of matches between the feature words and the keywords. The process of extracting the keyword that is executed by the speech control device includes:
Converting the conference voice of the current conference into text, performing morphological analysis, and extracting a first word and a first number of occurrences of the first word;
Converting a second conference speech of a past conference into text, performing a morphological analysis, and extracting a second word and a second number of occurrences of the second word for each conference;
5. The speech control program according to claim 4, further comprising a process for extracting the keyword based on the first word, the first number of occurrences, the second word, and the second number of occurrences. The speech control program according to any one of claims 1 to 5, characterized in that the speech control device uses the tf-idf method to extract the characteristic words and the keywords. The speech control device includes:
The speech control program according to any one of claims 1 to 6, further comprising a process of determining the participant having the highest speech priority as the specific person when speech start voices of at least two of the participants are received within a second predetermined time. The speech control device includes:
8. The speech control program according to claim 1, further comprising a process for muting speeches from participants other than the speaker when any of the participants is speaking. A system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, in which a plurality of participants using the plurality of participant terminals hold a conference by voice,
The speech control device,
calculating , for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice;
a process for muting speech from among the participants other than a specific person having the highest speech priority based on the speech priority, said process comprising: muting said speech from among the participants other than a specific person having the highest speech priority based on the speech priority. A speech control device used in a system having a speech control device and a plurality of participant terminals capable of communicating with each other via a network, the system comprising:
calculating , for each participant, a speech priority level that is higher for a participant having a greater number of matches between the characteristic words and the keywords based on the characteristic words of the email texts related to the conference and the keywords sequentially acquired from the conference voice;
A speech control device comprising: a control unit that executes a process of muting speeches of the participants other than those of a specific person having the highest speech priority based on the speech priority .

Images