JP6975755B2 - Voice analyzer, voice analysis method, voice analysis program and voice analysis system - Google Patents

Description

The present invention relates to a voice analyzer for analyzing voice, a voice analysis method, a voice analysis program and a voice analysis system.

The harkness method (also referred to as the harkness method) is known as a method for analyzing discussions in group learning and meetings (see, for example, Non-Patent Document 1). In the hackness method, the transition of each participant's remark is recorded by a line. This makes it possible to analyze each participant's contribution to the discussion and their relationship with others. The hackness method can also be effectively applied to active learning in which students study independently.

Paul Sevigny, "Extreme Discussion Circles: Preparing ESL Students for" The Harkness Method "", Polyglossia, Ritsumeikan Asia Pacific University Language Education Center, October 2012, No. 23, p. 181-191

When performing an analysis similar to the Harkness method using a computer, the sound acquired by the sound collector is obtained for each participant by setting the position of each participant based on the position of the sound collector such as a microphone. analyse. Therefore, there is a problem that it takes a lot of time and effort to set the position of each participant for each group.

The present invention has been made in view of these points, and provides a voice analyzer, a voice analysis method, a voice analysis program, and a voice analysis system that can reduce the trouble of setting the position of a participant when analyzing the voice of a discussion. The purpose is to provide.

The voice analyzer of the first aspect of the present invention is a setting unit that acquires information about a plurality of participants from a sound collecting device and sets the positions of each of the plurality of participants based on the acquired information about the participants. It has an acquisition unit that acquires voice from the sound collecting device, and an analysis unit that analyzes the voice emitted by each of the plurality of participants based on the position set by the setting unit.

The setting unit may set the position of each of the plurality of participants by acquiring voice from the sound collecting device as information about the participant and specifying the direction in which the acquired voice is emitted. ..

The setting unit acquires an image captured by an image pickup unit provided on the sound collecting device as information about the participant, and recognizes the faces of the plurality of participants included in the acquired image. The position of each of the plurality of participants may be set.

The setting unit acquires the information of the card read by the reading unit provided on the sound collecting device as the information about the participant, and each of the plurality of participants according to the direction in which the card is presented to the reading unit. You may set the position of.

The setting unit may set the position of each of the plurality of participants based on the information input in the communication terminal in addition to the information about the participants.

In the middle of the voice being analyzed by the analysis unit, there may be further a follow-up unit that updates the position set by the setting unit.

The following unit may update the position set by the setting unit when the direction in which the voice analyzed by the analysis unit is emitted does not correspond to the position set by the setting unit. ..

The following unit may update the position set by the setting unit in the direction in which the voice being analyzed by the analysis unit is emitted.

In the voice analysis method of the second aspect of the present invention, the processor acquires information about a plurality of participants from the sound collector, and sets the position of each of the plurality of participants based on the acquired information about the participants. A step of acquiring a voice from the sound collecting device, and a step of analyzing the voice emitted by each of the plurality of participants based on the position set in the setting step. ..

The voice analysis program of the third aspect of the present invention acquires information about a plurality of participants from a sound collector in a computer, and sets the positions of each of the plurality of participants based on the acquired information about the participants. A step of acquiring a voice from the sound collecting device, and a step of analyzing the voice emitted by each of the plurality of participants based on the position set in the setting step. ..

The voice analysis system according to the fourth aspect of the present invention includes a voice analyzer and a sound collector capable of communicating with the voice analyzer, and the sound collector acquires voice and a plurality of participants. The voice analyzer is configured to acquire information about the participant, and the voice analyzer acquires information about the participant from the sound collector and sets the position of each of the plurality of participants based on the acquired information about the participant. The setting unit, the acquisition unit that acquires the sound from the sound collecting device, and the analysis unit that analyzes the sound emitted by each of the plurality of participants based on the position set by the setting unit. Have.

According to the present invention, there is an effect that it is possible to reduce the trouble of setting the position of the participants when analyzing the voice of the discussion.

It is a schematic diagram of the voice analysis system which concerns on this embodiment. It is a block diagram of the voice analysis system which concerns on this embodiment. It is a schematic diagram of the voice analysis method performed by the voice analysis system which concerns on this embodiment. It is a figure which shows the whole flow chart of the voice analysis method performed by the voice analysis apparatus which concerns on this embodiment. It is a front view of the display part of the communication terminal which displays a setting screen. It is a schematic diagram of the automatic setting process performed by the voice analyzer which concerns on this embodiment. It is a figure which shows the flowchart of the position setting process performed by the voice analysis apparatus which concerns on this embodiment. It is a schematic diagram of the follow-up process performed by the voice analysis apparatus which concerns on this embodiment. It is a figure which shows the flowchart of the follow-up process performed by the voice analysis apparatus which concerns on this embodiment.

[Overview of voice analysis system S]
FIG. 1 is a schematic diagram of a voice analysis system S according to the present embodiment. The voice analysis system S includes a voice analysis device 100, a sound collection device 10, and a communication terminal 20. The number of sound collecting devices 10 and communication terminals 20 included in the voice analysis system S is not limited. The voice analysis system S may include other devices such as servers and terminals.

The voice analyzer 100, the sound collecting device 10, and the communication terminal 20 are connected via a network N such as a local area network or the Internet. At least a part of the voice analyzer 100, the sound collecting device 10, and the communication terminal 20 may be directly connected without going through the network N.

The sound collector 10 includes a microphone array including a plurality of sound collectors (microphones) arranged in different directions. For example, a microphone array contains eight microphones evenly spaced on the same circumference in a horizontal plane with respect to the ground. The sound collector 10 transmits the voice acquired by using the microphone array to the voice analyzer 100 as data.

The communication terminal 20 is a communication device capable of performing wired or wireless communication. The communication terminal 20 is, for example, a mobile terminal such as a smartphone terminal or a computer terminal such as a personal computer. The communication terminal 20 accepts the setting of analysis conditions from the analyst and displays the analysis result by the voice analyzer 100.

The voice analysis device 100 is a computer that analyzes the voice acquired by the sound collector 10 by a voice analysis method described later. Further, the voice analysis device 100 transmits the result of the voice analysis to the communication terminal 20.

[Configuration of voice analysis system S]
FIG. 2 is a block diagram of the voice analysis system S according to the present embodiment. In FIG. 2, the arrows indicate the main data flows, and there may be data flows not shown in FIG. In FIG. 2, each block shows not a hardware (device) unit configuration but a functional unit configuration. Therefore, the block shown in FIG. 2 may be mounted in a single device, or may be mounted separately in a plurality of devices. Data can be exchanged between blocks via any means such as a data bus, a network, and a portable storage medium.

The sound collecting device 10 has an imaging unit 11 for capturing an image of the participants in the discussion, and a reading unit 12 for reading information such as a card presented by the participants in the discussion. The imaging unit 11 is an imaging device capable of imaging a predetermined imaging range including the faces of each participant (that is, each of the plurality of participants). The image pickup unit 11 includes an image pickup element having a number and arrangement capable of capturing the faces of all the participants surrounding the sound collecting device 10. For example, the image pickup unit 11 includes two image pickup elements arranged in different directions every 180 degrees in a horizontal plane with respect to the ground. Further, the imaging unit 11 may image the faces of all the participants surrounding the sound collecting device 10 by rotating in a horizontal plane with respect to the ground.

The imaging unit 11 may perform imaging at a timing preset in the sound collecting device 10 (for example, every 10 seconds), or may perform imaging according to an imaging instruction received from the voice analyzer 100. The image pickup unit 11 transmits an image showing the captured content to the voice analyzer 100.

The reading unit 12 has a reading device (card reader) that reads information recorded on an IC (Integrated Circuit) card or a magnetic card (hereinafter collectively referred to as a card) presented by a participant by a contact method or a non-contact method. .. An IC chip built in a smartphone or the like may be used as an IC card. The reading unit 12 is configured to be able to identify the orientation of the participant who presented the card. For example, the reading unit 12 includes twelve reading devices arranged in different directions every 30 degrees in a horizontal plane with respect to the ground. Further, for example, the reading unit 12 may include a button for designating the orientation of the participants in addition to the reading device.

When the card is presented by the participant, the reading unit 12 reads the information on the card by the reading device and identifies the orientation of the participant based on which reading device reads the card. Then, the reading unit 12 associates the read information with the orientation of the participant and transmits it to the voice analyzer 100.

The communication terminal 20 has a display unit 21 for displaying various information and an operation unit 22 for receiving an operation by an analyst. The display unit 21 includes a display device such as a liquid crystal display and an organic light emission diode (OLED) display. The operation unit 22 includes operation members such as buttons, switches, and dials. The display unit 21 and the operation unit 22 may be integrally configured by using a touch screen capable of detecting the position of contact by the analyst as the display unit 21.

The voice analyzer 100 has a control unit 110, a communication unit 120, and a storage unit 130. The control unit 110 includes a position setting unit 111, a voice acquisition unit 112, a sound source localization unit 113, a tracking unit 114, an analysis unit 115, and an output unit 116. The storage unit 130 includes a position storage unit 131, a voice storage unit 132, and an analysis result storage unit 133.

The communication unit 120 is a communication interface for communicating with the sound collecting device 10 and the communication terminal 20 via the network N. The communication unit 120 includes a processor, a connector, an electric circuit, and the like for executing communication. The communication unit 120 performs predetermined processing on the communication signal received from the outside to acquire data, and inputs the acquired data to the control unit 110. Further, the communication unit 120 performs predetermined processing on the data input from the control unit 110 to generate a communication signal, and transmits the generated communication signal to the outside.

The storage unit 130 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 130 stores in advance the program executed by the control unit 110. The storage unit 130 may be provided outside the voice analyzer 100, and in that case, data may be exchanged with the control unit 110 via the communication unit 120.

The position storage unit 131 stores information indicating the positions of the participants in the discussion. The voice storage unit 132 stores the voice acquired by the sound collecting device 10. The analysis result storage unit 133 stores the analysis result indicating the result of analyzing the voice. The position storage unit 131, the voice storage unit 132, and the analysis result storage unit 133 may each be a storage area on the storage unit 130, or may be a database configured on the storage unit 130.

The control unit 110 is, for example, a processor such as a CPU (Central Processing Unit), and by executing a program stored in the storage unit 130, a position setting unit 111, a voice acquisition unit 112, a sound source localization unit 113, and a tracking unit 114 are executed. , As an analysis unit 115 and an output unit 116. The functions of the position setting unit 111, the voice acquisition unit 112, the sound source localization unit 113, the tracking unit 114, the analysis unit 115, and the output unit 116 will be described later with reference to FIGS. 3 to 9. At least some of the functions of the control unit 110 may be performed by an electric circuit. Further, at least a part of the functions of the control unit 110 may be executed by a program executed via the network.

The voice analysis system S according to the present embodiment is not limited to the specific configuration shown in FIG. For example, the voice analyzer 100 is not limited to one device, and may be configured by connecting two or more physically separated devices by wire or wirelessly.

[Explanation of voice analysis method]
FIG. 3 is a schematic diagram of a voice analysis method performed by the voice analysis system S according to the present embodiment. First, the position setting unit 111 of the voice analyzer 100 sets the position of each participant in the discussion to be analyzed by the position setting process described later (a). The position setting unit 111 sets the position of each participant by storing the position of each participant specified in the position setting process described later in the position storage unit 131.

When the voice acquisition unit 112 of the voice analyzer 100 starts the voice acquisition, the voice acquisition unit 112 transmits a signal instructing the voice acquisition to the sound collector 10 (b). When the sound collecting device 10 receives a signal instructing the voice analysis device 100 to acquire the voice, the sound collecting device 10 starts acquiring the voice. Further, the voice acquisition unit 112 of the voice analyzer 100 transmits a signal instructing the end of voice acquisition to the sound collector 10 when the voice acquisition is completed. When the sound collecting device 10 receives a signal instructing the end of voice acquisition from the voice analyzer 100, the sound collecting device 10 ends the voice acquisition.

The sound collecting device 10 acquires sound in each of the plurality of sound collecting units, and internally records the sound as the sound of each channel corresponding to each sound collecting unit. Then, the sound collecting device 10 transmits the acquired voices of the plurality of channels to the voice analyzer 100 (c). The sound collecting device 10 may sequentially transmit the acquired sound, or may transmit the sound of a predetermined amount or a predetermined time. Further, the sound collecting device 10 may collectively transmit the voices from the start to the end of the acquisition. The voice acquisition unit 112 of the voice analysis device 100 receives voice from the sound collection device 10 and stores it in the voice storage unit 132.

The voice analysis device 100 analyzes the voice at a predetermined timing by using the voice acquired from the sound collector 10. The voice analyzer 100 may analyze the voice when the analyst gives an analysis instruction by a predetermined operation on the communication terminal 20. In this case, the analyst selects the voice corresponding to the discussion to be analyzed from the voices stored in the voice storage unit 132.

Further, the voice analysis device 100 may analyze the voice when the acquisition of the voice is completed. In this case, the voice from the start to the end of the acquisition corresponds to the discussion to be analyzed. Further, the voice analyzer 100 may analyze the voice sequentially (that is, by real-time processing) during the acquisition of the voice. In this case, in the voice analyzer 100, the voice for the past predetermined time (for example, 30 seconds) retroactively from the current time corresponds to the discussion of the analysis target.

When analyzing the voice, first, the sound source localization unit 113 performs sound source localization based on the voices of the plurality of channels acquired by the voice acquisition unit 112 (d). The sound source localization is a process of estimating the direction of the sound source included in the voice acquired by the voice acquisition unit 112 every time (for example, every 10 milliseconds to 100 milliseconds). The sound source localization unit 113 associates the direction of the sound source estimated for each time with the position of the participant stored in the position storage unit 131.

The sound source localization unit 113 can use a known sound source localization method such as a MUSIC (Multiple Signal Classification) method or a beamforming method as long as the direction of the sound source can be specified based on the sound acquired from the sound collector 10. ..

Next, the analysis unit 115 analyzes the voice based on the voice acquired by the voice acquisition unit 112, the direction of the sound source estimated by the sound source localization unit 113, and the position of the participant stored in the position storage unit 131 (e). .. The analysis unit 115 may analyze the entire completed discussion, or may use a part of the discussion as the analysis target in the case of real-time processing.

Specifically, first, the analysis unit 115 analyzes the voice acquired by the voice acquisition unit 112, the direction of the sound source estimated by the sound source localization unit 113, and the position of the participant stored in the position storage unit 131. In the discussion, it is determined which participant speaks (speaks) every hour (for example, every 10 milliseconds to 100 milliseconds). The analysis unit 115 specifies a continuous period from the start to the end of the speech by one participant as the speech period, and stores the analysis result storage unit 133. When a plurality of participants make remarks at the same time, the analysis unit 115 specifies a remark period for each participant.

Further, the analysis unit 115 calculates the amount of speech of each participant for each hour and stores it in the analysis result storage unit 133. Specifically, the analysis unit 115 calculates the value obtained by dividing the length of time that the participant made a statement by the length of the time window in a certain time window (for example, 5 seconds) as the amount of speech for each hour. do. Then, the analysis unit 115 calculates the amount of speech for each participant for each participant while shifting the time window by a predetermined time (for example, 1 second) from the start time of the discussion to the end time (current in the case of real-time processing). repeat.

The tracking unit 114 acquires the latest participant positions at predetermined time intervals in the voice to be analyzed by the sound source localization unit 113 and the analysis unit 115 by the tracking process described later, and the participants stored in the position storage unit 131. Update the position of. As a result, even if the participant moves from the position already set during the acquisition of the voice, the sound source localization unit 113 and the analysis unit 115 can follow and analyze the voice.

The output unit 116 controls to display the analysis result by the analysis unit 115 on the display unit 21 by transmitting the display information to the communication terminal 20 (f). The output unit 116 is not limited to the display on the display unit 21, and may output the analysis result by other methods such as printing by a printer and data recording in a storage device.

FIG. 4 is a diagram showing an overall flowchart of a voice analysis method performed by the voice analysis device 100 according to the present embodiment. First, the position setting unit 111 identifies the position of each participant of the discussion to be analyzed by the position setting process described later, and stores the position in the position storage unit 131 (S1). Next, the voice acquisition unit 112 acquires voice from the sound collecting device 10 and stores it in the voice storage unit 132 (S2).

The voice analyzer 100 analyzes the voice acquired by the voice acquisition unit 112 in step S2 for each predetermined time range (time window) from the start time to the end time. The sound source localization unit 113 executes sound source localization in the time range of the sound to be analyzed, and associates the estimated direction of the sound source with the position of each participant stored in the position storage unit 131 (S3).

The tracking unit 114 acquires the latest participant position in the time range of the voice to be analyzed by the tracking process described later, and updates the participant position stored in the position storage unit 131 (S4).

The analysis unit 115 analyzes the voice based on the voice acquired by the voice acquisition unit 112 in step S2, the direction of the sound source estimated by the sound source localization unit 113 in step S3, and the position of the participant stored in the position storage unit 131. (S5). The analysis unit 115 stores the analysis result in the analysis result storage unit 133.

If the analysis has not been completed by the end time of the voice acquired by the voice acquisition unit 112 in step S2 (NO in S6), the voice analyzer 100 has the next time range in the voice to be analyzed in steps S3 to S5. repeat. When the analysis is completed by the end time of the voice acquired by the voice acquisition unit 112 in step S2 (YES in S6), the output unit 116 outputs the analysis result of step S5 by a predetermined method (S7).

[Explanation of position setting process]
First, the position setting process shown in step S1 of FIG. 4 will be described. FIG. 5 is a front view of the display unit 21 of the communication terminal 20 displaying the setting screen A. The position setting process includes a manual setting process in which the analyst sets the position of each participant by operating the communication terminal 20, and information for each participant to specify his / her own position is input to the sound collector 10. Includes automatic setting processing.

The communication terminal 20 displays the setting screen A on the display unit 21 and accepts the analysis condition setting by the analyst. The setting screen A includes a position setting area A1, a start button A2, an end button A3, and an automatic setting button A4. The position setting area A1 is an area for setting the direction in which each participant U is actually located with respect to the sound collecting device 10 in the discussion of the analysis target. For example, the position setting area A1 represents a circle centered on the position of the sound collector 10 as shown in FIG. 5, and further represents an angle with respect to the sound collector 10 along the circle.

The analyst who desires the manual setting process sets the position of each participant U in the position setting area A1 by operating the operation unit 22 of the communication terminal 20. Identification information (here, U1 to U4) that identifies each participant U is assigned and displayed in the vicinity of the position set for each participant U. In the example of FIG. 5, four participants U1 to U4 are set. The portion corresponding to each participant U in the position setting area A1 is displayed in a different color for each participant. This allows the analyst to easily recognize the orientation in which each participant U is set.

The start button A2, the end button A3, and the automatic setting button A4 are virtual buttons displayed on the display unit 21, respectively. When the start button A2 is pressed by the analyst, the communication terminal 20 transmits a start instruction signal to the voice analyzer 100. When the end button A3 is pressed by the analyst, the communication terminal 20 transmits a signal of an end instruction to the voice analyzer 100. In this embodiment, one discussion is from the start instruction to the end instruction by the analyst.

An analyst who desires the automatic setting process causes the voice analyzer 100 to start the automatic setting process by pressing the automatic setting button A4. When the automatic setting button A4 is pressed, the communication terminal 20 transmits a signal of an automatic setting instruction to the voice analyzer 100.

[Explanation of automatic setting process]
6 (a) to 6 (c) are schematic views of the automatic setting process performed by the voice analyzer 100 according to the present embodiment, respectively. When the automatic setting process is instructed, the voice analyzer 100 sets the position of the participant U by at least one of the processes shown in FIGS. 6 (a) to 6 (c).

FIG. 6A shows a process of setting the position of the participant U based on the voice emitted by the participant U. In this case, the position setting unit 111 of the voice analyzer 100 causes the sound collecting unit of the sound collecting device 10 to acquire the voice emitted by each participant U. The position setting unit 111 acquires the sound acquired by the sound collecting device 10.

The position setting unit 111 identifies the direction of each participant U based on the direction in which the acquired voice is emitted. The position setting unit 111 uses the result of sound source localization by the sound source localization unit 113 described above in order to specify the direction of the participant from the voice. Then, the position setting unit 111 stores the position of each specified participant U in the position storage unit 131.

The position setting unit 111 may identify the individual of the participant U by comparing the acquired voice of each participant U with the voice of an individual stored in advance in the voice analyzer 100. For example, the positioning unit 111 identifies an individual by comparing the voice prints (that is, the frequency spectrum of the voice) of each participant U's voice. As a result, the personal information of the participant U can be displayed together with the analysis result, or a plurality of analysis results of the same participant U can be displayed.

FIG. 6B shows a process of setting the position of the participant U based on the image of the face of the participant U. In this case, the position setting unit 111 of the voice analyzer 100 causes the image pickup unit 11 provided in the sound collector 10 to take an image of a region including the faces of all the participants U surrounding the sound collector 10. The position setting unit 111 acquires an image captured by the image pickup unit 11.

The position setting unit 111 recognizes the face of each participant U in the acquired image. The position setting unit 111 can use a known face recognition technique for recognizing a human face from an image. Then, the position setting unit 111 identifies the position of each participant U with respect to the sound collecting device 10 based on the position of the face of each participant U recognized from the image, and stores the position in the position storage unit 131. The relationship between the position in the image (for example, the coordinates of the pixels in the image) and the position with respect to the sound collecting device 10 (for example, the angle with respect to the sound collecting device 10) is set in the voice analyzer 100 in advance.

The position setting unit 111 may identify the individual of the participant U by comparing the face of each participant U recognized from the image with the face of the individual stored in advance in the voice analyzer 100. As a result, the personal information of the participant U can be displayed together with the analysis result, or a plurality of analysis results of the same participant U can be displayed.

FIG. 6C shows a process of setting the position of the participant U based on the information of the card C presented by the participant U. In this case, the position setting unit 111 of the voice analysis device 100 causes the reading unit 12 provided in the sound collecting device 10 to read the information of the card C presented by each participant U. The position setting unit 111 acquires the information of the card C read by the reading unit 12 and the orientation of the participant U who presented the card C. The position setting unit 111 identifies the position of each participant U with respect to the sound collecting device 10 based on the acquired information of the card C and the orientation of the participant U, and stores the position in the position storage unit 131.

The position setting unit 111 may identify the individual of the participant U by acquiring the personal information stored in the voice analyzer 100 in advance by using the acquired information of the card C. As a result, the personal information of the participant U can be displayed together with the analysis result, or a plurality of analysis results of the same participant U can be displayed.

The position setting unit 111 may execute the automatic setting process and the manual setting process in combination. In this case, for example, the position setting unit 111 displays the position of each participant U set by the automatic setting process of FIGS. 6A to 6C in the position setting area A1 of FIG. 5, and further analyzes the position. Accepts manual settings by the user. As a result, the position of each participant U set by the automatic setting process can be corrected by the manual setting process, and the position of each participant U can be set more reliably.

In this way, the voice analyzer 100 can automatically set the position of each participant U based on the information about the participant U acquired on the sound collector 10, so that the analyst can set the position of each participant U on the communication terminal 20 for all groups. It is possible to reduce the trouble of setting the position of each participant U. The voice analyzer 100 is not limited to voice, image, or card information as information about the participant U (that is, information for specifying the position of the participant U) that can be acquired on the sound collecting device 10. Other information that can identify the orientation may be used.

FIG. 7 is a diagram showing a flowchart of a position setting process performed by the voice analyzer 100 according to the present embodiment. First, the position setting unit 111 determines whether or not the automatic setting process is instructed by the analyst on the setting screen A in FIG. 5. When the automatic setting process is not instructed (that is, in the case of manual setting) (NO in S11), the position setting unit 111 of each participant according to the contents input on the setting screen A displayed on the communication terminal 20. The position is specified and set in the position storage unit 131 (S12).

When the automatic setting is instructed (YES in S11), the position setting unit 111 acquires information about the participant (that is, information for specifying the position of the participant) on the sound collecting device 10 (S13). The position setting unit 111 uses at least one of the above-mentioned voice emitted by the participant, the image of the participant's face, and the information of the card presented by the participant as the information regarding the participant.

The position setting unit 111 identifies the position of each participant U with respect to the sound collector 10 based on the acquired information about the participants (S14). Then, the position setting unit 111 sets the position of the participant by storing the position of each specified participant in the position storage unit 131 (S15).

[Explanation of tracking process]
Next, the follow-up process shown in step S4 of FIG. 4 will be described. FIG. 8 is a schematic diagram of the follow-up process performed by the voice analyzer 100 according to the present embodiment. The follow-up process is a process of updating the position of each participant U stored in the position storage unit 131 in the middle of the voice to be analyzed by the sound source localization unit 113 and the analysis unit 115.

The upper figure of FIG. 8 shows the position of each participant U before the update, and the lower figure of FIG. 8 shows the position of each participant U after the update. The upper figure of FIG. 8 shows a state in which the participant U1 has moved from the position P1 set in the position storage unit 131 to another position P2. In this state, the voice emitted by the participant U1 enters the sound collecting device 10 from a position P2 different from the set position P1 of the participant U1. Therefore, the analysis unit 115 cannot detect the remark of the participant U1 from the voice.

Therefore, the following unit 114 updates the position of the participant U1 from the position P1 to the position P2 in the position storage unit 131 as shown in the lower figure of FIG. As a result, the analysis unit 115 can correctly detect the remarks of the participant U1.

For updating the position of each participant U, the tracking unit 114 acquires the direction of the sound source estimated by the sound source localization unit 113 at predetermined time (for example, 1 minute). The tracking unit 114 states that when the estimated sound source orientation does not correspond to any of the positions of each participant U stored in the position storage unit 131, one of the participants U moves in the direction of the sound source. judge. Then, the following unit 114 identifies the moved participant U, and updates the position stored in the position storage unit 131 for the moved participant U to a position corresponding to the direction of the sound source.

For example, the tracking unit 114 identifies that the participant U set at the position closest to the direction of the sound source estimated from the sound acquired by the sound collecting device 10 has moved to the position corresponding to the direction of the sound source. In this case, the tracking unit 114 selects the moved participant U from the participants U set at positions within a predetermined range (for example, within the range of -30 degrees to +30 degrees) from the direction of the sound source. May be good. By limiting the range of movement, the tracking unit 114 can prevent the participant U from moving to an erroneous position, for example.

Further, the tracking unit 114 compares the voiceprint of the sound source with the voiceprint of each participant U, and identifies that the participant U having a voiceprint similar to the voiceprint of the sound source has moved to a position corresponding to the direction of the sound source. May be good. In this case, the tracking unit 114 may acquire the voiceprint of each participant U from the voice of each participant U at the start of analysis, or may acquire the voiceprint of each participant U stored in the storage unit 130 in advance. May be good. The tracking unit 114 calculates the similarity of the voiceprints between the voiceprints of the sound source and the voiceprints of each participant U. The follow-up unit 114 selects the participant U having the highest degree of similarity of the voiceprint in the group, or selects the participant U having the degree of similarity of the voiceprint equal to or higher than a predetermined threshold value. By identifying the participant U who has moved using the voiceprint, the accuracy of tracking can be improved.

Further, the tracking unit 114 acquires a face located in the direction of the sound source in the image captured by the image pickup unit 11 of the sound collecting device 10, and the participant U having a face similar to the acquired face is the direction of the sound source. It may be specified that it has moved to the position corresponding to. In this case, the following unit 114 may acquire the face of each participant U from the image captured by the imaging unit 11 at the start of analysis, or may acquire the face of each participant U previously stored in the storage unit 130. May be good. The tracking unit 114 calculates the degree of face similarity between the face located in the direction of the sound source and the face of each participant U. The follow-up unit 114 selects the participant U having the highest facial similarity in the group, or selects the participant U whose facial similarity is equal to or higher than a predetermined threshold value. By identifying the participant U who has moved using the face, the accuracy of tracking can be improved.

When following using a voiceprint or face, the tracking unit 114 has a voiceprint or a voiceprint for each participant U based on the difference between the orientation of the sound source and the position (orientation) stored in the position storage unit 131. Face similarity may be weighted. The closer the position set to the participant U and the direction of the sound source are, the higher the probability that the participant U has moved to the position of the sound source, and the farther the position set to the participant U and the direction of the sound source are, the more the participant U is. It can be said that the probability of moving to the position of the sound source is low. Therefore, the tracking unit 114 weights the similarity of the voiceprint or face higher as the difference between the position set for the participant U and the direction of the sound source is smaller, and the difference between the position set for the participant U and the direction of the sound source. The larger the value, the lower the weighting of the voiceprint or facial similarity. This makes it possible to further improve the accuracy of tracking.

FIG. 9 is a diagram showing a flowchart of the follow-up process performed by the voice analyzer 100 according to the present embodiment. First, the tracking unit 114 acquires the direction of the sound source estimated by the sound source localization unit 113. When the direction of the sound source corresponds to any of the positions of the participants U stored in the position storage unit 131 (YES in S41), the tracking unit 114 ends the process without updating the position. ..

When the direction of the sound source does not correspond to any of the positions of the participants U stored in the position storage unit 131 (NO in S41), the tracking unit 114 informs the participants on the sound collecting device 10. (That is, information for identifying the position of the participant) is acquired (S42). The tracking unit 114 uses at least one of the above-mentioned voice emitted by the participant and the image of the participant's face as information about the participant.

The follow-up unit 114 identifies which participant has moved based on the acquired information about the participant (S43). Then, the following unit 114 updates the position stored in the position storage unit 131 to the position corresponding to the direction of the sound source for the participant U specified to have moved (S44).

[Effect of this embodiment]
The voice analysis device 100 according to the present embodiment is information about each participant such as a voice emitted by a participant in a sound collecting device 10 arranged in each group, an image of a participant's face, and information on a card presented by the participant. And automatically set the position of each participant based on the acquired information. Therefore, it is possible to reduce the trouble of setting the position of each participant for each group when analyzing the audio of the discussion.

In addition, the voice analyzer 100 updates the position of each participant based on the information about each participant during the voice analysis. Therefore, even if the participant moves during the acquisition of the voice, the analysis can be followed.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in any unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

In the above description, the voice analyzer 100 is used for voice analysis in discussions, but can also be applied to other uses. For example, the voice analyzer 100 can also analyze the voice emitted by a passenger sitting in a car.

The processors of the voice analyzer 100, the sound collector 10, and the communication terminal 20 are the main constituents of each step included in the voice analysis methods shown in FIGS. 4, 7, and 9. That is, the processors of the voice analysis device 100, the sound collector 10 and the communication terminal 20 read a program for executing the voice analysis method shown in FIGS. 4, 7 and 9 from the storage unit, and execute the program for voice analysis. By controlling each part of the device 100, the sound collecting device 10, and the communication terminal 20, the voice analysis method shown in FIGS. 4, 7, and 9 is executed. The steps included in the voice analysis method shown in FIGS. 4, 7 and 9 may be partially omitted, the order between the steps may be changed, or a plurality of steps may be performed in parallel.

S Voice analysis system 100 Voice analysis device 110 Control unit 111 Position setting unit 112 Voice acquisition unit 114 Follow-up unit 115 Analysis unit 10 Sound collector 20 Communication terminal

Claims (14)

An acquisition unit that acquires sounds emitted by a plurality of participants surrounding the sound collector from the sound collector, and
An analysis unit that analyzes the voice emitted by each of the plurality of participants based on the positions set for each of the plurality of participants.
In the middle of the voice being analyzed by the analysis unit, the orientation of the sound source that emitted the voice being analyzed by the analysis unit with respect to the sound collector is set for the plurality of participants. A tracking unit that updates the position set for the plurality of participants when none of the positions is supported.
Have a,
The following unit has the characteristics of the voice analyzed by the analysis unit by comparing the characteristics of the voice analyzed by the analysis unit with the characteristics of the voice emitted by each of the plurality of participants. The participant who emits a voice having similar characteristics is selected, and the position of the selected participant is oriented with reference to the sound collector of the sound source which emits the voice analyzed by the analysis unit. A voice analyzer to update. The following unit calculates the degree of similarity between the characteristics of the voice analyzed by the analysis unit and the characteristics of the voice of each of the plurality of participants at the start of the voice analyzed by the analysis unit. , Select the participant whose similarity meets a predetermined condition,
The voice analyzer according to claim 1. The following unit calculates the degree of similarity between the characteristics of the voice analyzed by the analysis unit and the characteristics of the voice of each of the plurality of participants stored in advance in the storage unit, and the degree of similarity is predetermined. Select the participants who meet the conditions,
The voice analyzer according to claim 1. Each of the plurality of participants depends on the difference between the direction in which the voice being analyzed by the analysis unit is emitted and the direction corresponding to the position set for each of the plurality of participants. The degree of similarity is weighted and the weighted degree of similarity selects the participant who satisfies a predetermined condition.
The voice analyzer according to claim 2 or 3. Get the sound from the sound collecting device as the information for the previous SL participants, by specifying the acquired direction the sound is issued, further comprising a setting unit for setting the positions of the plurality of participants, wherein Item 6. The speech analyzer according to any one of Items 1 to 4. Acquiring an image by the imaging unit has captured with said sound collecting device as the information for the previous SL participants, by recognizing the face of the plurality of participants included in the acquired image, of each of the plurality of participants The voice analyzer according to any one of claims 1 to 4, further comprising a setting unit for setting a position. The sound collecting device has a reading unit that reads information recorded on the card.
Set acquires information the reader is recorded on the card read, the setting information recorded on the card, according Oite specified orientation to the reading unit, the respective positions of the plurality of participants The voice analyzer according to any one of claims 1 to 4 , further comprising a unit. The processor runs ,
A step of acquiring sounds emitted by a plurality of participants surrounding the sound collector from the sound collector, and
A step of, based on the position set for each of the plurality of participants, analyzing the voice which each of the plurality of participants uttered,
In the middle of the voice analyzed in the analysis step, the orientation of the sound source that emitted the voice analyzed in the analysis step with respect to the sound collector is directed to the plurality of participants. A step of updating the position set for the plurality of participants when none of the set positions is supported, and a step of updating the position.
Have,
The updating step is the voice analyzed in the analysis step by comparing the characteristics of the voice analyzed in the analysis step with the characteristics of the voice emitted by each of the plurality of participants. The participant who emits a voice having characteristics similar to the characteristics of the above is selected, and the position of the selected participant is referred to the sound collector of the sound source which emits the voice analyzed in the step of analysis. A voice analysis method that updates in the direction of. On the computer
A step of acquiring sounds emitted by a plurality of participants surrounding the sound collector from the sound collector, and
A step of, based on the position set for each of the plurality of participants, analyzing the voice which each of the plurality of participants uttered,
In the middle of the voice analyzed in the analysis step, the orientation of the sound source that emitted the voice analyzed in the analysis step with respect to the sound collector is directed to the plurality of participants. A step of updating the position set for the plurality of participants when none of the set positions is supported, and a step of updating the position.
To execute ,
The updating step is the voice analyzed in the analysis step by comparing the characteristics of the voice analyzed in the analysis step with the characteristics of the voice emitted by each of the plurality of participants. The participant who emits a voice having characteristics similar to the characteristics of the above is selected, and the position of the selected participant is referred to the sound collector of the sound source which emits the voice analyzed in the step of analysis. A voice analysis program that updates in the direction of. A voice analyzer and a sound collector capable of communicating with the voice analyzer are provided.
The voice analyzer is
An acquisition unit that acquires sounds emitted by a plurality of participants surrounding the sound collector from the sound collector, and
An analysis unit that analyzes the voice emitted by each of the plurality of participants based on the positions set for each of the plurality of participants.
In the middle of the voice being analyzed by the analysis unit, the orientation of the sound source that emitted the voice being analyzed by the analysis unit with respect to the sound collector is set for the plurality of participants. A tracking unit that updates the position set for the plurality of participants when none of the positions is supported.
Have a,
The following unit has the characteristics of the voice analyzed by the analysis unit by comparing the characteristics of the voice analyzed by the analysis unit with the characteristics of the voice emitted by each of the plurality of participants. The participant who emits a voice having similar characteristics is selected, and the position of the selected participant is oriented with reference to the sound collector of the sound source which emits the voice analyzed by the analysis unit. A voice analysis system to update. An acquisition unit that acquires sounds emitted by a plurality of participants surrounding the sound collector from the sound collector, and
An analysis unit that analyzes the voice emitted by each of the plurality of participants based on the positions set for each of the plurality of participants.
In the middle of the voice being analyzed by the analysis unit, the orientation of the sound source that emitted the voice being analyzed by the analysis unit with respect to the sound collector is set for the plurality of participants. A tracking unit that updates the position set for the plurality of participants when none of the positions is supported.
Have,
The following unit includes a face located in the direction in which the sound analyzed by the analysis unit in the image captured by the image pickup unit of the sound collecting device is emitted, and the faces of the plurality of participants. By comparison, the participant having a face similar to the face located in the direction in which the voice is emitted is selected by the analysis unit, and the analysis unit determines the position of the selected participant. A voice analyzer that updates the direction of the sound collector of the sound source that emitted the voice being analyzed with reference to the sound collector. The following unit includes a face located in the direction in which the sound analyzed by the analysis unit is emitted, and the image in the image captured by the image pickup unit at the start of the sound being analyzed by the analysis unit. The degree of similarity with the face of each of the plurality of participants is calculated, and the participant whose similarity satisfies a predetermined condition is selected.
The voice analyzer according to claim 11. The following unit calculates the degree of similarity between the face located in the direction in which the voice is emitted, which is being analyzed by the analysis unit, and the faces of the plurality of participants stored in the storage unit in advance. Select the participants whose similarity meets a predetermined condition,
The voice analyzer according to claim 11. Each of the plurality of participants depends on the difference between the direction in which the voice being analyzed by the analysis unit is emitted and the direction corresponding to the position set for each of the plurality of participants. The degree of similarity is weighted and the weighted degree of similarity selects the participant who satisfies a predetermined condition.
The speech analyzer according to claim 12 or 13.

Images