JP7309460B2 - sound detector - Google Patents

Description

Embodiments of the present invention relate to audio sensing devices.

Conventionally, there is a speech recognition device that performs speech recognition called an AI speaker (smart speaker). If speech recognition is always activated, the speech recognition device not only repeatedly executes unnecessary recognition processing, but also may cause an unexpected malfunction. For this reason, speech recognition devices often provide a trigger for activating speech recognition at a desired timing. For example, some speech recognition devices are triggered by the utterance of a specific word called a wake word.

However, a wake word-triggered speech recognition device needs to constantly capture sound and monitor whether the captured sound has a wake word. A speech recognition apparatus that operates in this manner has a problem of large power consumption even in standby mode. For example, a small voice recognition device that is powered by a battery has the problem that it is difficult to use it for a long time in an environment without charging.

JP-A-2002-91491

In order to solve the above-described problems, a voice detection device is provided that can control the operation of a terminal based on the voice uttered by a person.

According to embodiments, an audio sensing device comprises a microphone, a memory and a processor. Multiple microphones to pick up the sound. The memory stores information indicating locations of a plurality of terminals. The processor identifies the position of the sound source of the sound based on the phase difference of the sounds acquired by the plurality of microphones, and identifies one terminal based on the position of the sound source and the positions of the plurality of terminals stored in the memory. direct the action.

FIG. 1 is a diagram schematically showing a configuration example of a speech processing system including a speech detection device according to an embodiment. FIG. 2 is a block diagram showing a configuration example of the audio detection device according to the embodiment. FIG. 3 is a block diagram illustrating a configuration example of a terminal to which the voice detection device according to the embodiment instructs an operation; FIG. 4 is a diagram illustrating an example of processing functions realized by a processor included in the audio detection device according to the embodiment; FIG. 5 is a diagram illustrating a relationship example between a plurality of microphones and a plurality of terminals in the voice detection device according to the embodiment; FIG. 6 is a diagram for explaining sound source localization processing by the audio detection device according to the embodiment. FIG. 7 is a diagram illustrating a configuration example of a terminal position database referred to by the voice detection device according to the embodiment; FIG. 8 is a diagram illustrating an example of an operation instruction supplied to a terminal by the audio detection device according to the embodiment; FIG. 9 is a flowchart for explaining an operation example of the audio detection device according to the embodiment.

Hereinafter, embodiments will be described with reference to the drawings.
A voice detection device according to an embodiment described below collects sound with a plurality of microphones, detects a sound (speech) emitted by a person from the collected sound, and estimates the position of the person who sent the sound. . The voice detection device determines a voice recognition device (terminal) that performs voice recognition processing for recognizing the voice uttered by the person according to the estimated position of the person. The voice detection device activates a terminal determined from the person's position.

First, an example of a speech processing system including the speech detection device 2 according to the embodiment will be described.
FIG. 1 is a diagram showing a configuration example of a speech processing system including a speech detection device 2 according to an embodiment.
In the example shown in FIG. 1, the speech processing system comprises a speech detection device 2 and a plurality of terminals 3 (3a, 3b, . . . ). The voice detection device 2 detects a specific word (wake word) uttered by a person. The voice detection device 2 detects the position of the person who issued the wake word, and identifies the terminal 3 near the person. A terminal 3 near the person who issued the wake word is identified. The voice detection device 2 activates the terminal 3 in the vicinity of the person.

A plurality of terminals 3 are activated according to instructions from the voice detection device 2 . The terminal 3 is a small electronic device with a battery. For example, the terminal 3 is a speech recognition device realized by an AI speaker, tablet terminal, smartphone, or the like. After the terminal 3 as a speech recognition device is activated by the speech detection device 2, it recognizes the speech uttered by a person and executes processing according to the speech recognition result.
In this embodiment, the terminal 3 is described as being a speech recognition device, but it may be an electronic device other than the speech recognition device.

In the example shown in FIG. 1, the terminal 3 is installed for each table arranged within one floor (predetermined area). The voice detection device 2 is installed on one floor (predetermined area) where the terminal 3 is arranged. The voice detection device 2 may be installed so that voices uttered by people on the floor can be input by a plurality of microphones. That is, the voice detection device 2 (microphone of the voice detection device 2) is installed so that the position of the person can be specified from the voice uttered by the person at any place on the floor. For example, the audio detection device 2 is arranged on the ceiling near the center of the floor. Also, the voice detection device 2 may be installed near the center of the distribution of the plurality of terminals 3 or the plurality of tables on which the terminals 3 are installed.

Next, the configuration of the audio detection device 2 according to the embodiment will be described.
FIG. 2 is a block diagram showing a configuration example of the audio detection device 2 according to the embodiment.
As shown in FIG. 2, the voice detection device 2 includes a processing unit 20, a plurality of microphones M (M1, M2, M3, . . . ), a voice input interface (I/F) 21, a buffer memory 22, a data memory 23 and a terminal. It has an interface (I/F) 24 . The processing unit 20 also has a processor 25, a ROM 26, a RAM 27, and the like. Processor 25 of processing unit 20 is connected to voice input I/F 21, buffer memory 22, data memory 23 and terminal I/F 24 via internal interfaces.

The processor 25 controls the entire audio detection device 2 . Also, the processor 25 is, for example, a CPU. Processor 25 uses RAM 27 to implement various processes based on programs and control data stored in ROM 26 or data memory 23 . For example, the processor 25 performs various processes such as operation control of the voice detection device 2 and voice detection processing by executing programs stored in the ROM 25 .
Note that some of the various functions realized by the processor 25, which will be described later, may be realized by hardware.

The ROM 26 is a non-volatile memory that stores control programs and control data in advance. The ROM 26 is incorporated in the voice detection device 2 while storing control programs, control data, and the like.

RAM 27 is a volatile memory. RAM 27 functions as a working memory. The RAM 27 temporarily stores data being processed by the processor 25, for example. RAM 27 stores application programs based on instructions from processor 25 . Also, the RAM 27 may store data necessary for executing the application program, execution results of the application program, and the like.

A plurality of microphones M (M1, M2, M3, . . . ) each collects (obtains) sounds. Each microphone M, for example, collects sound input as analog information. A plurality of microphones M are arranged at different positions. At least three or more of a plurality of microphones M1, M2, M3, . . . Each microphone M is arranged so that the position of the sound source can be specified by the collected sound. For example, a plurality of microphones M are arranged so as to cover a 360-degree range on a floor (plane) on which a person can move, starting from a reference position.

Note that the data memory 23 holds information indicating the installation position of each microphone M in order to specify the sound source from the sounds collected by the multiple microphones M. FIG. The processor 25 identifies the position of the sound source by analyzing the sound collected (acquired) by the multiple microphones M while referring to the installation position of each microphone M.
However, the plurality of microphones M may include one connected as an external device of the sound detection device 2 . In this case, the voice detection device 2 stores information indicating the installation position of each microphone, including the microphones used as external devices, in a data memory or the like.

A voice input interface (I/F) 21 is an interface for inputting sounds collected by the microphones M (M1, M2, M3, . . . ). The voice input I/F 21 is configured to input sounds collected by the microphones M, respectively. The audio input I/F 21 has an AD converter and the like, and supplies the sound input by the microphone M to the processing unit 20 as a digital signal. Through the audio input I/F 21, the processor 25 of the processing unit 20 acquires audio data as sounds collected by the microphones M1, M2, . . .

The buffer memory 22 is a memory for temporarily holding data. The buffer memory 22 holds data of sound collected by each microphone M input via the audio input I/F 21 .
The data memory 23 is composed of a non-volatile memory in which data can be written and rewritten. For example, the data memory 23 is composed of an HDD (hard disk drive), SSD (solid state drive), EEPROM (registered trademark), flash ROM, or the like. The data memory 23 stores control programs, applications, and various data according to the operational use of the voice detection device 2 . The data memory 23 may also store a terminal location database that indicates the location of each terminal. The terminal location database will be explained in detail later.

A terminal interface (I/F) 24 is an interface for communicating with each terminal 3 . The terminal I/F 24 may communicate with the terminal 3 wirelessly, or may communicate with the terminal 3 by wire. The terminal I/F 24 is, for example, an interface corresponding to the communication interface that the terminal 3 has. The processor 25 of the processing unit 20 communicates with the designated terminal 3 via the terminal I/F 24 . For example, the terminal I/F 24 outputs a signal instructing the terminal 3 in the sleep state to wake up.

Next, the configuration of terminal 3 will be described.
FIG. 3 is a block diagram showing a configuration example of the terminal 3. As shown in FIG.
As shown in FIG. 3 , terminal 3 has processor 31 , ROM 32 , RAM 33 , data memory 34 , communication interface (I/F) 35 , microphone 36 , speaker 37 and battery 38 .

The processor 31 controls the terminal 3 as a whole. Also, the processor 31 is, for example, a CPU. Processor 31 uses RAM 33 to implement various processes based on programs and control data stored in ROM 32 or data memory 34 . For example, the processor 31 performs various processes such as speech recognition processing and response processing according to the speech recognition result by executing a program.
It should be noted that some of the various functions realized by the processor 31, which will be described later, may be realized by hardware.

The ROM 32 is a non-volatile memory that stores control programs and control data in advance. The ROM 32 is incorporated in the terminal 3 in a state in which a control program, control data, and the like are stored at the manufacturing stage.
RAM 33 is a volatile memory. RAM 33 functions as a working memory. The RAM 33 temporarily stores data being processed by the processor 31, for example. The RAM 33 stores application programs based on instructions from the processor 31 . Also, the RAM 33 may store data necessary for executing the application program, execution results of the application program, and the like.

The data memory 34 is composed of a non-volatile memory in which data can be written and rewritten. For example, the data memory 34 is composed of an HDD (hard disk drive), SSD (solid state drive), EEPROM, flash ROM, or the like. The data memory 34 stores applications according to the operational use of the terminal 3 and various data.

A communication interface (I/F) 35 is an interface for communicating with the sound detection device 2 . The communication I/F 35 may communicate with the terminal 3 wirelessly or may communicate with the terminal 3 by wire. The processor 31 is activated in response to an instruction received from the voice detection device 2 via the communication I/F 35 when the terminal 3 is in sleep mode.
Also, the communication I/F 35 may perform communication for connecting to an external network. Thereby, the processor 31 may perform response processing for the voice recognition result based on information acquired from an external network via the communication I/F 35 .

A microphone 36 inputs a voice uttered by a person. The microphone 36 does not function when the terminal 3 is in a sleep state, and accepts voice input after the terminal 3 is activated. In other words, the microphone 36 is controlled so that the power supply is turned off and the power consumption is suppressed when in the sleep state.

The speaker 37 outputs sound. A speaker 37 outputs a response message or the like to the person. The speaker 37 outputs, for example, a response message instructed by the voice detection device 2 that has detected the wake word. The speaker 37 also outputs a response message or the like in response to the recognition result of the voice input by the microphone 36 .
A battery 38 holds power and supplies power to each part. In this embodiment, the terminal 3 is assumed to be an electronic device that operates on power held by the battery 38 .

Next, processing functions of the audio detection device 2 according to the embodiment will be described.
FIG. 4 is a diagram showing processing functions realized by the processing unit 20 of the audio detection device 2 according to the embodiment.
The processing unit 20 of the voice detection device 2 realizes various processing units (processing functions) by the processor 25 executing programs. In the example shown in FIG. 4, the processing unit 20 has processing functions such as an audio detection unit 41, an utterance detection unit 42, a sound source localization unit 43, an utterance processing unit 44, a terminal allocation unit 45, and the like. As the voice detection unit 41 , the processor 25 executes voice detection processing for detecting sound input to the microphone M. FIG. As the speech detection unit 42, the processor 25 executes speech detection processing for detecting whether the detected sound is a wake word uttered by a person.

As the sound source localization unit 43, the processor 25 executes sound source localization processing for detecting the position of the sound source of the voice detected as the wake word (the position of the person who issued the wake word). As the sound source localization processing, a sound source such as a wake word is specified based on the difference in sounds picked up by the multiple microphones M. FIG. The sound source localization processing will be described later in detail. As the utterance processing unit 44 , the processor 25 determines a voice response message to be sent from the terminal 3 . As the terminal allocation unit 45, the processor 25 identifies the terminal 3 that responds to the person who issued the wake word based on the position of the person.

Next, sound source localization processing in the sound detection device 2 will be described.
FIG. 5 is a diagram showing an example of installation positions of the plurality of microphones M included in the audio detection device 2 and areas divided by the installation positions of the microphones M. As shown in FIG.
When a plurality of microphones M are installed at different positions, the area is divided into a plurality of areas based on the installation position of each microphone. In the example shown in FIG. 5, the voice detection device 2 has six microphones M1 to M6, and six terminals 3a to 3f are arranged in the voice processing system as a whole. Since a person moves on a flat floor, here, the arrangement position of each terminal 3 is specified by a position on the plane.

In the example shown in FIG. 5, the six microphones M1 to M6 are arranged on an elliptical line centered on the origin O on the plane on which the position of the terminal 3 is specified. For example, as shown in FIG. 5, six microphones M are evenly arranged in an ellipse on the plane. In this case, each microphone M is arranged at a position every 60 degrees with respect to the starting point O. FIG. For example, the microphone M1 is arranged at about 30 degrees, the microphone M2 at about 90 degrees, the microphone M3 at about 150 degrees, the microphone M4 at about 210 degrees, the microphone M5 at about 270 degrees, and the microphone M6 at about 330 degrees.

The processor 25 identifies the position of the sound source (or the direction from the starting point O) based on the difference in sounds acquired by the multiple microphones M. Various techniques are conceivable as a method (sound source localization processing) for specifying a sound source based on sounds acquired by a plurality of microphones. For example, from the perspective of cognitive psychology, sound source localization processing that identifies the position of a sound source based on volume difference, time difference, phase difference, or timbre difference can be applied. Also, from the viewpoint of signal processing, sound source localization processing that specifies the position of the sound source by delay time estimation, phase alignment (beamforming), or eigenvalue decomposition of a spatial interphase matrix can be applied. With either method, the position of the sound source is specified by the difference in the sounds picked up by the multiple microphones M. FIG. Therefore, three or more microphones at different positions are required to uniquely determine the position of the sound source on the plane.

FIG. 6 is a diagram showing six regions identified as the position of the sound source by differences in sounds picked up by three microphones.
In the sound source localization based on the phase difference and the time difference, the position of the sound source cannot be specified because the phase difference and the time difference cannot be calculated when there is only one microphone that acquires the sound. Also, when there are only two microphones that acquire sound, two points on the plane can be identified from the difference (phase difference, time difference) between the sounds acquired by the two microphones. Two points on the plane (for example, points A and B) specified from the difference between the sounds acquired by the two microphones are symmetrical with respect to the line connecting the two microphones. In this case, it is not possible to determine which of the two symmetrical points the sound source is located at (either point A or point B). Therefore, in order to specify one point on the plane as the position of the sound source, it is necessary to have three or more microphones that acquire the specific sound.

As shown in FIG. 6, three microphones at different positions form a triangle formed by three straight lines (axis A, axis B, and axis C) connecting the positions of the two microphones. These three axes are what separate the regions into six. In the example shown in FIG. 6, it is not possible to distinguish whether the position of the sound source is in the region R1 or in the region R6 based on the symmetry of the axis A only with the microphones m1 and m2 forming the axis A. In the example shown in FIG. In this case, if the microphone m3 forming the axis B or the axis C is used, it is possible to distinguish whether the position of the sound source is in the region R1 or in the region R6. Similarly, with only the microphones m2 and m3 forming the axis B, it is impossible to distinguish whether the position of the sound source is in the area R2 or in the area R3 due to the symmetry of the axis B. In this case, if the microphone m1 forming the axis C or the axis A is used, it is possible to distinguish whether the position of the sound source is in the area R2 or in the area R3.
As described above, when the position of a sound source on a plane is specified based on the difference in sounds acquired by multiple microphones M, the sound detection device 2 requires three or more microphones arranged at different positions.

Next, the process of specifying the terminal 3 based on the position (direction) of the sound source by the sound detection device 2 will be described.
Based on the position of the person (sound source) who issued the wayword, the voice detection device 2 identifies the terminal 3 that responds to the person. The voice detection device 2 refers to the terminal position database for the information indicating the position of each terminal 3, and specifies the terminal 3 that performs the response process to the person.

FIG. 7 is a diagram showing a configuration example of a terminal location database (DB) 71 indicating the locations of a plurality of terminals 3. As shown in FIG.
The terminal position DB 71 is stored in the data memory 23, for example. Also, the terminal position DB 71 may be stored in an external device such as a server device with which the voice detection device 2 can communicate. The voice detection device 2 identifies the terminal 3 that performs response processing based on the position of the sound source identified by the sounds (wake words) acquired by the multiple microphones M. FIG.

In the example shown in FIG. 7, information indicating the name, direction (position), and ID of each terminal 3 is stored in the terminal position DB 71 . In the example shown in FIG. 7 , the direction indicates the direction from the voice detection device 2 to the position of the terminal 3 . However, the terminal position DB 71 may indicate the position of the terminal by position information such as coordinates. The ID is identification information for uniquely identifying the terminal 3 . The ID may be information indicating the communication destination such as the IP address of the terminal 3 .

When referring to the terminal position DB 71 shown in FIG. 7, the voice detection device 2 determines the direction to the position of the sound source, and specifies the terminal 3 that is closest to the direction of the sound source. Further, when the position of the terminal 3 is indicated by position information such as coordinates in the terminal position DB 71, the voice detection device 2 identifies the terminal closest to the identified sound source.

After specifying the terminal 3 from the position of the wake word sound source, the voice detection device 2 sends an operation instruction to the terminal 3 to perform a response process to the person who issued the wake word. Each terminal 3 is in a sleep state in a standby state. Therefore, the voice detection device 2 activates the terminal 3 and instructs the terminal 3 to perform response processing to the person who issued the wake word.

FIG. 8 is a diagram showing an example of an operation instruction that the voice detection device 2 transmits to the terminal 3. As shown in FIG.
In the example shown in FIG. 8, the voice detection device 2 designates a response message to the terminal 3 and instructs (instructs) the start of voice capture and the start of voice recognition. When the operation instruction shown in FIG. 8 is received, the terminal 3 uses the speaker 37 to output a response message "What do you need?" In addition, the terminal 3 supplies power to the microphone 36 to start capturing voice, and activates processing such as voice recognition processing. As a result, the terminal 3 that has received the operation instruction can execute processing such as voice response to the person who issued the wake word.

Next, processing by the audio detection device 2 according to the embodiment will be described.
FIG. 9 is a flowchart for explaining the flow of processing by the audio detection device 2 according to the embodiment.
In the sound detection device 2, the processor 25 of the processing unit 20 acquires the sound input by each microphone M via the sound input I/F 21 (ACT 11). The processor 25 monitors the audio data input from the audio input I/F 21 and determines whether or not the audio uttered by a person has been detected (ACT 12). If speech is detected (ACT 12, YES), processor 25 determines whether the acquired speech is the wake word (ACT 13).

If it is determined that the wake word has been input (ACT13, YES), the processor 25 identifies the position (direction) of the sound source (the person who issued the wake word) that issued the detected wake word (ACT14). The processor 25 determines the position of the sound source based on the difference (phase difference, etc.) between the sounds of the wake word acquired by each microphone M, as described above.

After specifying the position of the sound source, the processor 25 specifies the terminal 3 that responds to the person according to the specified position of the sound source (ACT 15). For example, the processor 25 identifies the terminal 3 closest to the position of the wake word sound source based on the terminal position DB. The terminal 3 closest to the position of the sound source utters a response message to the wake word, and performs recognition processing and the like on the voice uttered by the person.

That is, when the terminal 3 is identified based on the position of the sound source, the processor 25 issues a response message to the identified terminal 3 and outputs an instruction to start speech recognition processing (ACT 16). Here, the processor 25 designates the terminal 3 as a destination, and sends an operation instruction including a response message to be output, start of voice capture, start of voice recognition, and the like. For example, the action instruction has contents as shown in FIG.

The processor 31 of the terminal 3 that has received the operation instruction from the voice detection device 2 wakes up from the sleep state and emits a response message instructed by the speaker 37 . Further, the processor 31 of the terminal 3 supplies power to the microphone 36 to start capturing voice and activate processing such as voice recognition processing. As a result, of the plurality of terminals 3, the terminal 3 that is in the vicinity of the person who issued the wake word is activated and executes processing such as voice response to the person.

As described above, the voice detection device according to the embodiment acquires the voice uttered by a person with a plurality of microphones, identifies the position of the sound source from the difference in the voices acquired by the plurality of microphones, and identifies the position of the sound source and the location of each terminal. A single terminal identified based on location is instructed to operate.

As a result, even if a plurality of terminals are on standby in a sleep state, it is possible to activate the terminal closer to the position of the sound source when the voice detection device detects the wake word. As a result, each terminal can reduce power consumption in a standby state, and even a terminal driven by a battery can be operated for a long time.

It should be noted that in the above-described embodiment, the case where the program to be executed by the processor is pre-stored in the memory within the apparatus has been described. However, the program executed by the processor may be downloaded to the device from a network, or may be installed to the device from a storage medium. Any storage medium such as a CD-ROM can be used as long as it can store the program and can be read by the device. Also, functions obtained by installing or downloading in advance may be implemented in cooperation with an OS (Operating System) or the like inside the apparatus.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.
The content described in the scope of claims as originally filed for this application will be added below.
[1]
multiple microphones to capture sound; and
a memory that stores information indicating the positions of a plurality of terminals;
The position of the sound source of the sound is specified from the difference between the sounds acquired by the plurality of microphones, and an operation is instructed to one terminal specified based on the position of the sound source and the positions of the plurality of terminals stored in the memory. a processor;
A voice detection device having
[2]
The plurality of microphones is three or more,
The audio detection device according to [1].
[3]
The position of the sound source is identified by the phase difference between the sounds picked up by multiple microphones.
The audio detection device according to any one of [1] and [2].
[4]
When the processor detects a predetermined word uttered as a voice by a person from the sound acquired by any of the microphones, the processor identifies the sound source from which the voice of the predetermined word is uttered.
The audio detection device according to any one of [1] to [3].
[5]
The processor instructs the identified terminal to start operation and instructs a response message corresponding to the predetermined word.
The audio detection device according to [4].

2 Audio detection device 3 (3a, 3b, ...) Terminal M (M1, M2, M3, ...) Microphone 20 Processing unit 21 Audio input interface 22 Buffer memory 23 Data memory , 24... terminal interface, 71... terminal location database.

Claims (4)

multiple microphones to capture sound; and
a memory that stores information indicating the positions of a plurality of terminals;
The position of the sound source of the sound is specified based on the phase difference of the sounds acquired by the plurality of microphones, and an operation is instructed to one terminal specified based on the position of the sound source and the positions of the plurality of terminals stored in the memory. a processor that
A voice detection device having The plurality of microphones is three or more,
The audio detection device according to claim 1. When the processor detects a predetermined word uttered as a voice by a person from the sound acquired by any of the microphones, the processor identifies the sound source from which the voice of the predetermined word is uttered.
3. The sound detection device according to claim 1 or 2 . The processor instructs the identified terminal to start operation and instructs a response message corresponding to the predetermined word.
The audio detection device according to claim 3 .