JP7143579B2 - voice input device - Google Patents

Description

The present invention relates to a voice input device suitable for voice input in a mobile phone or the like.

As a technical means for making a person who does not have vocal cords produce sound, there is an electric artificial larynx (see, for example, Patent Document 1). This electric artificial larynx is a device that vibrates the human throat and creates air vibrations in the oral cavity that are similar to air vibrations caused by the vibration of the vocal cords. The user can produce sounds from the mouth by vibrating the throat with the electric artificial larynx and changing the shape of the mouth. People who do not have vocal cords can use this electrolarynx to have normal conversations as well as make calls using telephones.

WO 1999/12501

By the way, in a telephone, an audio signal picked up by a microphone is transmitted as it is. Therefore, if the picked-up audio signal is difficult to hear, the audio signal heard by the other party is also difficult to hear. As an example of a situation in which such a situation occurs, there is a case in which a person speaks on a mobile phone in a low voice that cannot be heard by people around him or her on a train. Here, when the user of the mobile phone speaks in a low voice, the voice signal obtained by the microphone of the mobile phone has an extremely low level, and the voice signal waveform is similar to that of the voice signal at normal volume. distorted in comparison. Therefore, even if the voice signal obtained by the microphone is sent to the called party, it is difficult for the called party to recognize what the mobile phone user is saying. This is a problem that occurs both when a person without vocal cords speaks in a low voice using an electrolarynx and when a healthy person speaks in a low voice.

SUMMARY OF THE INVENTION The present invention has been made in view of the circumstances described above, and aims to provide a technical means capable of appropriately transmitting the pronounced voice even when the voice is uttered in a low voice. aim.

The present invention comprises a voice acquiring means for acquiring a voice signal representing a voice pronounced from the user's mouth in response to vibration applied to the user's throat, and voice conversion processing for facilitating recognition of the voice signal. A voice input device characterized by comprising a voice conversion means for applying a voice.

According to the present invention, even if the volume of the audio signal acquired by the audio acquisition means is too low to be heard by surrounding people, the audio signal is converted into easily recognizable information by the audio conversion means. be done. Therefore, even when the voice is uttered in a low voice, the uttered voice can be properly transmitted.

1 is a block diagram showing the configuration of a mobile phone including a voice input device that is one embodiment of the present invention; FIG. It is a block diagram which shows the structure of the audio|voice conversion part in the same embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a mobile phone 1000 including a voice input device 100 that is one embodiment of the present invention. In addition to the voice input device 100, FIG. 1 shows a transmission unit 201 and a reception unit 202 of the mobile phone 1000, and the head 10 of the user of the mobile phone 1000. As shown in FIG. The mobile phone 1000 has various devices such as an antenna, which are the same as those of a normal mobile phone, but the illustration thereof is omitted in FIG.

In the voice input device 100 , the sound insulation hood 132 and the actuator 131 housed therein are fixed to the neck of the user by a belt 133 . Here, the actuator 131 functions as means for bringing its vibration surface into contact with the user's throat and giving vibration to the user's throat. The sound insulation hood 132 is means for shielding the vibration sound of the actuator 131 from leaking to the surroundings.

The tilt sensor 151 is an ear-mounted sensor, worn on the ear of the user of the mobile phone 1000, and detects the tilt of the user's head.

The pronunciation start button 134 is a push button for instructing the start of pronunciation. The user of the mobile phone 1000 holds the operator provided with the sounding start button 134 and turns on the sounding start button 134 to start sounding. The sound generation start button 134 may be a button switch that is ON only while it is pressed, or may be a button switch that reverses from OFF to ON and from ON to OFF every time it is pressed.

The control unit 110 functions as a control center for the voice input device 100 and also functions as a control center for the mobile phone 1000 as a whole.

The operation display unit 120 is, for example, a touch panel, and is a device that displays various types of information to the user and receives user operations.

The drive control unit 130 is a drive control means that provides a periodic drive pulse waveform to the actuator 131 under the control of the control unit 110, thereby generating vibrations applied to the user's throat. More specifically, control unit 110 causes drive control unit 130 to start outputting a drive pulse waveform when sound generation start button 134 is turned on.

While the actuator 131 is vibrating the throat, the user can open and close the mouth while changing the shape of the oral cavity 1 to produce a desired sound. The microphone 140 is a means for picking up the voice of the user, as is the case with ordinary mobile phones. function as voice acquisition means for acquiring a voice signal representing a voice pronounced from the user's mouth.

In this embodiment, it is possible to control the volume of pronunciation when using the actuator 131 . More specifically, in this embodiment, the user can set the pulse width of the drive pulse waveform given to the actuator 131 by the drive control section 130 by operating the operation display section 120 . Here, when the pulse width of the drive pulse waveform is lengthened, the volume of the voice pronounced from the user's mouth increases. Also, when the pulse width of the driving pulse waveform is shortened, the volume of the voice produced by the user's mouth is reduced. If the pulse width of the drive pulse waveform is made shorter than a predetermined length, the volume of the voice pronounced from the user's mouth can be set to a low volume that is not audible to surrounding people. Normally, when an electrolarynx is used for conversation, the electrolarynx is driven with a large amplitude in order to ensure the volume of the spoken voice. However, when making a call using the mobile phone 1000 in a train, the actuator 131 is driven with a small amplitude to make the volume of the voice pronounced from the user's mouth low enough to be heard by people around him/her.

In addition, the voice input device 100 according to the present embodiment provides an intonation, that is, a , has the function of giving a change in pitch.

More specifically, the output signal of the tilt sensor 151 is supplied to the controller 110 . Based on the output signal of the tilt sensor 151, the control unit 110 generates intonation information indicating the pitch conversion ratio of the voice pronounced by the user. This pitch conversion ratio is the ratio of the pitch of speech after pitch conversion to the pitch before pitch conversion.

Here, it is assumed that the inclination angle when the front surface of the user's head faces the front is 0°. When the front surface of the head faces upward, the tilt angle changes from 0° to the positive direction, and when the front surface of the head faces downward, the tilt angle changes from 0° to the negative direction.

In this case, when the tilt angle changes from 0° to the positive direction, the control unit 110 increases the pitch conversion ratio indicated by the intonation information from 1, and when the tilt angle changes from 0° to the negative direction, the control unit 110 increases the intonation information Decrease from 1 the pitch conversion ratio indicated by .

The drive control section 130 controls the cycle of the drive pulse waveform given to the actuator 131 based on the intonation information output by the control section 110 . When the pitch conversion ratio indicated by the intonation information is 1, the drive control unit 130 gives the actuator 131 a drive pulse waveform with a standard cycle. As a result, a standard pitched voice is produced from the user's mouth. When the pitch conversion ratio indicated by the intonation information increases from 1, the drive control section 130 shortens the cycle of the drive pulse waveform given to the actuator 131 according to the increase in the pitch conversion ratio. As a result, the pitch of the voice pronounced from the user's mouth rises from the standard pitch. Also, when the pitch conversion ratio indicated by the intonation information decreases from 1, the drive control unit 130 lengthens the period of the drive pulse waveform given to the actuator 131 according to the decrease in the pitch conversion ratio. As a result, the pitch of the voice pronounced from the user's mouth is lowered from the standard pitch.

The voice conversion unit 150 is means for performing voice conversion processing to facilitate recognition of the voice signal output from the microphone 140 . Regarding this voice conversion processing, three types of modes are prepared: a voice/character conversion mode, a voice/voice conversion mode, and a voice processing mode. Here, the voice/text conversion mode is a mode in which voice recognition processing is performed on the voice signal output from the microphone 140 and text information is output. The speech/speech conversion mode is a mode in which speech recognition processing is performed on the speech signal output from the microphone 140, and speech signals are synthesized based on character information obtained as a result. In the voice processing mode, voice recognition processing is performed on the voice signal output from the microphone 140, and based on the result of the voice recognition processing, the voice signal output from the microphone 140 is processed into an easily recognizable voice signal. mode. In this embodiment, the user of the mobile phone 1000 can select a desired mode from among three modes by operating the operation display section 120 and cause the voice conversion section 150 to execute the selected mode.

FIG. 2 is a block diagram showing the configuration of the audio conversion section 150. As shown in FIG. As shown in FIG. 2 , the speech conversion section 150 has a speech recognition section 151 , a speech synthesis section 152 , a speech processing section 153 and a switch 154 .

The voice recognition unit 151 is means for recognizing voice signals output from the microphone 140 and outputting character information. Here, when the pulse width of the driving pulse waveform given to the actuator 131 is set to be short and the volume of the sound produced by the user's mouth is low enough to be heard by people around him/her, the sound output from the microphone 140 The level of the vowel intervals of the signal is very low, and the level of the consonant intervals even lower. In this case, recognition of vowels in the speech signal is possible, but recognition of consonants is difficult.

Therefore, the speech recognition unit 151 according to the present embodiment determines word breaks in the speech signal based on the intonation (pitch change) of the speech signal, and executes speech recognition. Specifically, the speech recognition unit 151 stores a database of pitch change patterns of speech measured for various words used in conversation. Then, when a pitch change pattern matching any of the pitch change patterns in the database appears in the speech signal output from the microphone 140, the speech recognition unit 151 recognizes the section of the speech signal corresponding to the pitch change pattern. Treat it as a single word and perform speech recognition.

In addition, the speech recognition unit 151 in the present embodiment, while executing the speech recognition process, refers to the speech recognition results obtained up to the present time, and determines the length of the section in which the recognition is not completed (the section in which the type of unknown phoneme exists). ) to perform speech recognition. For example, in speech recognition processing, assume that the type of consonant in a certain section of the speech signal is unknown. In this case, in the speech recognition process, the type of the consonant is estimated from the context indicated by the characters that are the results of the speech recognition process in the sections before and after the section. Specifically, several consonant candidates are selected based on the speech signal in the section in which the type of consonant is unknown. Then, one consonant is selected from among consonant candidates and applied to the section, and it is determined whether or not the characters in the section consisting of the section and the sections before and after it form a meaningful sentence. Then, consonants that form meaningful sentences are selected as speech recognition results.

The speech synthesis unit 152 is means for synthesizing a speech signal based on the character information output from the speech recognition unit 151 . Specifically, the speech synthesis unit 152 stores a database of speech waveforms of various speech units such as consonants and vowels, and reads speech waveforms of speech units of consonants and vowels indicated by character information from the database. , on the time axis to synthesize an audio signal.

In a preferred embodiment, the speech synthesizing unit 152 stores a database of speech waveforms of speech segments corresponding to various kinds of speech such as husky male speech and highly transparent female speech. The user can select a desired type of database by operating the operation display unit 120 and use it for speech synthesis.

The speech synthesizing unit 152 in this embodiment has means for controlling the pitch of the synthesized speech signal based on the intonation information. Here, the intonation information is information generated by the control unit 110 based on the output signal of the tilt sensor 151, and indicates changes in the pitch of the voice pronounced by the user's mouth. Therefore, the speech signal output from the speech synthesizing unit 152 is a speech signal having a pitch change similar to that of the voice uttered from the user's mouth.

The voice processing unit 153 is means for processing the voice signal output from the microphone 140 into an easily recognizable voice signal based on the voice recognition processing result of the voice recognition unit 151 . As described above, when the user uses the actuator 131 to make a sound at a low volume that cannot be heard by people around him or her, the volume of the audio signal output from the microphone 140 is extremely low, especially in the consonant section, and is difficult to recognize. be. Therefore, in this embodiment, a database of audio signal waveforms representing various consonants is stored in advance in the audio processing unit 153 . Then, the voice processing unit 153 converts the voice signal waveform of the section determined as a consonant by the voice recognition processing of the voice recognition unit 151 into an appropriate voice signal corresponding to the consonant in the database, in the voice signal output from the microphone 140 . Replace with waveform. Further, the voice processing unit 153 amplifies the voice signal after replacing the voice signal waveform of the consonant to an appropriate level for easy listening, and outputs the amplified voice signal.

The switch 154 selects any one of the character information output by the voice recognition unit 151, the voice signal output by the voice synthesis unit 152, and the voice signal output by the voice processing unit 153, and outputs it to the transmission unit 201 shown in FIG. It is a means. When the voice conversion section 150 is set to the voice/text conversion mode, the voice recognition section 151 is activated, and the character information output by the voice recognition section 151 is supplied to the transmission section 201 via the switch 154 . When the speech conversion unit 150 is set to the speech/speech conversion mode, the speech recognition unit 151 and the speech synthesis unit 152 are activated, and the speech signal output by the speech synthesis unit 152 is supplied to the transmission unit 201 via the switch 154. be done. When the voice conversion unit 150 is set to the voice processing mode, the voice recognition unit 151 and the voice processing unit 153 are activated, and the voice signal output by the voice processing unit 153 is supplied to the transmission unit 201 via the switch 154. .

In FIG. 1, a transmission unit 201 is means for transmitting a voice signal or character information to a call partner. When the voice conversion unit 150 is set to the voice/text conversion mode, the transmission unit 201 transmits the text information output from the voice conversion unit 150 to the phone of the other party of the call of the user of the mobile phone 1000 . When the voice conversion unit 150 is set to the voice/voice conversion mode or the voice processing mode, the transmission unit 201 transmits the voice signal output from the voice conversion unit 150 to the phone of the other party of the call of the user of the mobile phone 1000. do. Mobile phone 1000 negotiates with the other party's phone when starting a call with the other party's phone. At that time, mobile phone 1000 transmits information indicating whether the transmission from transmission unit 201 is text information or a voice signal to the phone of the other party. As a result, the other party's telephone can respond by displaying text information when it is sent, and emitting a sound when voice signals are sent.

The receiving unit 202 is means for receiving a voice signal from the telephone of the other party. The audio signal received by the receiver 202 is sent to the speaker 170 via the adder 163 and emitted by the speaker 170 .

In this embodiment, the user of the mobile phone 1000 can monitor the processing result of the voice conversion section 150 by operating the operation display section 120 . For example, when the voice/text conversion mode is set, the user can turn on the switch 162 by operating the operation display unit 120 . As a result, the character information output by the speech recognition section 151 of the speech conversion section 150 is sent to the operation display section 120 via the switch 162 and displayed on the operation display section 120 . In addition, the user can turn on the switch 161 by operating the operation display unit 120 when the voice/voice conversion mode or the voice processing mode is set. As a result, the audio signal output from the audio synthesizing section 152 or the audio processing section 153 of the audio converting section 150 is sent to the speaker 170 via the switch 161 and the adder 163 and emitted from the speaker 170 .
The details of the mobile phone 1000 according to the present embodiment have been described above.

In this embodiment, the user of the mobile phone 1000 sets the pulse width of the drive pulse waveform applied to the actuator 131 to the lowest value by operating the operation display unit 120 when making a call using the mobile phone 1000 on a train, for example. . Then, the user turns on the pronunciation start button 134 to change the shape of the oral cavity 1 . As a result, a voice is pronounced from the user's mouth at a volume that is not audible to surrounding people, and an audio signal representing this voice is output from the microphone 140 . At that time, the user can change the intonation information by tilting the head, and can give intonation, that is, change in pitch to the voice pronounced from the user's mouth.

When the voice/text conversion mode is set, the voice recognition unit 151 of the voice conversion unit 150 performs voice recognition on the voice signal output from the microphone 140 and outputs text information. Here, the level of the audio signal output from the microphone 140 is low, and recognition of consonants is particularly difficult. Therefore, the speech recognition unit 151 determines the division of words in the speech signal based on the pitch change appearing in the speech signal, and estimates the consonant based on the context indicated by the characters before and after the consonant in the speech recognition process. generates character information from the audio signal. The character information output from the speech recognition unit 151 is sent to the telephone of the other party by the transmission unit 201 and displayed by the telephone.

When the voice/voice conversion mode is set, the voice recognition unit 151 of the voice conversion unit 150 performs voice recognition processing on the voice signal output from the microphone 140, and outputs character information. Then, the speech synthesizing unit 152 synthesizes a speech signal from this character information, controls the pitch of the speech signal based on the intonation information, and outputs it. Then, the voice signal output from the voice synthesizing section 152 is sent by the transmitting section 201 to the other party's telephone, and the sound is emitted from the speaker of the telephone.

Also, when the voice processing mode is set, the voice recognition unit 151 of the voice conversion unit 150 performs voice recognition processing on the voice signal output from the microphone 140 . Then, based on the result of the speech recognition processing, the speech processing unit 153 replaces the speech signal waveform of the consonant period in the speech signal output from the microphone 140 with an appropriate speech signal waveform of the consonant, and converts the speech after the replacement into Amplifies the signal to an appropriate level and outputs it. Then, the audio signal output from the audio processing unit 153 is sent to the telephone of the other party by the transmission unit 201, and the sound is emitted from the speaker of the telephone.

As described above, according to the present embodiment, the user of the mobile phone 1000 makes use of the vibration of the actuator 131 to produce a sound at a low volume that is not audible to people around him or her. Voice can be input to the input device 100 . Even when the voice is input in a low voice in this way, in the voice input device 100, the voice conversion unit 150 performs voice conversion processing to facilitate recognition of the voice signal obtained by the microphone 140, and transmits the voice signal. The unit 201 transmits to the other party of the call. Therefore, even if the user speaks in a low voice, the user can appropriately convey the desired information to the other party.

Further, in this embodiment, in the voice/text conversion mode, the voice signal obtained by the microphone 140 can be converted into text information and transmitted to the other party of the call. Therefore, the user can accurately transmit the desired information to the other party.

Further, in the present embodiment, in the speech/speech conversion mode, the speech signal obtained by the microphone 140 can be converted into character information, and the speech signal can be synthesized from this character information and transmitted to the other party. Therefore, the user can transmit desired information to the other party of the call in a form similar to that of a normal call.

In addition, in the present embodiment, in the voice processing mode, only the voice signal waveform of the consonant section of the voice signal obtained by the microphone 140 can be replaced with the voice signal waveform of an appropriate consonant and transmitted to the other party. Therefore, the user can transmit a voice close to the voice uttered by the user to the other party of the call. This is suitable for users who want to deliver their own voice to the other party.

Also, the voice input device 100 according to the present embodiment can be used by a healthy person with vocal cords. Therefore, both a person without vocal cords and an able-bodied person with vocal cords use the mobile phone 1000 having the voice input device 100 to talk in a low voice that is not heard by people around them. In this way, a person without vocal cords can make a call in the same manner as a healthy person with vocal cords, so that the person without vocal cords can be motivated to make a call using the mobile phone 1000 .

<Other embodiments>
Although the embodiments of the present invention have been described above, other embodiments of the present invention are conceivable. For example:

(1) In the above embodiment, the period of the drive pulse waveform given to the actuator 131 is controlled by the intonation information obtained from the output signal of the tilt sensor 151, but this period may not be controlled. In this case, the speech recognition unit 151 may determine word breaks in the speech signal output from the microphone 140 based on the pitch change indicated by the intonation information.

(2) In the above embodiment, the intonation information is generated by the tilt sensor 151 worn on the user's head. However, the intonation information may be generated by a sensor other than the tilt sensor. For example, an acceleration sensor or the like may be attached to a part of the user's body, and the intonation information may be generated by this sensor. Also, the sensor for generating intonation information may be worn on a part other than the user's head. Also, the intonation information may be generated by having the user operate an operator such as a slide switch.

(3) In the speech/text conversion mode of the above embodiment, the intonation information may be transmitted to the other party's phone in synchronization with the text information output by the speech recognition section 151 . In this case, the other party's telephone synthesizes a voice signal from the received character information, and controls the pitch of the voice signal obtained from this character information based on the intonation information received in synchronization with the character information. may

(4) In the speech/speech conversion mode of the above embodiment, instead of using the intonation information obtained from the output signal of the tilt sensor 151, the pitch ratio of the pitch of the speech signal output from the microphone 140 to the standard pitch is calculated. It is also possible to generate intonation information indicating the pitch, and control the pitch of the speech signal synthesized by the speech synthesizing unit 152 based on this intonation information.

(5) An operator may be provided to indicate the pronunciation timing of the beginning of a word, and the speech recognition unit 151 may detect the timing of the beginning of a word in the voice signal based on the operation of this operator and perform voice recognition. Alternatively, the sounding start button 134 in the above embodiment may be used as an operator for instructing the sounding timing of the beginning of the word.

(6) In the speech/speech conversion mode, a speech signal is synthesized from character information obtained by speech recognition processing, but a speech signal may be synthesized from results obtained in the process of speech recognition processing, such as formant information. .

(7) In the voice processing mode, the voice signal may be processed by simple waveform conversion processing without using voice recognition processing. For example, a consonant segment in the voice signal is detected based on the level and frequency of the voice signal obtained from the microphone 140, and level conversion processing and consonant emphasis processing are performed only for the detected consonant segment. By performing such audio processing, the audio signal can be brought closer to the audio signal of normal audio.

(8) The voice input device 100 transmits the voice signal obtained from the microphone 140 to the server via the network, causes the server to execute the processing of the voice conversion unit 150, and transmits the voice signal and character information resulting from the execution from the server. It may be received and transmitted to the other party by the transmission unit 201 . According to this aspect, since there is no need to provide the voice converting section 150 in the mobile phone 1000, an increase in the cost of the mobile phone 1000 can be avoided. In order to achieve the object of the present invention, there are provided a voice acquisition process for acquiring a voice signal representing a voice pronounced from the user's mouth in response to vibration applied to the user's throat, and recognition of the voice signal. Any device may execute the voice conversion processing to facilitate the processing. It is arbitrary what kind of device the device for executing the voice acquisition process and the device for executing the voice conversion process are and how they are arranged.

(9) In the above embodiments, the voice input device according to the present invention was used for telephones, but the application of the voice input device according to the present invention is not limited to this. The voice input device according to the present invention can be applied to voice input devices in general including, for example, text input devices using voice.

DESCRIPTION OF SYMBOLS 1000... Mobile phone 201... Transmission part 202... Reception part 100... Voice input device 110... Control part 120... Operation display part 130... Drive control part 131... Actuator, 132 Sound insulation hood 133 Belt 140 Microphone 150 Voice converter 161, 163, 154 Switch 163 Adder 151 Voice recognizer 152 Voice synthesis Part, 153... Voice processing part, 10... Head, 1... Mouth.

Claims (6)

a voice acquisition means for acquiring a voice signal representing a voice pronounced from the user's mouth in response to vibration applied to the user's throat;
a sensor that detects body movement;
and audio conversion means for performing audio conversion processing on the audio signal,
The voice input device, wherein the voice converting means performs pitch control according to at least body movements detected by the sensor. 2. The voice input device according to claim 1, wherein the sensor detects changes in tilt of the head. 3. The voice input device according to claim 1, further comprising drive control means for generating vibration applied to the user's throat. The speech converting means performs speech recognition processing on the speech signal, synthesizes the speech signal based on the result of the speech recognition processing, and performs the pitch control on the synthesized speech signal. Item 4. The voice input device according to any one of items 1 to 3. The voice input device comprises an operator for manipulating the period of vibration,
5. The voice input device according to claim 4, wherein said voice converting means divides said voice signal on the time axis based on a pitch change of said voice signal, and performs said voice recognition processing. 5. The speech conversion means, while executing the speech recognition processing, refers to speech recognition results obtained up to the present time, and executes speech recognition for a section in which recognition has not been completed. 6. The voice input device according to 5.

Images