JPH0157370B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a speech input typewriter using a monosyllabic input speech recognition device.

Various speech recognition devices have been proposed, but speech recognition is classified into word recognition and monosyllable recognition depending on the target. Among these, when considering the application of monosyllable recognition to voice typewriters, since the target is monosyllables, in principle, an infinite number of languages can be recognized.

Regarding the monosyllable recognition device, refer to the Acoustical Society of Japan Speech Research Group Material No. S77-46 (December 1977 issue)
It is shown in "Monosyllabic voice typewriter (Hokkaido University Applied Electrical Research Institute)" and is roughly shown in Figure 1. In Figure 1, 1 is a microphone, 2
is a preamplifier, 3 is a 16-channel bandpass filter (hereinafter referred to as 16CH BPF), and 4 is a 16-channel analog-to-digital converter (hereinafter referred to as 16CH BPF).
5 is a read-only memory that logarithmically converts the input value (hereinafter referred to as logarithmic conversion).
6 is a pitch extractor, 7 is a counter timer, 8 is a microprocessor (hereinafter referred to as micro CPU), and 9 is a bus line (hereinafter referred to as BUS).
10 is a read-only memory (hereinafter referred to as ROM), 11 is a random access memory (hereinafter referred to as RAM), 12 is a flop disk drive device (hereinafter referred to as FDD), and 13 is a character display. Device (hereinafter referred to as CRT), 1
4 is a keyboard device (hereinafter referred to as KB), and 15 is a hardware computing unit.

The voice typewriter having the above configuration operates as follows. First, since the audio input from the microphone 1 is attenuated at 6 dB/octave, the preamplifier 2 emphasizes the high range at a rate of 6 dB/octave to flatten this power spectrum. The output is inputted to the 16CH BPF3, the frequency of 200 to 4400Hz is divided into 16 channels, and the output is inputted to the A/D converter 4 for each CH. After being converted into a 12-bit digital quantity by the A/D converter 4, it is input to the logarithmic conversion ROM 5, where it is converted into an 8-bit logarithm. This is to approximate the logarithmic nature of human hearing, and
Furthermore, this process takes into consideration the advantage that when normalizing the power spectrum, calculations can be performed using only addition and subtraction. The output of the A/D converter 4 is connected to the micro CPU 8 through the BUS LINE 9.
Pull into CPU8. The micro CPU 8 includes a ROM 10 for control, a RAM 11 for data area,
FDD12 to store data such as registered voices, CRT13 to output and display voice recognition results, micro
KB14 for manually inputting to CPU8
are connected to form a small electronic computer system.

Further, the output of the 16CH BPF 3 is input to the pitch extractor 6, and the configuration is configured to trigger the 16CHA/D converter every pitch period. One method for the pitch extractor 6 is proposed in, for example, Japanese Patent Application Laid-Open No. 54-162405 entitled "Pitch Frequency Extraction Apparatus", and it may be considered that pitch frequencies are extracted by such a method. A pitch frequency exists in the case of a vowel or a voiced consonant, but in the case of a voiceless consonant, the sound source is noise and the pitch frequency does not exist. In order to sample these unvoiced consonants, a counter timer 7 generates pulses every 1 msec at a constant frequency, for example, 1 KHz, and inputs them to the 16CH A/D converter 4.
The 16CH A/D converter 4 samples the output of the 16CH BPF 3 every pitch cycle in the case of a vowel or voiced consonant, and every fixed cycle in the case of a voiceless consonant, and performs analog-to-digital conversion to obtain the input speech waveform. Envelope characteristics of the power spectrum can be obtained.

The digitized input voice waveform obtained in the above manner is subjected to power spectrum normalization, and then the square distance between it and each data of the registered voice waveform is calculated. Calculation of this squared distance requires several thousand times, and since real-time processing cannot be performed using the micro CPU 8, a hardware calculator 15 is used to calculate the squared distance.
Calculations are performed using hardware. The squared distance is calculated between the input speech waveform and the registered speech waveform, and the monosyllable that shows the lowest value of the squared distance is recognized as the input speech.

The recognized single syllables are sequentially output, and when they are grouped into words or phrases, they are converted into kanji and kana mixed sentences with reference to a word dictionary and a grammar dictionary. This kana-kanji conversion is described in ``Information Processing Society of Japan Journal VOL 15 No.
1 (January 1974) P.2 to P.9 Kanji-kana mixed sentence conversion system (Matsushitaji), a known method may be used. Therefore, the speech recognition unit recognizes monosyllables, and when inputting, the speech is inputted by dividing it into syllables. In other words, if you are trying to input the word "warm", type "a" and "ta" one syllable at a time.
You have to pronounce "ta", "ka", and "i" and input them into the speech recognition unit.

On the other hand, Japanese monosyllables consist of 101 syllables, including voiced, semi-voiced, and persistent sounds. These structures consist of vowels a,
It consists of i, u, e, o (5 syllables), a cursive N (1 syllable), 62 syllables in the form of a consonant + vowel, and 33 syllables. Looking at the most common consonant + vowel format,
It can be said that the amount of information that can be obtained from audio waveforms is extremely small. If we talk about when people listen to audio, when they listen to what another person is saying, they first try to predict what the other person will talk about based on what the topic is about. It can be said that the content of what is being said can be understood because the person is listening. To give you a familiar example, when I picked up the phone at my desk, the operator quickly said, ``Mr. △△ of 〇〇 company is calling Mr. XX.''
In most cases, students can understand the meaning of ``〇〇company,''``△△san,'' and ``XXsan.'' This is because ``Mr. △△ of 〇〇 company'' sometimes calls me, and ``Mr. be. However, if ``Mr. △△ of 〇〇 company'' is a partner with whom we have not done business in the past, and ``Mr. △△'' is an unusual or strange name, it is difficult to understand. Furthermore, if you pick up a phone from another department rather than your desk phone, you may not even be able to understand ``Mr.

As mentioned above, it can be said that people recognize and understand speech by matching it with information in their conscious mind, rather than by listening to it. In other words, it does not understand individual monosyllables of speech input and assemble them. Incidentally, when I conducted a monosyllable recognition experiment on people, I found mistakes in mishearing between G, D, and B lines, between PA, C, and T lines, and between M and N lines. It cannot be used as a monosyllable recognizer with a 100% recognition rate for many people.

It is almost impossible for a machine to 100% recognize monosyllables that humans cannot understand even when they hear them.
It had a serious and essential drawback that the recognition rate was limited to 95%. Furthermore, since the input voice is uttered by a person, the state of the voice changes depending on the uttered sound, that is, the health condition and emotional state of the speaker. In conventional speech recognition devices, the error between the registered speech waveform and the input speech waveform is calculated, and those other than a certain threshold are rejected and re-inputted by the speaker.For example, even if the error is less than the threshold, the error is Methods such as taking the smaller one as the recognition result have been used, but all of them have the disadvantage that they depend only on the recognition rate of the speech recognition unit.

The purpose of the present invention is to eliminate the above-mentioned drawbacks in monosyllabic input voice typewriters, by focusing on the fact that the input voice is a word or structure that can be recognized when a person hears it, and the recognition result is not a word or structure that cannot be recognized as a word or structure. If it is a word or sentence, it is not recognized as a recognition result, but some monosyllables are replaced with a second candidate monosyllable output from the speech recognition unit, thereby creating a word or syntax that can be recognized by humans. The purpose is to increase the speech recognition rate and will be explained in detail below.

FIG. 2 is a block diagram showing an embodiment of the present invention, in which 16 is a speech recognition unit that outputs two recognized syllables and their respective probabilities for one input syllable;
17 is a recognition result register for storing recognition results;
8 is a word syntax understanding processing section, 19 is a word/grammar dictionary, 20 is an output device, and 21 is a retry processing section.

Next, its operation will be explained.

First, the speech is input from the microphone 1 into the speech recognition section 16 by dividing it into monosyllables in phrase units. For example, when inputting the message ``A warm spring has arrived,'' first change the first phrase ``warm'' by typing ``a'' and ``ta.''
Enter "ta", "ka", and "i", separated by single syllables.
As shown in FIG. 3, the speech recognition unit 16 first performs vowel recognition and determines that it is a line, for example, and then inputs the syllable and accuracy register 2 shown in FIG.
All characters in the A line are stored in the syllable register 22a of 2, and are stored in the registered pattern memory 23 shown in FIG.
"A" stored from address 1000 to address 12FF
Registration pattern (16 channel digital code)
23a is transferred to the comparison register 23b, and address F002, which is the first address for storing the accuracy, is set in the accuracy storage address register 23c. Next, the contents of the comparison register 23b and the contents of the input voice register 23d are compared, and the square distance, that is, the error of the input voice waveform with respect to the registered voice waveform, is calculated with the accuracy of 100 when the error is 0, and this accuracy is calculated for the syllable and the accuracy. The beginning of the accuracy register 22b of the register 22 is stored. Similarly, the search address of the search address register 23e that stores the first register at the time of checking the registered pattern 23 is incremented by 1000 addresses, and the search address is sequentially added to the comparison register 23b.
The following registered patterns are transferred, each is compared with the contents of the input audio register 23d, and the contents of the accuracy storage address register 23c are incremented by "10" and sequentially stored in the accuracy register 22b which is the comparison result. After storing all the accuracies of row A in this manner in the accuracy register 22b, the accuracies of the first and second ranks are searched, and the respective syllables and accuracies are output to the recognition result register 17. The flow of these processes is shown in FIG. This recognition result register 17 stores the syllable of the first rank and the accuracy of
The syllable of the second rank and the accuracy are stored in the second candidate register 17b. In addition
No. is a number indicating the input order.

In the example shown in FIG. 7, in order to input "warm", the input is divided into monosyllables such as "a", "ta", "ta", "ka", and "i", and the speech recognition unit 16 recognizes it as the first candidate. It shows that the result was "Atata Kai". The recognition accuracy for each is 9 in the input order as shown in Figure 7.
5, 85, 81, 86, 96. The second candidates are sir 55, par 70, tar 75, tar 73, and key 60, as shown in FIG. 7, showing the recognized syllables and accuracy in the order of input. Note that from the sixth input onward, a mark (*) indicating that no input was made is stored. The word syntax understanding processing unit 18 uses the word dictionary in the word/grammar dictionary 19 to check whether the monosyllable group stored in the recognition result register 17 is valid as one word by the longest match method. That is, the recognized syllable ``Atatakai'' in the first candidate register 17a is transferred to the verification register 18a shown in FIG. 9, and a test is performed to see if this word ``Atatakai'' exists in the word dictionary. If not, ``i'' is deleted. The test will be conducted using the "Atakaka" test. In this way, the test is performed sequentially for "Ataka", "Ata", and "A". "A" means "to meet", "to meet", "to brighten", "to dawn", "to hit", "to bathe"
In the stems of verbs such as ``aru'' and ``aru'', the ``takaka'' that follows ``a'' does not match the ending of the conjugated form. In this way, if none of them are found in the heading of the word dictionary, next check whether the first syllable "a" is a prefix. In this example, since "a" is not a prefix, prefix processing is not performed. However, if there is a prefix such as "o" in the "a" part, remove this prefix and use the remaining four syllables to test whether or not they are valid as a single word using the longest match method. In this example, there is nothing that can be identified as a word in the syllable string "atatakai."

In the conventional method, hiragana or katakana were displayed on the display device as is, and the speaker had to make a decision, but in the method according to the present invention, a monosyllable string that a person can recognize as a word or a sentence is input. Focus on this and process it. An example of the processing method will be explained in detail below using FIG. 6. That is, when the word syntax understanding processing unit 18 tests and cannot identify the word as a word, it sends a retry signal to the retry processing unit 21 to start retry processing. When the retry processing unit 21 receives the retry signal, it checks the retry counter 21b shown in FIG. Extract the lowest accuracy from
Set to 1c. In addition, the retry counter 21
When the count value of b is "1", this count value is increased by "1", the recognized syllable in the first candidate register 17a is loaded again into the verification register 18a, and the highest accuracy is extracted from the second candidate register 17b. , sets a number indicating the input order in the replacement number register 21c. Further, when the count value of the retry counter 21b is "2", the retry counter 21b is set to "0".
After resetting to , an error signal is sent to the word syntax understanding processing section 18. In this example, this is the first retry, and the count value of the retry counter 21b is "0", so "3" is set in the replacement number register 21c. Next, the retry processing unit 21 reads the third content "ta" of the second candidate register 17b based on the content "3" of the replacement number register 21c, and sets the third content "ta" of the verification register 18a. After storing, a test signal is sent to the word syntax understanding processing section 18.
The word syntax understanding processing unit 18 performs the test again as described above, but at this time, since the content of the test register 18a is "atatakai", the word "warm" is included in the word dictionary in the word grammar dictionary 19. It appears in the heading, and we can see that it is an adjective final form. Therefore, the word grammar dictionary 19 outputs the kanji code for ``warm'' and the kana code for ``ka'' and ``i'', which are printed or displayed on the output device 20.

Also, unlike this example, if identification is not possible after the first retry, the first candidate register 17a is used again.
The recognized syllables are transferred to the verification register 18a, and the second
The recognized syllable of the candidate register 17b with the highest accuracy is replaced with the corresponding syllable of the test register 18a and retested, but if it still cannot be recognized, the recognized syllable of the first candidate register 17a is sent to the output device 20 in kana code, Leave it to the speaker's discretion. In addition, even if there is only an adjective verb with the heading "warm" in the word dictionary, it will be identified in the second test using the longest match method, and as a result, the adjective verb "warm" will have the suffix "i" attached. Therefore, it is not necessary to have a huge number of headwords, and this point results in a problem with the maintenance of the word/grammar dictionary 19 and the algorithm of the word syntax understanding processing unit 18, which hinders the present invention. Please note that this is not the case. The word syntax understanding processing unit 18 remembers that the adjective is "warm" and outputs the kanji code "warm" and the kana code "ka" and the kana code "ii" to the output device 20 in the above order. "Warm" is output. The output device 20 may be a CRT 13 included in the voice recognition section 16, or
Alternatively, another printer or the like may be used. The word syntax understanding processing unit 18 tests not only the words explained in detail using the word/grammar dictionary 19 but also the syntax etc. from a grammatical point of view.
The reason why we memorize the adjective ``warm'' is to use it as reference data for the word syntax understanding process for the next input phrase. It is possible to even estimate whether the input will be made.

As explained above, in the above embodiment, it is assumed that the monosyllabic input string is input in units of words or phrases.
Furthermore, focusing on the words and syntax that people can understand when heard, the speech recognition unit 16 outputs the first candidate syllable, the second candidate syllable, and recognition accuracy information for each, and the word syntax understanding processing unit 18 tests them. By outputting words and phrases that people can understand when they hear it, it becomes possible to improve the effective recognition rate of the voice typewriter device without relying solely on the recognition rate of the voice recognition unit 16. There is a big advantage.

In the embodiment described above, the speech recognition unit 16 is configured in a simple manner as explained in FIG. 1, but the present invention is not limited to the speech recognition method having the structure shown in FIG. Needless to say, even if a recognition method is used, it will contribute to improving the overall effective recognition rate as a voice typewriter.

In addition, in the above embodiment, for simplicity, the voice recognition unit 1
Although the single syllable string output from 6 to the word syntax understanding processing unit 18 through the recognition result register 17 has been explained in units of words or phrases, the function of the word syntax understanding processing unit 18 also includes a function of breaking it down into phrases. If configured in this way, the monosyllable string can be input as an entire sentence without being limited to words or phrases, and by adding a meaning understanding function to the word syntax understanding processing section 18, the effective recognition rate can be increased. It can be improved more and more.

Furthermore, in the above embodiment, if the first candidate monosyllable string cannot be identified as a word or phrase, the one with the lowest recognition accuracy in the first candidate group is eliminated and replaced with the second monosyllable candidate. If it was not recognized again after retesting, the most accurate monosyllable in the second candidate monosyllable string was replaced with the corresponding syllable in the first candidate monosyllable string and retested. If it cannot be identified, the first candidate monosyllable sequence may be output as is and left to the speaker's discretion.

Also, when inputting something that a person cannot understand as a word or phrase, such as cipher text,
A selection switch is provided to output the word to the output device 20 without using the verification function of the word syntax understanding processing section 18, or to go one step further, the word syntax understanding processing section 18 and word/grammar that match the grammar, rules, etc. of the ciphertext are provided. Applications such as preparing a dictionary 19 and testing its legality may also be considered.

As explained in detail above, according to the present invention, by skillfully aligning the characteristics of the Japanese language and performing word syntax understanding processing, the serious drawback of misrecognition of voice input can be eliminated, and an ideal voice typewriter can be created. can be provided. Furthermore, the present invention can be said to be an invention that brings about a significant change not only in voice typewriters but also in voice input Japanese processing technology including voice input word processors and the like.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the speech recognition section of a conventional monosyllabic typewriter, FIG. 2 is a block diagram showing an embodiment of the present invention, FIG. 3 is a diagram for explaining the operation of the speech recognition section, and FIG. The figure and Figure 5 are of the speech recognition section.
Figure 6 is a diagram showing the contents of the RAM, Figure 6 is a diagram explaining the operation of the retry processing unit, Figure 7 is a diagram showing the contents of the recognition result register, and Figure 8 is a diagram showing the elements of the retry processing unit. , FIG. 9 is a diagram showing the verification register. 1...Microphone, 2...Preamplifier, 3
...16CHBPF, 4...16CHA/D converter,
5... Logarithmic conversion ROM, 6... Pitch extractor, 7
... Counter timer, 8 ... Micro CPU, 9
...BUS LINE, 10...ROM, 11...
RAM, 12...FDD, 13...CRT, 14...
...KB, 15...Hardware computing unit, 16...
Speech recognition unit, 17...Recognition result register, 17a
...First candidate register, 17b...Second candidate register, 18...Word syntax understanding processing unit, 18a...
Certification register, 19...Word/grammar dictionary, 20...
...output device, 21...retry processing unit, 22...syllable and accuracy register, 22a...syllable register, 22b...accuracy register, 23...registered pattern memory, 23a...registered pattern, 23b...
...Comparison register, 23c...Accuracy storage register,
23d...Input audio register.

Claims (1)

[Claims] 1. A speech input typewriter that inputs speech, processes it, and outputs it as information in a predetermined format, which outputs at least two recognized syllables and their respective accuracies for one input syllable. a recognition result register that classifies and stores the recognized syllables and accuracies into a plurality of syllable groups based on differences in accuracy, a word/grammar dictionary that stores words and grammar, and the recognition result register. A word/syntax understanding processing unit extracts a group of syllables from the word/syntax dictionary and tests whether it is correct or incorrect by referring to the words and grammar stored in the word/grammar dictionary. a retry processing unit that replaces a part of the syllable group with a part of another syllable group stored in the recognition result register and performs a retest;
A monosyllabic voice input typewriter comprising an output device that prints or displays the output of a syntax understanding processing section.