JP6996186B2 - Information processing equipment, language judgment method and program - Google Patents

Description

The present invention relates to an information processing apparatus, a language determination method and a program.

Patent Document 1 discloses a technique of performing voice recognition, converting voice into a character string, and generating a subtitle screen based on the obtained character string. Here, when converting a voice into a character string, it is necessary to specify the language of the voice (for example, Japanese, English, Chinese, etc.). For example, if voice recognition is performed in English even though the language of the voice is Japanese, there is a possibility that a character string that does not make sense is generated or that it becomes impossible to generate the character string. .. Therefore, the user needs to set the language of the input voice in the voice input terminal in advance. However, setting the language in advance may be a troublesome task for the user.

In relation to this technique, Patent Document 2 discloses a translation system for determining a language spoken by a stranger and establishing a conversation in a translation device for having a conversation with a stranger. In Patent Document 2, the language spoken by the other party is determined from the characteristics of the voice of the other party, the race of the other party, and the like.

Japanese Unexamined Patent Publication No. 6-141240 Japanese Unexamined Patent Publication No. 2006-268710

Voice characteristics and the race of the speaker do not necessarily specify the language. For example, if the race is Mongoloid, the language spoken by that person may be Japanese or Chinese. Therefore, in the technique of Patent Document 2, there is a possibility that the language spoken by the other party cannot be accurately determined.

In view of the above problems, it is an object of the present invention to provide an information processing device, a language determination method and a program capable of accurately determining a vocal language without setting a vocal language in advance.

Therefore, in the present invention, there is a first method in which a conversion unit that converts input voice into a character string, a selection unit that selects a plurality of languages that are candidates for the voice, and one or more words and phrases for each language are associated with each other. Based on the table, the converted character string includes a determination unit for determining whether or not a phrase associated with the language that is a candidate for the voice is included, and if so, a determination unit. Provided is an information processing apparatus including a language determining unit that determines a language associated with a phrase determined to be the language of the voice.

Further, the present invention includes a selection step of selecting a plurality of languages that are candidates for the input voice, a conversion step of converting the voice into a character string based on the plurality of selected languages, and one or more for each language. Based on the first table associated with the words and phrases of, the converted character string includes a determination step of determining whether or not the words and phrases associated with the language that is the candidate for the voice are included. If so, the present invention provides a language determination method including a language determination step of determining a language associated with a phrase determined to be included as the language of the voice.

Further, the present invention includes a selection step of selecting a plurality of languages that are candidates for the input voice, a conversion step of converting the voice into a character string based on the plurality of selected languages, and one or more for each language. Based on the first table associated with the words and phrases of, the converted character string includes a determination step of determining whether or not the words and phrases associated with the language that is the candidate for the voice are included. If so, the present invention provides a program for causing a computer to execute a language determination step of determining a language associated with a phrase determined to be included as the language of speech.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an information processing apparatus, a language determination method and a program capable of accurately determining a vocal language without setting a vocal language in advance.

It is a figure which shows the subtitle generation system which concerns on Embodiment 1. FIG. It is a figure which shows the structure of the information processing apparatus which concerns on Embodiment 1. FIG. It is a figure which illustrates the country / candidate table which concerns on Embodiment 1. FIG. It is a figure which illustrates the phrase table which concerns on Embodiment 1. FIG. It is a flowchart which shows the process performed by the information processing apparatus which concerns on Embodiment 1. FIG. It is a figure which illustrates the display terminal which displays the subtitles which concerns on Embodiment 1. FIG. It is a figure which shows the structure of the information processing apparatus which concerns on Embodiment 2. It is a flowchart which shows the process performed by the information processing apparatus which concerns on Embodiment 2. It is a figure which shows the structure of the information processing apparatus which concerns on Embodiment 3. FIG. It is a figure which illustrates the phrase table which concerns on Embodiment 3. FIG. It is a flowchart which shows the process performed by the information processing apparatus which concerns on Embodiment 3. FIG. It is a figure which shows the structure of the information processing apparatus which concerns on Embodiment 4. It is a figure which illustrates the race / candidate table which concerns on Embodiment 4. It is a flowchart which shows the process performed by the information processing apparatus which concerns on Embodiment 4. It is a figure which shows the structure of the information processing apparatus which concerns on Embodiment 5. It is a flowchart which shows the process performed by the information processing apparatus which concerns on Embodiment 5.

(Embodiment 1)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that substantially the same components are designated by the same reference numerals.

FIG. 1 is a diagram showing a subtitle generation system 1 according to the first embodiment. The subtitle generation system 1 includes an input terminal 10, a display terminal 20, and an information processing device 100. The input terminal 10, the display terminal 20, and the information processing device 100 are communicably connected to each other via a wired or wireless network 2.

The input terminal 10 is a device for inputting video, audio, and the like. The input terminal 10 is, for example, a camera, a mobile phone, a smartphone, a tablet terminal, or the like. The display terminal 20 is, for example, a personal computer (PC), a smartphone, a tablet terminal, or the like. For example, the display terminal 20 has an application or a web browser that can realize a streaming function such as live streaming. The information processing device 100 is, for example, a computer such as a server.

The input terminal 10 receives audio and video. The information processing apparatus 100 converts the voice input to the input terminal 10, that is, the input voice into a character string. Then, the information processing apparatus 100 uses the obtained character string to generate subtitles to be added to the video input to the input terminal 10. The display terminal 20 reproduces the video and audio input to the input terminal 10. Further, the display terminal 20 adds subtitles to the video and displays it. Here, the information processing apparatus 100 converts the input voice into a character string without the user setting the language of the voice. This improves user convenience.

The input terminal 10 has a communication unit 12, an audio input unit 14, a video input unit 16, and a position acquisition unit 18. The communication unit 12 performs processing necessary for communicating with the information processing device 100 and the like via the network 2. The voice input unit 14 may be composed of, for example, a microphone. The voice input unit 14 inputs voice or the like emitted by a user around the input terminal 10. Further, the voice input unit 14 generates voice data indicating the input voice, that is, the input voice.

The video input unit 16 may be composed of an image pickup element such as a camera. The video input unit 16 captures an image in front of the video input unit 16. Then, the video input unit 16 generates video data indicating the input video, that is, the input video. The position acquisition unit 18 acquires the position information of the input terminal 10 by GPS (Global Positioning System) or the like.

The communication unit 12 transmits audio data, location information, and video data to the information processing apparatus 100 via the network 2. Hereinafter, the term "input voice" also means voice data indicating input voice. Similarly, the term "video" also means video data indicating video. The term "position" also means position information indicating a position. Further, the term "character string" also means text data indicating a character string. The term "subtitle" also means subtitle data indicating subtitles.

The display terminal 20 has a communication unit 22, an audio output unit 24, a video display unit 26, and a subtitle output unit 28. The communication unit 22 performs processing necessary for communicating with the information processing device 100 and the like via the network 2. The communication unit 22 receives audio data, video data, and subtitle data from the information processing device 100.

The audio output unit 24 may be composed of, for example, a speaker or the like. The voice output unit 24 reproduces the voice received from the information processing apparatus 100. The video display unit 26 may be composed of, for example, a display or the like. The video display unit 26 reproduces the video received from the information processing device 100. The subtitle output unit 28 adds subtitles corresponding to audio to the video display unit 26 and outputs the subtitles.

FIG. 2 is a diagram showing the configuration of the information processing apparatus 100 according to the first embodiment. The information processing apparatus 100 has a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF; Interface) as a main hardware configuration. The control unit 102, the storage unit 104, the communication unit 106, and the interface unit 108 are connected to each other via a data bus or the like.

The control unit 102 is, for example, a processor such as a CPU (Central Processing Unit). The control unit 102 has a function as an arithmetic unit that performs control processing, arithmetic processing, and the like. The storage unit 104 is a storage device such as a memory or a hard disk. The storage unit 104 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The storage unit 104 has a function for storing a control program, an arithmetic program, and the like executed by the control unit 102. In addition, the storage unit 104 has a function for temporarily storing processed data and the like.

The communication unit 106 performs processing necessary for communicating with the input terminal 10 and the display terminal 20 via the network 2. The communication unit 106 may include a communication port. The interface unit 108 (IF; Interface) is, for example, a user interface (UI). The interface unit 108 has an input device such as a keyboard, a touch panel or a mouse, and an output device such as a display or a speaker. The input terminal 10 and the display terminal 20 may also have the same hardware configuration as the control unit 102, the storage unit 104, the communication unit 106, and the interface unit 108.

Further, the information processing apparatus 100 has a language determination unit 110 and a subtitle generation unit 130. The language determination unit 110 determines the language of the input voice, as will be described later. The subtitle generation unit 130 converts the input voice into a character string in the determined language to generate the subtitle. Then, the subtitle generation unit 130 transmits the generated subtitles to the display terminal 20 using the communication unit 106. Details will be described later. The information processing device 100 can function as a language determination device with such a configuration.

The language determination unit 110 and the subtitle generation unit 130 can be realized by, for example, executing a program under the control of the control unit 102. More specifically, the language determination unit 110 and the subtitle generation unit 130 can be realized by the control unit 102 executing the program stored in the storage unit 104. Further, the language determination unit 110 and the subtitle generation unit 130 may be realized by recording a necessary program on an arbitrary non-volatile recording medium and installing it as needed. Further, each component of the language determination unit 110 and the subtitle generation unit 130 is not limited to being realized by software by a program, but may be realized by any combination of hardware, firmware, and software. .. Further, each component of the language determination unit 110 and the subtitle generation unit 130 may be realized by using a user-programmable integrated circuit such as an FPGA (field-programmable gate array) or a microcomputer. In this case, this integrated circuit may be used to realize a program composed of each of the above components. The above is the same in other embodiments described later.

The language determination unit 110 includes an input voice acquisition unit 112, a position information acquisition unit 114, a candidate selection unit 116, a country / candidate table 118 (second table), a conversion unit 120, a determination unit 122, and a phrase. It has a table 124 (first table) and a language determination unit 126. Each component of these language determination units 110 will be described in detail later with reference to FIG. 5 and the like, but some components will be described below.

FIG. 3 is a diagram illustrating the country / candidate table 118 according to the first embodiment. The country / candidate table 118 associates a country with a language candidate that is a candidate for the language of the input voice. A language candidate is a language that is likely to be spoken by a person in the corresponding country in the country / candidate table 118. In the example shown in FIG. 3, the country name "Japan" is associated with "Japanese", "English", etc. as language candidates. Further, the country name "US" is associated with "English", "Spanish", etc. as language candidates. In other words, in Japan, there is a high possibility that "Japanese" or "English" will be spoken. Also, in "US", "English" or "Spanish" is likely to be spoken.

FIG. 4 is a diagram illustrating the phrase table 124 according to the first embodiment. The phrase table 124 associates each of the plurality of languages with one or more phrases corresponding to each of the plurality of languages. Here, the words and phrases illustrated in FIG. 4 are words and phrases that are likely to occur at the beginning of an input voice, that is, an utterance, such as a greeting. In the example shown in FIG. 4, the language "Japanese" is associated with the phrases "good morning", "hello", and the like. Further, the language "English" is associated with the phrases "Good morning", "Hello" and the like. In the utterance of "Japanese", there is a high possibility that "Good morning" or "Hello" will be spoken at the beginning. Further, in the utterance of "English", there is a high possibility that "Good morning" or "Hello" or the like is uttered at the beginning.

FIG. 5 is a flowchart showing a process performed by the information processing apparatus 100 according to the first embodiment. Here, in the flowchart shown in FIG. 5, the processes of S102 to S114 correspond to the language determination method. In step S102, the input voice acquisition unit 112 acquires the input voice from the input terminal 10. The input voice acquisition unit 112 outputs the acquired input voice to the conversion unit 120. Further, the position information acquisition unit 114 acquires position information from the input terminal 10. The position information acquisition unit 114 outputs the acquired position information to the candidate selection unit 116.

In step S104, the candidate selection unit 116 selects a plurality of language candidates corresponding to the country in which the input voice is emitted. The candidate selection unit 116 outputs a plurality of selected language candidates to the conversion unit 120. Specifically, the candidate selection unit 116 determines the country related to the position where the input voice is input from the position information acquired by the position information acquisition unit 114. The candidate selection unit 116 selects a language corresponding to the country as a language candidate in the country / candidate table 118. For example, when the location information indicates a certain point in Japan, the candidate selection unit 116 selects Japanese, English, or the like as language candidates using the country / candidate table 118.

Next, in step S106, the conversion unit 120 converts the input voice into a character string for each of the plurality of languages selected in the process of S104. The conversion unit 120 outputs the obtained character string to the determination unit 122. Specifically, the conversion unit 120 performs voice recognition processing for each of the plurality of selected languages, and generates a character string from the input voice. In the example in which the language candidates are Japanese and English, the conversion unit 120 performs voice recognition processing in Japanese on the input voice and acquires a Japanese character string. Similarly, the conversion unit 120 performs voice recognition processing on the input voice in English to acquire an English character string.

The conversion unit 120 does not need to convert all the acquired input voices into character strings. The conversion unit 120 may perform conversion processing only on the voice input in a predetermined period to the extent that the input voice includes the first five words and phrases. For example, the conversion unit 120 may perform the conversion process only for about 10 seconds from the beginning of the input voice. In addition, as a result of performing voice recognition processing with a certain language candidate, text conversion may not be performed satisfactorily because the input voice does not include the pronunciation of that language. In this case, the character string may not be generated.

Next, the determination unit 122 determines, for each of the plurality of language candidates, whether or not at least one of the words and phrases corresponding to each of the plurality of language candidates in the phrase table 124 is included in the character string obtained in the process of S106. .. Specifically, in step S108, the determination unit 122 extracts words and phrases corresponding to each of the selected language candidates from the word and phrase table 124. In the example in which the language candidates are Japanese and English, the determination unit 122 extracts the words and phrases corresponding to the language candidate "Japanese" from the word and phrase table 124, such as "good morning" and "hello". Similarly, the determination unit 122 extracts words such as “Good morning” and “Hello” corresponding to the language candidate “English” in the word / phrase table 124.

Next, in step S110, the determination unit 122 determines whether or not the phrase extracted from the phrase table 124 is included in the character string obtained by the text conversion process of S106. Then, the determination unit 122 outputs the determination result to the language determination unit 126. If at least one of the extracted words and phrases is included in the character string obtained in the process of S106 (YES in S110), the process proceeds to S112. On the other hand, when all of the extracted words and phrases are not included in the character string obtained in the process of S106 (NO in S110), the process proceeds to S114.

In step S112, the language determination unit 126 determines the language candidate corresponding to the phrase included in the character string as the language of the input voice. Specifically, when the input voice is "Hello, it is fine weather today", the phrase "Hello" extracted from the phrase table 124 is included in the input voice. Then, the phrase "hello" corresponds to the language candidate "Japanese" in the phrase table 124. Therefore, the language determination unit 126 determines the language of the input voice, that is, the input language as "Japanese".

Further, when the input voice is "Good morning. It's a beautiful day today.", The phrase "Good morning" extracted from the phrase table 124 is included in the input voice. Then, the phrase "Good morning" corresponds to the language candidate "English" in the phrase table 124. Therefore, the language determination unit 126 determines the input language as "English".

On the other hand, in step S114, the language determination unit 126 determines the language of the input voice, that is, the input language by a method different from the processing of S104 to S112. The "method different from the processing of S104 to S112" is a method in which at least one of the country / candidate table 118 and the phrase table 124 is not used. For example, if the input voice is Spanish, the words "Good morning", "Hello", "Good morning" and "Hello" corresponding to the language candidates "English" and "Japanese" may not be included in the input voice. Very high sex. Therefore, in this case, the language determination unit 126 determines the language of the input voice, that is, the input language by a method different from the processing of S104 to S112. Also, for example, if the input voice is "everyone, it's nice weather today", not only the English words "Good morning" and "Hello" but also the Japanese words "Good morning" and "Hello" are input voices. Not included in. Therefore, also in this case, the language determination unit 126 determines the language of the input voice, that is, the input language by a method different from the processing of S104 to S112.

In addition, there are some methods in the processing of S114, that is, "a method different from the processing of S104 to S112". For example, the language determination unit 126 may newly select a language other than the language candidate selected in S104 as a language candidate. That is, the language determination unit 126 may select a language candidate without using the country / candidate table 118. Then, when the language candidate and the phrase that are not related to the newly selected position information are associated with each other in the phrase table 124, the language determination unit 126 performs the processes of S106 to S110 for the language candidate. May be good. Then, when the input language is not determined even if the processes of S106 to S110 are performed for all the languages set in the phrase table 124, the following method may be performed. The following method may be performed without performing the method of newly selecting the above language candidate.

For example, the language determination unit 126 may determine the input language by setting the language of the input voice by the user. Further, the language determination unit 126 may determine the input language by analyzing the phonology and the like of the input voice. Further, the language determination unit 126 provides a dictionary for all the languages used all over the world, converts the input voice into a character string for all the languages, and converts the character string and the word registered in the dictionary into a character string. A language with a high percentage of matching may be determined as the language of the input voice. In this case, the user may set the priority so that the frequently used language has a higher priority, and the above comparison may be performed from the language with the highest priority.

The language determination unit 126 outputs a signal indicating the determined input language and the input voice to the subtitle generation unit 130. In steps S120 and S122, the subtitle generation unit 130 generates subtitles. Specifically, the subtitle generation unit 130 converts the input voice into a character string in the input language determined by the language determination unit 126 (S120). The conversion method in the process of S120 may be substantially the same as the conversion method in the process of S106. On the other hand, the processing of S120 is different from the processing of S106 in that the subtitle generation unit 130 converts only the determined input language and converts all of the input audio.

The subtitle generation unit 130 generates subtitles using the character string obtained by the conversion (S122). Specifically, the subtitle generation unit 130 generates subtitles by converting the character string into a format that can be superimposed on the video. Then, the subtitle generation unit 130 transmits the generated subtitles to the display terminal 20. Then, when the voice input is not completed (NO in step S124), the processes of S120 to S122 are repeated. When the voice input is completed (YES in step S124), the process is completed.

FIG. 6 is a diagram illustrating a display terminal 20 displaying subtitles according to the first embodiment. An image is displayed on the image display unit 26. In addition, the input voice "Hello, it's fine weather today" is output from the voice output unit 24. Then, the subtitle output unit 28 displays the subtitle corresponding to the input sound so as to be superimposed on the video displayed on the video display unit 26.

As described above, the information processing apparatus 100 according to the present embodiment selects a plurality of voice language candidates and converts the voice into a character string for each of the selected plurality of candidate languages. Further, the information processing apparatus 100 determines whether or not at least one of the words and phrases corresponding to each of the plurality of candidates in the phrase table 124 is included in the obtained character string for each of the plurality of candidate languages. Then, when at least one of the above words and phrases is included in the character string, the information processing apparatus 100 determines the candidate language corresponding to the included words and phrases as the vocal language.

When the phrase included in the phrase table 124 is included in the character string obtained by converting the input voice with the language candidate, it is very likely that the input language is the language corresponding to the phrase. For example, if the phrase "hello" is included in the character string corresponding to the input voice, it is very likely that the input voice is in Japanese. Therefore, with the above configuration, the information processing apparatus 100 according to the present embodiment can accurately determine the voice language without setting the voice language in advance.

Further, the information processing apparatus 100 according to the present embodiment selects a language candidate and converts the input voice into a character string by the language candidate. This eliminates the need to convert input speech to strings in any language around the world. Therefore, in the information processing apparatus 100 according to the present embodiment, it is possible to more accurately determine the language of speech while suppressing an increase in the amount of processing. Therefore, the information processing apparatus 100 according to the present embodiment can quickly and accurately determine the language of speech.

Further, the phrase table 124 according to the first embodiment associates a phrase that may occur at the beginning of an utterance, such as a greeting, with a language. The words and phrases that can occur at the beginning of an utterance are often limited to standard phrases such as greetings. Furthermore, by using words and phrases that can occur at the beginning, when converting the input voice into a character string in the language candidate, it is only necessary to convert only the first few seconds of the input voice. Therefore, the information processing apparatus 100 according to the first embodiment can determine the language of the input voice more quickly and accurately by using the phrase table 124 including the phrase that can occur at the beginning.

Further, the information processing apparatus 100 according to the first embodiment selects a language candidate using the country / candidate table 118. The languages mainly used in the country are limited to some extent. Therefore, by selecting a language candidate using the country / candidate table 118, it is highly possible that the selected language candidate includes the input language. Therefore, the information processing apparatus 100 according to the first embodiment can determine the language of the input voice more accurately by using the country / candidate table 118. Furthermore, identifying a country from location information is relatively easy and can be done in a short amount of time. Therefore, it is possible to shorten the processing time by selecting the language candidate using the country / candidate table 118.

(Embodiment 2)
Next, the second embodiment will be described. The second embodiment differs from the first embodiment in that the country / candidate table 118 and the phrase table 124 are updated. Since the other configurations of the second embodiment are substantially the same as those of the first embodiment, the description thereof will be omitted.

FIG. 7 is a diagram showing the configuration of the information processing apparatus 100 according to the second embodiment. The hardware configuration of the information processing apparatus 100 according to the second embodiment is substantially the same as that according to the first embodiment. The information processing apparatus 100 according to the second embodiment has a language determination unit 210 and a subtitle generation unit 130. The language determination unit 210 has an update unit 220 in addition to the components of the language determination unit 110.

The update unit 220 updates the table according to the processing of the language determination unit 126. The update unit 220 has a country / candidate table update unit 222 (second update unit) and a phrase table update unit 224 (first update unit). The country / candidate table update unit 222 updates the country / candidate table 118. The phrase table update unit 224 updates the phrase table 124. The method of updating the table will be described below.

FIG. 8 is a flowchart showing a process performed by the information processing apparatus 100 according to the second embodiment. The processes of S102 to S114 and S122 to S124 are substantially the same as the processes in the flowchart shown in FIG. In S114, when the input language is determined by a method different from the processing of S104 to S112, the country / candidate table update unit 222 updates the country / candidate table 118 (step S202).

Specifically, the country / candidate table update unit 222 adds the input language determined in the process of S114 to the country / candidate table 118 as a language candidate. For example, a case where the position information acquired in the process of S102 corresponds to "Japan" will be described. At this time, it is assumed that "Japanese" and "English" are selected as language candidates, but the input language is "Chinese". In this case, the country / candidate table update unit 222 adds "Chinese" to the language candidate of the country name "Japan" of the country / candidate table 118.

Next, after the input voice is converted into a character string in the input language (S120), the phrase table update unit 224 updates the phrase table 124 (step S204). Specifically, the phrase table update unit 224 extracts the first phrase from the character string obtained in the process of S120. Then, the phrase table update unit 224 adds the extracted phrase to the phrase table 124 as a phrase corresponding to the input language. For example, when the input language is "Chinese" and the opening phrase is "Niihao", the phrase table update unit 224 adds "Niihao" to the phrase of the language "Chinese" in the phrase table 124. Further, for example, when the input language is "Japanese" and the input voice is "everyone, it's nice weather today ...", the phrase table update unit 224 uses the language "Japanese" in the phrase table 124. Add "everyone" to the phrase.

When the information processing apparatus 100 according to the present embodiment cannot determine the input language by the processing of S104 to S112 shown in FIG. 5, in the processing of S114, the input language is used by a method different from the processing of S104 to S112. decide. On the other hand, in the process of S114, when the user inputs an input language, annoying work is added to the user. Further, in the method of newly selecting a language candidate, the method of using the phonology of the input voice, and the method of using the comparison with the dictionary for all the languages, the accuracy of determining the input language is compared with the processing of S104 to S112. It is bad and the processing time may be long. Therefore, it is desirable to determine the input language by using the processes of S104 to S112 as much as possible.

On the other hand, the information processing apparatus 100 according to the second embodiment is configured to update the country / candidate table 118 and the phrase table 124 by self-learning. Therefore, even if the input language cannot be determined by the processes of S104 to S112 shown in FIG. 5, a voice containing the same words and phrases as the words and phrases contained in the input voice is subsequently input in the same language as the input voice. In this case, the input language can be determined by the processing of S104 to S112 shown in FIG. As a result, the more the information processing apparatus 100 performs the processing shown in FIG. 8, the more the processing of S104 to S112 can be performed. Therefore, the accuracy of determining the input language can be improved and the processing time can be shortened. It becomes.

(Embodiment 3)
Next, the third embodiment will be described. The third embodiment is different from the other embodiments in that a plurality of word / phrase tables corresponding to each of the plurality of time zones are provided. Since the other configurations of the third embodiment are substantially the same as those of the other embodiments, the description thereof will be omitted.

FIG. 9 is a diagram showing the configuration of the information processing apparatus 100 according to the third embodiment. The hardware configuration of the information processing apparatus 100 according to the third embodiment is substantially the same as that according to the other embodiments. The information processing apparatus 100 according to the third embodiment has a language determination unit 310 and a subtitle generation unit 130. The language determination unit 310 has a phrase table 320 instead of the phrase table 124. The phrase table 320 has a plurality of tables depending on the time zone.

FIG. 10 is a diagram illustrating the phrase table 320 according to the third embodiment. The phrase table 320 has, for example, a morning time zone table 322 and a daytime zone table 324. The morning time zone table 322 associates each of the plurality of languages with words and phrases corresponding to each of the plurality of languages that are likely to be emitted in the morning time zone. The daytime zone table 324 associates each of the plurality of languages with words and phrases corresponding to each of the plurality of languages that are likely to be emitted during the daytime zone.

In the example shown in FIG. 10, in the morning time zone table 322, the language "Japanese" is associated with the phrase "good morning" and the like. Further, in the morning time zone table 322, the language "English" is associated with the phrase "Good morning" and the like. On the other hand, in the example shown in FIG. 10, in the daytime time zone table 324, the language "Japanese" is associated with the phrase "hello" and the like. Further, in the daytime time zone table 324, the language "English" is associated with the phrase "Hello" and the like. In the morning hours, there is a high possibility that "Good morning" etc. will be uttered at the beginning of the utterance of "Japanese", and "Good morning" etc. will be uttered at the beginning of the utterance of "English". On the other hand, in the daytime, there is a high possibility that "Hello" or the like is uttered at the beginning of the utterance of "Japanese", and "Hello" or the like is uttered at the beginning of the utterance of "English".

The phrase table 320 shown in FIG. 10 is merely an example. The phrase table 320 may have a phrase table corresponding to the night time zone. Also, "time zone" does not mean only the period of the day. For example, "time zone" can also mean a period of the year, such as spring, summer, autumn and winter.

FIG. 11 is a flowchart showing the processing performed by the information processing apparatus 100 according to the third embodiment. The process according to the third embodiment is a modification of the process of S108 in the flowchart shown in FIG. Regarding the processes other than S108, the process according to the third embodiment is substantially the same as the process according to the first embodiment.

In step S300, the determination unit 122 acquires time information indicating the time when the input voice is emitted. The time information may be acquired from the input terminal 10. Further, the time when the information processing apparatus 100 acquires the input voice may be set as the time when the input voice is emitted.

Next, in step S302, the determination unit 122 determines the time zone in which the voice is emitted. Specifically, the determination unit 122 determines in which time zone in the phrase table 320 the time indicated by the time information acquired in S300 is included. For example, when the time indicated by the time information acquired in S300 is included in the time zone from 5:00 to 12:00, the determination unit 122 determines that the time zone in which the voice is emitted is the "morning time zone". You may. Further, for example, when the time indicated by the time information acquired in S300 is included in the time zone from 12:00 to 18:00, the determination unit 122 sets the time zone in which the voice is input as the “daytime zone”. May be determined.

Next, in step S304, the determination unit 122 extracts words and phrases corresponding to each of the selected language candidates from the word and phrase table 320 corresponding to the time zone determined by the process of S302. For example, when the language candidates are "Japanese" and "English" and the time zone is "morning time zone", the determination unit 122 inputs the words "good morning" and "Good morning" from the morning time zone table 322. Extract.

The information processing apparatus 100 according to the third embodiment has a plurality of word / phrase tables corresponding to each of the plurality of time zones as described above. Therefore, since the phrase table can be switched according to the time zone in which the voice is emitted, it is possible to speed up the language determination process.

(Embodiment 4)
Next, the fourth embodiment will be described. The fourth embodiment is different from other embodiments in that a language candidate is selected from the race of the person regarding the input voice. Since the other configurations of the fourth embodiment are substantially the same as those of the first embodiment and the like, the description thereof will be omitted.

FIG. 12 is a diagram showing the configuration of the information processing apparatus 100 according to the fourth embodiment. The hardware configuration of the information processing apparatus 100 according to the fourth embodiment is substantially the same as that according to the other embodiments. The information processing apparatus 100 according to the fourth embodiment has a language determination unit 410 and a subtitle generation unit 130.

The language determination unit 410 includes an input audio acquisition unit 112, a video acquisition unit 412, a race identification unit 414, a race database 416, a candidate selection unit 418, and a race / candidate table 420 (third table). It has a conversion unit 120, a determination unit 122, a phrase table 124, and a language determination unit 126. The video acquisition unit 412, the race identification unit 414, the race database 416, and the candidate selection unit 418 will be described in detail later with reference to FIG. 14 and the like.

FIG. 13 is a diagram illustrating the race / candidate table 420 according to the fourth embodiment. The race / candidate table 420 associates race with language candidates. A language candidate can be a language that is likely to be spoken by a person of the corresponding race in the race / candidate table 420. In the example shown in FIG. 13, race A (for example, Mongoloid, etc.) is associated with "Japanese", "Chinese, etc." as language candidates. Further, race B (for example, Caucasian race) is associated with "English", "French" and the like as language candidates. That is, a person of race A is likely to speak "Japanese" or "Chinese". Also, people of race B are more likely to speak "English" or "French".

FIG. 14 is a flowchart showing the processing performed by the information processing apparatus 100 according to the fourth embodiment. The processing of S106 to S124 is substantially the same as the processing in the flowchart shown in FIG. First, in step S400, the input voice acquisition unit 112 acquires the input voice from the input terminal 10. Further, the video acquisition unit 412 acquires video information from the input terminal 10. The video acquisition unit 412 outputs the acquired video information to the race identification unit 414. The video information acquired from the input terminal 10 preferably indicates the video captured when the input audio is input. Further, it is desirable that this video includes the speaker of the input audio.

Next, in step S402, the race identification unit 414 identifies the race of the person included in the video by using the race database 416. The race identification unit 414 outputs a signal indicating the specified race to the candidate selection unit 418. The race database 416 associates physical features such as faces of people of various races with the corresponding races. The physical characteristics are, for example, skin color, eye color, hair color, nose height, skeleton and the like. The race identification unit 414 recognizes the human face included in the image. Then, the race identification unit 414 detects the physical characteristics that characterize the race, and uses the detected feature amount of the physical characteristics and the race database 416 to determine the race of the person included in the image. Identify.

Next, in step S404, the candidate selection unit 418 selects a plurality of language candidates corresponding to the specified race. The candidate selection unit 418 outputs a plurality of selected language candidates to the conversion unit 120. Specifically, the candidate selection unit 418 selects a language corresponding to the specified race as a language candidate in the race / candidate table 420. For example, when the specified race is "Race A", the candidate selection unit 418 selects "Japanese" and "Chinese" as language candidates.

The information processing apparatus 100 according to the fourth embodiment identifies a race of a person related to voice, and selects a language corresponding to the specified race as a language candidate. Some countries, such as Japan, are likely to speak Japanese, while others, such as the United States, are likely to speak many languages. Therefore, the information processing apparatus 100 according to the fourth embodiment can determine the input language more accurately by selecting the language candidate according to the race of the person regarding the input voice.

(Embodiment 5)
Next, the fifth embodiment will be described. The fifth embodiment is different from other embodiments in that a language candidate is selected from the position information indicating the position where the voice is input and the race of the person related to the input voice. Since the other configurations of the fifth embodiment are substantially the same as those of the first embodiment and the fourth embodiment, the description thereof will be omitted.

FIG. 15 is a diagram showing the configuration of the information processing apparatus 100 according to the fifth embodiment. The hardware configuration of the information processing apparatus 100 according to the fifth embodiment is substantially the same as that according to the other embodiments. The information processing apparatus 100 according to the fifth embodiment has a language determination unit 510 and a subtitle generation unit 130. The language determination unit 510 includes an input audio acquisition unit 112, a position information acquisition unit 114, a country / candidate table 118, a video acquisition unit 412, a race identification unit 414, a race database 416, and a race / candidate. It has a table 420, a candidate selection unit 518, a conversion unit 120, a determination unit 122, a phrase table 124, and a language determination unit 126. The candidate selection unit 518 will be described later with reference to FIG.

FIG. 16 is a flowchart showing a process performed by the information processing apparatus 100 according to the fifth embodiment. The processing of S106 to S124 is substantially the same as the processing in the flowchart shown in FIG. First, in step S500, the input voice acquisition unit 112 acquires the input voice from the input terminal 10. Further, the position information acquisition unit 114 acquires position information from the input terminal 10. The position information acquisition unit 114 outputs the acquired position information to the candidate selection unit 518. Further, the video acquisition unit 412 acquires video information from the input terminal 10. The video acquisition unit 412 outputs the acquired video information to the race identification unit 414.

In step S502, the race identification unit 414 identifies the race of the person included in the video by using the race database 416 in the same manner as the process of S402 shown in FIG. The race identification unit 414 outputs a signal indicating the specified race to the candidate selection unit 518.

In step S504, the candidate selection unit 518 selects a plurality of language candidates corresponding to the country or race. Specifically, in the same manner as the process of S104 shown in FIG. 5, the candidate selection unit 518 selects a plurality of language candidates corresponding to the country in which the input voice is input. Further, in the same manner as the process of S404 shown in FIG. 14, the candidate selection unit 518 selects a plurality of language candidates corresponding to the specified race. For example, in the example using the tables illustrated in FIGS. 3 and 13, when the country in which the voice is input is "Japan" and the race is specified as "Race A", the candidate selection unit 518 is set to "Race A". Select "Japanese", "English" and "Chinese" as language candidates.

When selecting a language candidate using only the country / candidate table 118, the attribute of the speaker is not taken into consideration. Therefore, if the speaker is a traveler, the input language may not be included in the language candidates. It gets higher. Also, even when selecting a language candidate using only the race / candidate table 420, for example, if the person is a Mongoloid resident living in the United States, the person is likely to speak English, so the input language is the language. It is more likely that you will not be included in the candidates.

On the other hand, the information processing apparatus 100 according to the fifth embodiment is configured to select a language candidate by using both the country / candidate table 118 and the race / candidate table 420. Therefore, many language candidates can be selected, and therefore the input language is likely to be included in the language candidates. As a result, there is a high possibility that the input language can be determined by the processing of S104 to S112 shown in FIG. Therefore, the information processing apparatus 100 according to the fifth embodiment can determine the input language more accurately.

(Modification example)
The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. For example, the order of each process in the above-mentioned flowchart can be changed as appropriate. Also, the plurality of embodiments described above are applicable to each other.

For example, the second embodiment may be applied to the third embodiment. In this case, the update unit 220 may update the phrase table 320 according to the time zone in which the input voice is emitted. Further, the second embodiment may be applied to the fourth embodiment. In this case, the update unit 220 may update the race / candidate table 420 in the same manner as in S202 shown in FIG. That is, the update unit 220 may associate the race specified in the process of S402 with the input language determined in S114 in the race / candidate table 420. At this time, the update unit 220 functions as a third update unit.

Further, in the above-described embodiment, the country and the language candidate are associated with each other in the country / candidate table 118, but the configuration is not limited to this. Instead of associating the language candidate with the country, the language candidate may be associated with the location, that is, the location of the area or facility. For example, in places where many foreigners gather, such as tourist spots and international conference halls, many language candidates may be associated.

Further, in the above-described embodiment, the phrase included in the phrase table is a phrase that can occur at the beginning of the input voice, but the configuration is not limited to this. The words and phrases contained in the word and phrase table do not have to occur at the beginning. Further, in this case, in the flowchart according to the second embodiment shown in FIG. 8, the phrase table 124 may be updated even when YES in S110. That is, among the words and phrases included in the input voice, words and phrases other than the first word and phrase may be added to the word and phrase table 124.

As mentioned above, the words at the beginning are limited to some extent, such as greetings. Then, by using a phrase that can occur at the beginning, when converting the input voice into a character string in the language candidate, it is only necessary to convert only the first few seconds of the input voice. Therefore, by using the phrase table 124 including the phrase that can occur at the beginning, it is possible to determine the language of the input voice more quickly and accurately.

Further, in the above-described embodiment, the input terminal 10 is a terminal separate from the display terminal 20, but the input terminal 10 may be integrated with the display terminal 20. That is, the information processing device 100 converts the voice input to the input / output device in which the input terminal 10 and the display terminal 20 are integrated into a character string, and transmits the obtained character string to the input / output device. You may. The present invention also includes such a configuration.

In addition, the programs described above can be stored and supplied to a computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws. , Including semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

1 Subtitle generation system 10 Input terminal 20 Display terminal 100 Information processing device 110 Language judgment unit 112 Input voice acquisition unit 114 Position information acquisition unit 116 Candidate selection unit 118 Country / candidate table 120 Conversion unit 122 Judgment unit 124 Word table 126 Language determination unit 130 Subtitle generation unit 210 Language judgment unit 220 Update unit 222 Country / candidate table update unit 224 Word table update unit 310 Language judgment unit 320 Word / phrase table 322 Morning time zone table 324 Daytime zone table 410 Language judgment unit 412 Video acquisition unit 414 people Species identification unit 416 Race database 418 Candidate selection unit 420 Race / candidate table 510 Language judgment unit 518 Candidate selection unit

Claims (7)

A conversion unit that converts the input voice into a character string,
A selection unit that selects multiple languages that are candidates for voice, and
Based on the first table in which one or more words and phrases are associated with each language, it is determined whether or not the converted character string includes words and phrases associated with the language that is a candidate for the voice. Department and
An information processing device including a language determination unit that determines, if included, a language associated with a phrase determined to be included as the language of the voice. The information processing device according to claim 1, wherein the conversion unit converts voice input within a predetermined period into a character string. Based on the second table in which the position and the language that is the candidate for the voice are associated with each other, the selection unit selects the language corresponding to the position where the voice is emitted as the language that is the candidate for the voice. The information processing apparatus according to claim 1 or 2. The determination unit is based on a plurality of the first tables provided for each of the plurality of time zones, and whether or not the converted character string contains a phrase associated with the language that is a candidate for the voice. The information processing apparatus according to any one of claims 1 to 3. Based on the third table in which the race of the person who uttered the voice and the language that is the candidate for the voice are associated with each other, the selection unit selects the language corresponding to the race as the candidate for the voice. The information processing apparatus according to any one of claims 1 to 4, which is selected as a language. A selection step to select multiple candidate languages for the input voice, and
A conversion step for converting the voice into a character string based on the plurality of selected languages, and
Based on the first table in which one or more words and phrases are associated with each language, it is determined whether or not the converted character string includes words and phrases associated with the language that is a candidate for the voice. Steps and
A language determination method including a language determination step for determining a language associated with a phrase determined to be included as the speech language, if included. A selection step to select multiple candidate languages for the input voice, and
A conversion step for converting the voice into a character string based on the plurality of selected languages, and
Based on the first table in which one or more words and phrases are associated with each language, it is determined whether or not the converted character string includes words and phrases associated with the language that is a candidate for the voice. Steps and
If it is included, a program that causes the computer to execute a language determination step that determines the language associated with the phrase determined to be included as the language of the voice.

Images