JP6802264B2 - Display device, output device, and information display method - Google Patents

Description

The present invention relates to a display device, an output device, and an information display method, and more particularly to a technique effective for switching slides in a presentation requiring translation.

In presentations such as conferences and presentations, it is widely practiced for speakers to give explanations while displaying slides on a display device or the like. When giving a presentation in two languages or the like, after the explanation in the first language, the second language translated from the first language is played by a speaker or headphones.

For example, in a presentation in Japanese and English, after the speaker explains the slides in Japanese, the voice translated from the explanation into English is played from the speaker. Then, when the voice in English ends, the speaker instructs the slide feed and the next slide is displayed. In the case of headphones, the translated English voice is played from the headphones during the explanation of Japanese by the speaker.

As for this kind of presentation technique, for example, there is one that records audio data and a pointer locus in a presentation to reduce a synchronization shift between the audio data and the pointer locus during reproduction (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2009-157677

However, in the above-mentioned bilingual presentation technology, when the speaker cannot understand the translated voice, the timing of ending the translated voice may not be known. Therefore, a situation occurs in which a slide feed instruction is issued before the end of the translated voice, or a slide feed instruction is issued after the translation voice ends, and smooth slide feed, in other words, a smooth presentation cannot be performed. I have the problem I said.

Further, even when the speaker sends a translation through headphones during the slide explanation, the speaker does not know the end timing of the translated voice, and there is a problem that the slide feed cannot be smoothly performed as described above.

An object of the present invention is to provide a technique capable of giving a smooth presentation by performing accurate slide feed in a presentation accompanied by translation.

The above and other purposes and novel features of the present invention will become apparent from the description and accompanying drawings herein.

A brief outline of the typical inventions disclosed in the present application is as follows.

That is, a typical display device has a video output unit, an audio input unit, and a control unit. The video output unit outputs the video to be displayed. A voice signal is input to the voice input unit. The control unit extracts voice from the voice signal input to the voice input unit, outputs the voice, and reproduces the translated voice signal obtained by translating the voice as the translated voice. Then, the control unit suspends the switching of the video displayed by the video output unit during the period during which the translated audio is being reproduced.

Further, the control unit extracts voice from the voice signal input to the voice input unit based on the recognition signal. The recognition signal is a signal that is input from the outside and indicates that the pre-translational explanation for the video has been completed.

In particular, the control unit adds a display symbol indicating that the translated sound is being played to the video output from the video output unit during the period during which the translated sound is being played.

Among the inventions disclosed in the present application, the effects obtained by typical ones will be briefly described as follows.

A smooth presentation can be realized.

It is explanatory drawing which shows an example of the structure in the image display system by Embodiment 1. FIG. It is a block diagram which shows an example of the structure in the projection apparatus and translation server which the image display system of FIG. 1 has. It is explanatory drawing which shows an example of the operation and speaker operation by the image display system of FIG. It is explanatory drawing which shows an example of the structure in the video display system by embodiment. It is explanatory drawing which shows an example of the operation and speaker operation by the image display system of FIG. It is a block diagram which shows an example of the structure in the projection apparatus and translation server which the image display system according to Embodiment 3 has. It is explanatory drawing which shows an example of the operation and speaker operation by the image display system of FIG. It is explanatory drawing which shows an example of the operation and the speaker operation in the video display system of FIG. 6 according to Embodiment 4. FIG. It is explanatory drawing which shows an example of the display of the slide screen by Embodiment 5. It is explanatory drawing which shows an example of the image display system which has the function of changing the pointer position of FIG. 9 according to the translated word order. It is explanatory drawing which shows an example of the structure in the image display system according to Embodiment 6. It is explanatory drawing which shows an example of the structure in the portable device which the image display system of FIG. 11 has. It is explanatory drawing which shows an example of the structure in the image display system according to Embodiment 7.

In the following embodiments, when necessary for convenience, the description will be divided into a plurality of sections or embodiments, but unless otherwise specified, they are not unrelated to each other, and one is the other. It is related to some or all of the modified examples, details, supplementary explanations, etc.

Further, in the following embodiments, when the number of elements (including the number, numerical value, quantity, range, etc.) is referred to, when it is specified in particular, or when it is clearly limited to a specific number in principle, etc. Except, the number is not limited to the specific number, and may be more than or less than the specific number.

Furthermore, in the following embodiments, the components (including element steps, etc.) are not necessarily essential unless otherwise specified or clearly considered to be essential in principle. Needless to say.

Similarly, in the following embodiments, when the shape, positional relationship, etc. of the constituent elements are referred to, the shape is substantially the same, except when it is clearly stated or when it is considered that it is not clearly the case in principle. Etc., etc. shall be included. This also applies to the above numerical values and ranges.

Further, in all the drawings for explaining the embodiment, the same members are, in principle, given the same reference numerals, and the repeated description thereof will be omitted. In addition, in order to make the drawing easy to understand, hatching may be added even if it is a plan view.

<Display system configuration example>
Hereinafter, embodiments will be described in detail.

FIG. 1 is an explanatory diagram showing an example of a configuration in a video display system according to the first embodiment.

The video display system is a translation presentation system that gives presentations in two languages. The video display system translates the speaker's audio into different languages and plays it back.

The slide feed of the image is suspended during the playback of the translated voice, and the slide feed is performed after the playback of the translated voice is completed. Then, the speaker is notified of the end of the translated voice, and the next slide explanation is started.

As shown in FIG. 1, this video display system includes a video source 12, a projection device 13, a controller 16, a microphone 17, a speaker 18, and a translation server 20.

The video source 12 is, for example, a personal computer or the like, and outputs a video signal for presentation or the like. The microphone 17 collects the voice of the speaker and transmits it to the projection device 13. The microphone 17 and the projection device 13 may be connected either wirelessly or by wire.

The projection device 13 projects and displays the video signal output from the video source 12 on the screen 15, and reproduces the speaker's voice transmitted from the microphone 17 on the speaker 18.

Further, the projection device 13 is connected to a network 19 such as the Internet or a communication line, extracts a speaker's voice element transmitted from the microphone 17, and transmits the voice element to the translation server 20 via the network 19. The translation server 20 translates the voice transmitted via the network 19 into a language specified in advance.

The translation data translated by the translation server 20 is transmitted to the projection device 13 via the network 19. The projection device 13 converts the translation data received from the translation server 20 into an audio signal and outputs it to the speaker 18 for reproduction.

The actuator 16 indicates a speaker voice period indicating the period being explained by the speaker. The instruction of the speaker voice period is given, for example, by operating a button provided on the actuator 16. By operating this button, a recognition signal is generated.

Based on this recognition signal, the translation target period of the speaker voice can be recognized. The recognition of the translation target period based on the recognition signal is performed by the sound recognition period indicator 235 of FIG. As a result, it can be recognized that the explanation before translation has been completed.

Further, the recognition signal can be used to determine the timing at which the extracted voice element is transmitted to the translation server 20. The generation of the recognition signal may be voice recognition, gesture input, or the like. Moreover, you may combine them. The actuator 16 and the projection device 13 may be connected either wirelessly or by wire.

For example, after determining the approximate timing by button operation or gesture, the speaker voice is translated from the time when the speaker voice is first detected to the time when the speaker voice is last detected before button operation or gesture input. The period. As a result, the translation language reproduction time can be shortened.

The actuator 16 may be provided with a mute input. By this mute input, it is possible to specify a voice portion that is not to be translated from the speaker's voice. As a result, the presentation time, which takes nearly twice as long as one language, can be shortened by alternately playing the original language and the translated language.

Further, the actuator 16 may also have the function of a laser pointer that points to the explanation location of the image projected on the screen 15. The actuator 16 may have a function of sending the pointer position to the image source 12 or the projection device 13 as an electronic pointer and superimposing the pointer on the image.

<Configuration example of projection device and translation server>
FIG. 2 is a block diagram showing an example of the configuration of the projection device 13 and the translation server 20 included in the video display system of FIG.

The projection device 13 includes a communication unit 231,234,238, a video input unit 232, a display unit 241, a sound recognition period indicator unit 235, a voice input unit 236, a voice element extraction unit 237, a translated voice reproduction unit 239, and a voice switching unit. Has 240.

The video signal output from the video source 12 is input to the video input unit 232. The display unit 241 which is the video output unit projects and displays the video signal input from the video input unit 232 on the screen 15 of FIG.

The voice input unit 236 receives the speaker voice collected by the microphone 17. The voice switching unit 240 outputs either the speaker voice received by the voice input unit 236 or the translated voice reproduced by the translated voice reproduction unit 239 so as to be reproduced by the speaker 18 based on the translation reproduction timing signal described later. To do.

The voice element extraction unit 237 constituting the control unit extracts the voice element of the speaker voice input to the voice input unit 236. The communication unit 238 communicates with the translation server 20 via the network 19. The voice element extracted by the voice element extraction unit 237 is transmitted from the communication unit 238 to the translation server 20 via the network 19 for voice recognition.

The communication unit 231 communicates with the actuator 16. The instruction of the speaker voice period by the actuator 16 is transmitted to the sound recognition period instruction unit 235 via the communication unit 231. The sound recognition period indicator 235 generates a voice recognition timing signal, a translation / reproduction timing signal, a slide feed timing signal, and the like. The voice recognition timing signal, which is a recognition signal, is generated based on the above-mentioned recognition signal.

The voice recognition timing signal is a signal output when the voice element extraction unit 237 extracts the voice element of the speaker voice. The translation / playback timing signal is a signal output when the voice switching unit 240 reproduces the translated voice.

The slide feed timing signal is a signal that causes the image source 12 to display the next slide. The slide feed timing signal is output to the video source 12 via the communication unit 231.

The translation server 20 has a communication unit 201, a voice characterization unit 202, and a translation voice unit 203. The communication unit 201 communicates with the communication unit 238 of the projection device 13. As described above, the voice element transmitted from the projection device 13 is received by the communication unit 201 via the network 19.

The voice character conversion unit 202 converts the voice element received by the communication unit 201 into characters. The translation voice unit 203 translates the characters converted by the voice character conversion unit 202 to generate translation data. The translation data generated by the translation voice unit 203 is transmitted from the communication unit 201 to the projection device 13.

The translation data received by the communication unit 238 is output to the translation voice reproduction unit 239. The translated voice reproduction unit 239 that constitutes the control unit reproduces the translated data as an audio signal.

The reproduced translated voice signal is switched by the voice switching unit 240 after the explanation of the Japanese voice, and the voice is reproduced by the speaker 18. The switching timing is switched based on the translation / playback timing signal described above.

The flow of speaker operation and device operation will be described below.

<Operation example of video display system>
FIG. 3 is an explanatory diagram showing an example of an operation and a speaker operation by the video display system of FIG.

The “display” shown on the left side of FIG. 3 shows the images # 1 and the images # 2 corresponding to the explanatory slides in chronological order.

The "voice" shown on the right side of the "display" indicates the speaker's voice and the translated English voice in chronological order. # 1 Japanese and # 2 Japanese are the voices of the speaker. Also, # 1 English and # 2 English are translated English voices.

"Speaker operation and device operation" shown on the right side of "voice" indicates speaker operation and device operation. (U *) indicates an operation by the speaker. (A *) indicates the operation by the video display system.

First, the speaker inputs the timing of starting the talk into the actuator 16 and starts the translation presentation (u1). When the video signal output from the video source 12 is output to the projection device 13, the video # 1 is displayed on the screen 15. As a result, the speaker 11 starts the explanation of the video # 1 in Japanese (u2).

The Japanese voice explained by the speaker is collected by the microphone 17. At this time, the voice switching unit 240 is switched so that the voice collected by the microphone 17 is output from the speaker 18. Therefore, the collected sound is amplified by the speaker 18 connected to the projection device 13.

Then, when the explanation of the video # 1 in Japanese is completed, the speaker 11 inputs the end timing of the explanation to the actuator 16 (u3). If the presentation is in Japanese only, the projection device 13 instructs the image source 12 to provide the next image # 2 at this end timing.

However, in the presentation here, since there is translated audio after the explanation in Japanese, the projection device 13 gives a slide feed instruction requesting video # 2 from the video source 12 until the reproduction of the translated audio is completed. Does not output a certain slide feed timing signal.

The Japanese voice elements explained by the speaker 11 are sequentially extracted by the voice element extraction unit 237 of the projection device 13 and transmitted to the translation server 20 via the network 19. The translation server 20 translates Japanese into English based on the transmitted voice elements. The translated data translated into English is transmitted to the projection device 13 via the network 19.

The translated data is temporarily stored in, for example, a storage unit (not shown) included in the translated voice reproduction unit 239. Upon the end of the Japanese explanation for the video # 1 (u3), the translated audio reproduction unit 239 reproduces the translated data. The English voice reproduced by the translated voice reproduction unit 239 is output from the voice switching unit 240 and loudened by the speaker 18 (a1).

The sound recognition period instruction unit 235 generates a translation / reproduction timing signal based on the instruction of the speaker voice period by the actuator 16. The translation / playback unit 239 transmits the generated translation / playback timing signal to the voice switching unit 240.

The voice switching unit 240 switches so that the voice to be reproduced by the translated voice reproduction unit 239 is output based on the translation / reproduction timing signal, and outputs the voice reproduction start signal to the translation voice reproduction unit 239. Upon receiving the voice reproduction start signal, the translation voice reproduction unit 239 reproduces the translation data temporarily stored in the above-mentioned storage unit or the like.

When the reproduction of the translated English sound for the video # 1 is completed (a2), the sound recognition period indicator 235 of the projection device 13 instructs the video source 12 to output the video # 2, which is the next slide (the video # 2). a3).

When the image source 12 outputs the image signal of the image # 1, the projection device 13 displays the image # 2 output by the image source 12 on the screen 15. After confirming that the displayed video has been switched from video # 1 to video # 2, the speaker understands that the English translation audio has ended, and begins the explanation of video # 2 in Japanese.

Hereinafter, since (u5) (a4) (a5) (a6) are the same as (u3) (a1) (a2) (a3) described above, the description thereof will be omitted.

The audience listening to the speaker's audio and the audience listening to the translated audio will listen while watching the same video. The video may be written in English, the speaker audio may be in Japanese, and the translated audio may be in English or Chinese.

Compared to audio, video has many graphic elements other than characters, so even if the text in the video cannot be understood, it can be supplemented with audio. In addition, both the speaker's voice and the translated voice may be written together in the video. Writing both languages has the advantage of making it easier for each viewer to understand.

In this way, the translated voice and the slide to be explained can be synchronized so that the slide feed is stopped during the playback of the translated voice and the next slide is played when the playback of the translated voice is completed.

As described above, the speaker can visually confirm the switching of the slides after the translation voice is completed, so that the timing for starting the next slide explanation can be easily grasped. As a result, a smooth translation presentation can be realized.
(Embodiment 2)
<Overview>
In the first embodiment, the speaker voice and the translated voice are alternately played from the speaker 18, but in the second embodiment, the speaker voice is output by the speaker and the translated voice is output by the headphones. The technique of flowing is explained. As a result, it is possible to avoid turbidity between the speaker voice and the translated voice and shorten the presentation time.

<Configuration example of video display system>
FIG. 4 is an explanatory diagram showing an example of the configuration in the video display system according to the second embodiment.

The video display system of FIG. 4 differs from the video display system of FIG. 1 of the first embodiment in that the translation voice reproduction device 21, the voice distribution device 22, and the headphones 30 are newly added. ..

In the video display system of FIG. 1, the translated sound is reproduced by the projection device 13, but in the video display system of FIG. 4, the translated sound is reproduced by the translated sound reproduction device 21 connected to the network 19.

The translation data is received from the translation server 20 via the network 19. The translated voice signal reproduced by the translated voice reproducing device 21 is output to the voice distribution device 22. The voice distribution device 22 wirelessly distributes the translated voice communication to the headphones 30 used by the audience. The distribution of the translated voice message by the voice distribution device 22 is not limited to wireless communication, and may be wired communication.

Since the other configurations are the same as those in FIGS. 1 and 2 of the first embodiment, the description thereof will be omitted. Further, each function of the translated voice reproducing device 21 and the voice distribution device 22 of FIG. 4 may be provided in the projection device 13 included in the video display system of FIG. In that case, the function of the translated audio reproduction device 21 can be performed by the translated audio reproduction unit 239 of FIG.

<Operation example of video display system>
FIG. 5 is an explanatory diagram showing an example of an operation and a speaker operation by the video display system of FIG.

The “display” shown on the left side of FIG. 5 shows the video # 1 and the video # 2 corresponding to the explanatory slides in chronological order. The "speaker voice" shown on the right side of the "display" shows the speaker voice in chronological order, and the "translated voice" shown on the right side of the "speaker voice" is the translated voice being played. Is shown in chronological order.

The "speaker operation and device operation" shown on the right side of the "translated voice" indicates the speaker operation and device operation. (U *) indicates an operation by the speaker. (A *) indicates the operation by the video display system.

In FIG. 3 of the first embodiment, the start of reproduction of the translated voice (a1) is after the end of the speaker explanation (u3). On the other hand, in FIG. 5, after the start of the speaker's explanation (u2), the playback starts as soon as the translated voice data is prepared, that is, slightly later than the speaker's voice.

The translated audio is played by the headphones 30 and is inaudible to an audience who does not need the translated audio. Therefore, the speaker's voice is reproduced without being interrupted.

Further, in FIG. 5, even if the speaker instructs the end of the explanation (u3), the slide feed is not performed until the translated voice ends (a2). When the translated sound is completed (a2), the sound recognition period indicator 235 of the projection device 13 sends the slide feed timing signal to the video source 12 (a3).

As a result, the slide of video # 2 is displayed. When the speaker confirms that the slide of the displayed video # 2 is displayed, the speaker starts the next explanation (u4).

In this way, a smooth translation presentation can be realized. Further, since the translated voice can be played through the headphones while the speaker's explanation voice is output by the speaker, the presentation time can be shortened while avoiding the turbidity between the speaker voice and the translated voice.

(Embodiment 3)
<Overview>
In the first and second embodiments, the end of the translated voice is notified to the speaker by switching the slides, but in the third embodiment, the translated voice is played while the translated voice is played. A technique for superimposing and displaying characters and marks indicating that the above will be described.

During playback of the translated voice, the characters and symbols indicating it are superimposed and displayed on the screen. This allows the speaker to clearly indicate even if the differences between the slides are small. In addition, it is possible to clearly inform the speaker and the audience that the translated voice is being played. By clearly indicating that the translated voice is being played, there is an advantage that one slide explanation can be divided into small parts and explained.

<Configuration example of projection device and translation server>
FIG. 6 is a block diagram showing an example of the configuration of the projection device 13 and the translation server 20 included in the video display system according to the third embodiment.

Since the configuration of the video display system is the same as that of FIG. 1 of the first embodiment, the description thereof will be omitted.

The difference between the projection device 13 shown in FIG. 6 and the projection device 13 of FIG. 2 is that the character superimposing portion 233 is newly provided. Since other connection configurations are the same as those of the projection device 13 of FIG. 2, description thereof will be omitted. Further, the translation server 20 of FIG. 6 is the same as the translation server 20 of FIG. 2, so the description thereof will be omitted.

The character superimposing unit 233 constituting the control unit superimposes characters and symbols indicating that translation / reproduction is in progress on the video signal input from the video source 12 to the video input unit 232. The character superimposition unit 233 superimposes characters, symbols, and the like indicating that translation / reproduction is in progress on the video based on the translation / reproduction timing signal generated by the sound recognition period indicating unit 235.

<Operation example of video display system>
FIG. 7 is an explanatory diagram showing an example of operation and speaker operation by the video display system of FIG.

The “display” shown on the left side of FIG. 7 shows the images # 1 and the images # 2 corresponding to the explanatory slides in chronological order. The "voice" shown on the right side of the "display" indicates the speaker's voice and the translated English voice in chronological order. # 1 Japanese and # 2 Japanese are the voices of the speaker. Also, # 1 English and # 2 English are translated English voices.

"Speaker operation and device operation" shown on the right side of "voice" indicates speaker operation and device operation. (U *) indicates an operation by the speaker. (A *) indicates the operation by the video display system.

In FIG. 7, the major difference from FIG. 3 of the first embodiment is that a character indicating that the translated voice is being played back, for example, <translating> is written, and at the end of the voice explanation. The point is that instructions are input according to whether or not slide feed is possible.

The speaker operations (u1) and (u2) and the display device operation (a1) are the same as those in FIG. When the speaker instructs the video # 1 to pause the explanation (u13), the projection device 13 superimposes and displays the characters and marks of "translating" on the video # 1 (b1).

Then, after the end of the translated voice on the video # 1 (a2), the video superposition of the characters or marks "translating" is stopped (b2). At this time, the explanation pause operation (u13), which is the operation of suspending the explanation for the video # 1, does not include the slide feed instruction. Therefore, it differs from FIG. 3 in that the slide feed timing signal is not output to the video source 12.

When the speaker confirms the erasure (b2) of the character or mark "translating", the speaker starts the explanation # 1a following the video # 1 (u14). When the speaker gives an instruction (u15) to feed the slide from the actuator 16 together with the instruction to end the explanation of the image # 1, the projection device 13 superimposes the characters or marks of "translating" on the image # 1. Display (b3).

Then, after the end of the translated voice (a5), the video superimposition of the character or mark "translating" is stopped (b4). At the same time, the slide feed timing signal is output to the video source 12 as in FIG. 3 (a6).

In this way, when the explanation is given a plurality of times using one slide, it is preferable to indicate whether or not the slide feed is performed in the instruction input at the end of the explanation. As described above, the explanation pause button without slide feed and the explanation end button with slide feed may be attached. Further, the slide feed and the explanation end buttons may be made independent, and the slide feed attachment explanation end may be indicated when both buttons are input.

As described above, after the speaker instructs the end of the slide feed instruction explanation, the slide feed during the translation audio playback is stopped by temporarily suspending the transmission of the slide feed command, and the corresponding video display is displayed to the translated audio audience. You can continue.

By outputting the slide feed timing signal to the video source 12 and displaying the next slide after the translation audio reproduction is completed, the timing for the speaker to resume the explanation can be grasped.

Also, if the next slide is the same image as the previous slide, it is difficult to grasp the timing, but you can check that the text and marks during the translated audio playback are erased, so the speaker will explain. It has the advantage of being able to reliably tell when to resume.

Furthermore, in the case of the explanation end input without slide feed, the speaker should grasp the timing to resume the explanation by seeing the text and the mark being erased during the translated voice playback without switching to the next slide. Can be done. The speaker can enter the continuation of the slide by stopping the video and the text and marks in the translated audio output disappear.

While the translated voice is being played, the slide feed is stopped and the translated voice and the slide to be explained are synchronized to realize a smooth translation presentation. Further, the speaker has an advantage that he / she can see the switching of slides after the end of the translated voice and grasp the timing to start the next slide explanation.

(Embodiment 4)
<Overview>
In the first embodiment, the display during the explanation of the speaker and the display during the reproduction of the translated voice are common. In the fourth embodiment, a technique of replacing the characters in the displayed image with the translated characters during the reproduction of the translated voice will be described. This makes it easier for the audience to understand the translated voice.

<Operation example of video display system>
FIG. 8 is an explanatory diagram showing an example of an operation and a speaker operation in the video display system of FIG. 6 according to the fourth embodiment. Since the configuration of the video display system is the same as that of FIG. 6 of the third embodiment, the description thereof will be omitted.

Also in FIG. 8, the “display” shown on the left shows the video # 1 and the video # 2 corresponding to the explanatory slides in chronological order. The "voice" shown on the right side of the "display" indicates the speaker's voice and the translated English voice in chronological order. # 1 Japanese and # 2 Japanese are the voices of the speaker. Also, # 1 English and # 2 English are translated English voices.

"Speaker operation and device operation" shown on the right side of "voice" indicates speaker operation and device operation. (U *) indicates an operation by the speaker. (A *) indicates the operation by the video display system.

First, when the start (u1) of the translation presentation is instructed, the projection device 13 extracts characters in the video input from the video source 12 and transmits the characters to the translation server 20 via the network 19. The process of extracting characters in the video is performed by, for example, the character superimposing unit 233 of FIG. Alternatively, the projection device 13 of FIG. 6 may be newly provided with the character extraction unit 434 shown in FIG. 10 to be described later. The character extraction unit 434 executes a process of extracting characters in the video.

Then, the translation server 20 starts translating the characters in the received video (c1). When the speaker's explanation is completed and the instruction to end the explanation is output from the actuator 16 (u3), the projection device 13 overwrites the characters translated by the translation server 20 on the character extraction area and projects them on the screen 15. (C2).

Also in this process, for example, the character superimposing unit 233 in FIG. 6 shall perform this process. Alternatively, the projection device 13 of FIG. 6 may be newly provided with the translated character overwriting unit 431 shown in FIG. 10 described later. The translated character overwriting unit 431 overwrites the characters translated by the translation server 20 in the character extraction area.

As described above, the video overwritten with the translated characters can be displayed during the playback of the translated voice. After the end of the translated sound (a2), the projection device 13 outputs the slide feed timing signal to the image source 12 and feeds it (a3) to display the next slide image as in FIG.

As a result, the translated voice can be heard together with the screen displayed in the translated characters of the same language as the translated voice. As a result, the understanding of the audience can be enhanced.

(Embodiment 5)
<Overview>
In presentations, pointers are often used in the explanation of the speaker. In the fifth embodiment, a technique of moving the position of the pointer pointed by the speaker according to the translated word order during the reproduction of the translated voice will be described.

In the fourth embodiment, the characters of the displayed video are translated and the characters are overwritten with the translated characters during the reproduction of the translated voice. However, depending on the word order of the translated voice, it may differ from the word order of the speaker voice.

<Example of slide display>
FIG. 9 is an explanatory diagram showing an example of display of a slide screen according to the fifth embodiment.

The display screens 31 and 32 shown on the left side of FIG. 9 show an example of the display screen at the time of explaining the speaker. Further, the display screens 33 and 34 shown on the right side of FIG. 9 show an example of the display screen displayed during the reproduction of the translated voice.

On the display screens 31 and 32, a pencil 311 is displayed on the left side, and a "pencil" is displayed below the pencil 311 as an explanatory text. A box 312 is displayed on the right side of the pencil 311 and a "box" is displayed below the box 312 as an explanatory text.

On the other hand, in the display screens 33 and 34 during translation reproduction shown on the right side of FIG. 9, "pencil" which is the explanation below the pencil 311 is translated and replaced with "pencil". Similarly, in the box 312. The description below, "box," has been translated and replaced with "Box."

Here, if the speaker's Japanese explanatory voice is, for example, "put a pencil on the box" as shown in the upper left side of FIG. 9, the translated sentence is in the upper right side of FIG. As shown, it becomes "Put a pencil on the box." In this case, in Japanese sentences, the word order of "box" and "pencil" is "box" and "pencil".

On the other hand, in the case of an English sentence translated from the above Japanese, the order is "pencil" "Box", which is the reverse word order of Japanese. Therefore, when the speaker explains using the display screens 31 and 32, the pointer pointing location moves from the pointer insertion position 313 to the pointer insertion position 323 as shown by the alternate long and short dash line on the left side of FIG. become.

On the other hand, when the display screens 33 and 34 are used during playback of the translated voice, as shown by the alternate long and short dash line on the right side of FIG. 9, the pointer points to the translated voice, that is, the translation word order. It will be easier to understand if it is moved from the pointer position 333 to the pointer position 343.

<Configuration example of video display system>
FIG. 10 is an explanatory diagram showing an example of a video display system having a function of changing the pointer position of FIG. 9 according to the translated word order.

The difference between the video display system of FIG. 10 and the video display system of FIG. 6 is that the camera 440 is newly provided. Further, the projection device 13 is newly provided with a character extraction unit 434, a translated character overwriting unit 431, a failure diagnosis unit 435, a microphone array 436, a speaker analysis unit 437, and a pointer position determination unit 439. In the translation server 20, a minutes preparation unit 402 and a character translation unit 401 are newly provided.

The microphone array 436 is composed of a plurality of microphones and collects the voice of the speaker. Here, the microphone array 436 is provided with directivity in consideration of the case where the projection device 13 is away from the speaker.

The speaker analysis unit 437 analyzes the position of the speaker based on the sound collection result of the microphone array 436. The speaker voice extraction unit 438 extracts the speaker voice. This is to separate the speaker voice from the noise of the projection device 13 and the surroundings. For example, the speaker voice is extracted by a noise removal filter or a separation technique using the characteristics of each frequency characteristic.

The speaker voice extracted by the speaker voice extraction unit 438 is transmitted to the translation server 20 after the voice elements are extracted by the voice element extraction unit 237.

In the translation server 20, the speaker voice converted into characters by the voice characterization unit 202 is translated by the translation voice unit 203. The translation data, which is the translation result, is transmitted to the projection device 13. The translation data consists of data including the speaker voice and the word order of the translated voice.

The translated voice reproduction unit 239 reproduces the received translation data into a translated voice signal, and the translated voice is reproduced by the speaker 18 via the voice switching unit.

The video of the video source 12 received by the video input unit 232 is input to the character extraction unit 434 and the translated character overwrite unit 431, respectively.

The character extraction unit 434 extracts characters in the video and transmits the characters together with the position information of the characters to the translation server 20. Then, it is sent back to the projection device 13 translated by the character translation unit 401.
When the translated character overwriting unit 431 receives the translated character, the translated character overwriting unit 431 overwrites the translated character on the corresponding character area based on the extracted character area information obtained from the character extraction unit 434. Overwriting with the translated characters is as described in the fourth embodiment.

The camera 440 captures the pointer pointing positions 313 and 323 of FIG. 9 pointed to by the speaker. The pointer position determination unit 439 determines the position, size, time, etc. of the pointer insertion positions 313 and 323 based on the shooting result taken by the camera 440, and a memory (not shown) provided in the pointer position determination unit 439 is provided. Remember it in such as.

The pointer synchronization synthesis unit 432 generates pointer insertion positions 333 and 343 during reproduction of the translated voice according to the determination result stored in the pointer position determination unit 439. Then, based on the word order information included in the translation data, it is superimposed on the video in accordance with the word order of the translated voice and output from the display unit 241.

As described above, the pointer can be displayed on the display screen according to the word order of the translated voice. This can enhance the understanding of the audience.

In the above description, an example in which the pointer position is detected by the camera 440 is shown, but when the speaker operates the actuator and superimposes it on the image as an electronic pointer, the position information from the actuator is used. The position of the electronic pointer may be detected.

Further, the voice of the speaker transcribed by the voice transcribing unit 202 may be recorded by, for example, the minutes creation unit 402 and recorded as the minutes. Further, in the question and answer session after the speaker's explanation, the speaker analysis unit 437 analyzes the questioner's voice, the speaker voice extraction unit 438 extracts the questioner's voice, and the translation server 20 outputs the questioner's voice. You may want to record it.

When the questioner is the language to which the speaker voice is translated, the translation server 20 may perform translation opposite to the speaker voice translation and reproduce the questioner's voice. In this case, the speaker and the questioner can have a question and answer session through dialogue translation, which can improve convenience.

Further, in the first to fifth embodiments, the example in which the slide feed command to the video source 12 is temporarily stopped to hold the video during the translated audio reproduction has been described, but instead of the pause of the slide feed command, Then, the video for translated audio may be stored in a storage device (not shown), and the stored video may be displayed. Coordination control can be simplified when the operator for operating the slide feed and the indicator for instructing the speaker voice input timing for translation are independently configured.

(Embodiment 6)
<Overview>
The video display system according to the first to fifth embodiments is configured to use, for example, a personal computer as a video source. In the sixth embodiment, a technique of substituting a video source with a mobile device will be described.

<Configuration example and operation example of video display system>
FIG. 11 is an explanatory diagram showing an example of the configuration in the video display system according to the sixth embodiment.

The image display system of FIG. 11 differs from the image display system of FIG. 1 in that it is replaced with a portable device 56 having the functions of the image source 12 and the actuator 16 of FIG. The portable device 56, which is an output device, comprises, for example, a smartphone or a tablet terminal, and can be connected to the network 19.

<Example of mobile device configuration>
FIG. 12 is an explanatory diagram showing an example of the configuration of the portable device 56 included in the video display system of FIG.

As shown in FIG. 12, the portable device 56 includes an audio input unit 57, a control unit 58, and a video signal output unit 59. The voice input unit 57, which is the second voice input unit, receives the voice of the microphone 17 using, for example, Bluetooth.

The video signal output unit 59 outputs the video signal displayed by the projection device 13 to the projection device 13. The video signal is transmitted by radio such as wireless LAN, Bluetooth®, or Miracast®. Alternatively, the portable device 56 and the projection device 13 may be connected by wire.

The control unit 58, which functions as a voice extraction output unit and a translation voice conversion unit, extracts a voice element from the voice received by the voice input unit 57 and transmits it to the translation server 20. Further, the control unit 58 receives the translation data translated by the translation server 20, converts the received translation data into an audio signal, and transfers the received translation data to the projection device 13.

The projection device 13 receives the transferred voice signal, reproduces it, and outputs the translated voice from the speaker 18. Of course, the voice signal of the speaker collected by the microphone 17 may also be transferred from the control unit 58 of the portable device 56 to the projection device 13 to alternately output the speaker voice and the translated voice.

Further, the control unit 58 has the function of the actuator 16 of FIG. 1 described above. Specifically, it is a function of instructing a speaker voice period, which is a period explained by the speaker. The instruction of the speaker voice period is given, for example, by operating a button provided on the mobile device 56. By operating the button, a recognition signal is generated.

The portable device 56 can recognize the translation target period of the speaker voice based on the recognition signal. As a result, it can be recognized that the explanation before translation has been completed. Further, it is possible to determine the timing at which the voice element extracted by the recognition signal is transmitted to the translation server 20. In addition, the portable device 56 may also have a function such as an electronic pointer.

The operations and various functions of the control unit 58 may be realized by, for example, an application installed in the mobile device 56. When realized by an application, a CPU (Central Prosessing Unit) (not shown) included in the control unit 58 or the mobile device 56 is stored in a program storage memory (not shown) also provided in the mobile device 56. Performs processing based on the software.

The portable device 56 outputs the video signal displayed by the projection device 13 to the projection device 13. The video signal is transmitted by radio such as wireless LAN, Bluetooth®, or Miracast®. Alternatively, the portable device 56 and the projection device 13 may be connected by wire.

Further, the portable device 56 has a function of receiving the voice of the microphone 17 using Bluetooth or the like, extracting the voice element, and transmitting the voice element to the translation server 20. Then, the mobile device 56 receives the translation data translated by the translation server 20. The received translation data is converted into a voice signal by the portable device 56 and transferred to the projection device 13.

The projection device 13 receives the transferred voice signal, reproduces it, and outputs the translated voice from the speaker 18. Of course, the voice signal of the speaker collected by the microphone 17 may also be transferred from the portable device 56 to the projection device 13 to alternately output the speaker voice and the translated voice.

The portable device 56 has the function of the actuator 16 of FIG. 1 described above. Specifically, it is a function of instructing a speaker voice period, which is a period explained by the speaker. The instruction of the speaker voice period is given, for example, by operating a button provided on the mobile device 56. By operating the button, a recognition signal is generated.

The portable device 56 can recognize the translation target period of the speaker voice based on the recognition signal. As a result, it can be recognized that the explanation before translation has been completed. Further, it is possible to determine the timing at which the voice element extracted by the recognition signal is transmitted to the translation server 20. In addition, the portable device 56 may also have a function such as an electronic pointer.

The operations and various functions of these mobile devices 56 are realized by, for example, applications installed in the mobile device 56. When realized by an application, a CPU (Central Prosessing Unit) (not shown) of the mobile device 56 is based on program-type software stored in a program storage memory (not shown) also provided in the mobile device 56. , Execute the processing of each function. Alternatively, it may be configured by hardware having the above-mentioned functions.

As described above, the video source 12 and the operating device 16 can be eliminated, so that the configuration of the video display system can be simplified. Further, the projection device 13 can be miniaturized by the portable device 56 extracting the voice element or the like.

Further, by realizing the various functions of the portable device 56 described above by an application, the image source 12, the operating device 16, and the like can be eliminated, so that the cost of the image display system can be reduced.

(Embodiment 7)
<Configuration example and operation example of video display system>
FIG. 13 is an explanatory diagram showing an example of the configuration in the video display system according to the seventh embodiment.

The image display system shown in FIG. 13 is provided with two projection devices 13 and 13a. The projection device 13 is a projection device used for explaining the speaker, and the projection device 13a is a projection device used for reproducing the translated voice.

The screens 15 and 15a and the speakers 18 and 18a are provided accordingly. The screen 15 displays an image projected by the projection device 13. The screen 15a displays an image projected by the projection device 13a. The speaker 18 enhances the speaker's explanatory voice output from the projection device 13, and the speaker 18a enhances the translated voice output from the projection device 13a.

The video display system shown in FIG. 13 allows the speaker explanation and the reproduction of the translated voice to proceed almost simultaneously, but the translated voice reproduction is often delayed from the speaker explanation due to the translation process. Therefore, it is useful to suppress slide feed during translation audio reproduction.

The projection device 13 receives the video signal output from the video source 12, and displays a video for explaining the speaker on the screen 15. The projection device 13a receives a video signal from the projection device 13 and displays a video for explaining the translated audio on the screen 15a for the translated audio viewer.

As shown in the fourth embodiment, the display image of the projection device 13a may be an overwritten character extracted from the image signal of the image source 12 and translated. Character extraction and translation may be performed by either the projection device 13 or the projection device 13a.

As shown in the fifth embodiment, the display image of the projection device 13a is a superimposed display in which the pointer pointed by the speaker on the screen 15 is delayed by the translation delay time, or is superimposed and displayed in the order of the translated spoken words. It may be something to do. The superimposed display of the pointer may be performed by either the projection device 13 or 13a.

Even if the speaker gives a slide feed instruction, the display image of the projection device 13 is configured not to shift to the next slide during the reproduction of the translated voice. For example, during playback of the translated audio, (1) the slide feed command is held, (2) the projection device 13 stores the image and continues the image output to the projection device 13a, and (3) the projection device 13a is the image. It can be realized by a technique such as storing and holding a video output.

The translated voice is sent from the projection device 13a to the audience listening to the translation through the speaker 18a. The translated voice can be realized by the projection device 13a having the function of the translated voice reproducing device 21 shown in FIG. 4 of the second embodiment.

Alternatively, it can be realized by having the translation sound reproduction unit 239 of the third embodiment and transmitting the translated sound to the projection device 13a together with the video signal.

In this way, using the two projection devices 13 and 13a. Even if the speaker explanation and the translation audio are played back almost at the same time, by continuing to display the video corresponding to the translated audio, at least for the audience listening to the translated audio, during the playback of the translated audio. , Translated audio and video can be synchronized, and a presentation that is easy for the audience to understand can be realized.

Although the embodiments 1 to 7 have been described above by taking a still image presentation as an example, the same effect can be obtained even with an animation or a moving image. In the case of alternating switching between the speaker explanation and the translated audio, instead of holding the video switching, the video playback may be instructed to return to the beginning of the slide video at the start of the playback of the translated audio.

In the seventh embodiment, the video may be sent and displayed for the translation delay time, and the video and the translated audio may be synchronized.

Although the invention made by the present inventor has been specifically described above based on the embodiment, the present invention is not limited to the embodiment and can be variously modified without departing from the gist thereof. Needless to say.

In the first to seventh embodiments, the projection device, which is a projection type image display device for projecting an image on the screen 15, has been described as an example. However, the device for displaying the image is not limited to the projection device, and a liquid crystal or an OLED (Organic). It may be a general display device including a direct-view type display such as Light Emitting Diode).

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations.

It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. .. Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

12 Video source 13 Projection device 13a Projection device 15 Screen 15a Screen 16 Operator 17 Microphone 18 Speaker 18a Speaker 19 Network 20 Translation server 21 Translation audio playback device 22 Audio distribution device 30 Headphones 56 Portable device 58 Audio input unit 58 Control unit 59 Video Signal output unit 201 Communication unit 202 Voice characterization unit 203 Translated voice unit 231 Communication unit 232 Video input unit 233 Character superimposition unit 241 Display unit 235 Sound recognition period indicator 236 Voice input unit 237 Voice element extraction unit 238 Communication unit 239 Translated voice Playback unit 240 Voice switching unit 401 Character translation unit 402 Minutes creation unit 431 Translated character overwrite unit 432 Pointer synchronization synthesis unit 434 Character extraction unit 435 Failure diagnosis unit 436 Microphone array 437 Speaker analysis unit 438 Speaker voice extraction unit 439 Pointer position Discriminating unit 440 camera

Claims (13)

A display device that displays images
A video output unit that outputs the video to be displayed and
The audio signal input section where the audio signal is input and
A control unit that extracts and outputs a translation target audio signal from the audio signal input to the audio signal input unit and reproduces the translated audio signal obtained by translating the translation target audio signal as translated audio.
Have,
The control unit is a display device that suspends switching of the video displayed by the video output unit while the translated audio signal is being reproduced. In the display device according to claim 1,
Based on the recognition signal, the control unit extracts the audio signal to be translated from the audio signal input to the audio signal input unit, and then extracts the audio signal to be translated.
The recognition signal is a display device that is input from the outside and indicates that the pre-translational explanation for the video has been completed. In the display device according to claim 1,
The control unit adds a display symbol indicating that the translated audio signal is being reproduced to the video output from the video output unit while the translated audio signal is being reproduced, and displays the display device. .. In the display device according to claim 1,
When the translated audio signal is reproduced, the control unit replaces the characters in the video displayed by the video output unit with the translated translated characters.
The video output unit is a display device that displays the translated characters replaced by the control unit on the video. In the display device according to claim 4,
The control unit determines the position of the pointer pointed to the video displayed by the video output unit, and when reproducing the translated audio signal, the insertion position of the pointer matches the word order of the translated audio signal. A display device that rearranges and displays the image. An output device connected to a display device that displays video and a translation device that converts audio into a translated audio signal, which is a translated audio signal.
The output device is
A video signal output unit that outputs a video signal to the display device,
An audio signal input unit into which the audio signal is input and
An audio signal extraction output unit that extracts an audio signal to be translated from the audio signal input to the audio signal input unit and outputs it to the translation device.
A translated voice conversion unit that receives the translated voice signal translated by the translation device, switches between the received translated voice signal and the voice signal input to the voice signal input unit, and outputs the translated voice signal to the voice reproduction unit .
Have,
The video signal output unit is an output device that outputs a video signal to be displayed next to the display device after the reproduction of the translated audio signal is completed. In the output device according to claim 6,
It has a recognition unit that recognizes that the explanation before translation has ended based on the recognition signal.
The audio signal extraction / output unit is an output device that outputs the audio signal extracted when the pre-translation explanation is completed when the recognition unit recognizes that the pre-translation explanation has been completed. In the output device according to claim 6,
An output device having a pointer function for pointing to an image displayed by the display device. An information display method using a display device for displaying video, a translation device for converting voice into a translated voice signal which is a translated voice signal, and an output device for outputting a video signal to the display device.
When the display device displays an image,
A step in which the display device extracts an audio signal explaining the displayed video, and
A step in which the translation device converts the voice signal extracted by the display device into a translated translated voice signal and outputs the voice signal to the display device.
A step in which the display device reproduces the translated audio signal received from the translation device.
Have,
The step of reproducing the translated audio signal is an information display method in which the display device suspends switching of images while the translated audio signal is being reproduced. In the information display method according to claim 9,
The step of extracting the audio signal is an information display method for extracting the audio signal when the explanation of the displayed video is completed. In the information display method according to claim 9,
The display device has a step of generating a display symbol indicating that the translated audio signal is being reproduced.
The step of reproducing the translated audio signal is an information display method in which the display symbol is added to the displayed video while the translated audio signal is being reproduced. In the information display method according to claim 11,
The step that the display device extracts characters in the displayed image, and
A step in which the translation device translates the extracted characters,
When the display device reproduces the translated audio signal, the characters in the video are replaced with the characters translated by the translation device and displayed.
Information display method. In the information display method according to claim 12,
A step in which the display device determines the position of a pointer pointed to by the displayed image,
When the translated audio signal is reproduced, the display device rearranges the determined pointer insertion position so as to match the word order of the translated audio signal and displays it on the video.
Information display method.

Images