JP3017221B2 - Information processing method and apparatus - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing method and an apparatus therefor, and more particularly to an information processing method and an apparatus for outputting a sound in association with a changing part of a screen to be displayed. .

2. Description of the Related Art Conventionally, as a screen explanation apparatus using sound, there has been one in which an operator displays a result of a database search or the like on a screen, and explains the entire screen using sound stored together with a database. This is called voice guidance,
After displaying one screen and adding a message, it was like a picture-story show.

However, in such voice guidance, it is difficult to explain a specific part on the screen. For example, on a screen in which similar figures are arranged in a matrix of vertical and horizontal (10 × 10), when specifying a figure in the screen, a voice saying “what number from the top, what number from the left” Information must be added. As described above, when one screen is described by voice, the voice description tends to be longer than the displayed content. In addition, some survey results, etc.,
Even when explaining on a screen where graphs, tables, etc. are mixed, where the conventional voice guidance is,
Next, it was difficult for the explained person to understand where to try to explain, because the correspondence with the screen could not be taken.

For these reasons, the screen explanation device with voice guidance is not always easy for the listener to understand and cannot be said to be an efficient explanation device.

[Problems to be Solved by the Invention] The present invention eliminates the above-mentioned conventional drawbacks, and enables a user to instruct image information to be changed in a series of output of audio information without instructing the image information at each change. Provided are an information processing method and an apparatus capable of outputting the information.

[Means for Solving the Problem] To solve the problem, an information processing method of the present invention is an information processing method using audio information and image information, and specifies a range of the image information. By creating a plurality of partial image information from the image information, input the relevant information to associate the audio information and each of the partial image information, the plurality of related information associated with a plurality of the partial image information The audio information is output, and in accordance with the output of the audio information, the output target is determined by the new relevant information from the first partial image information associated with the previous relevant information in accordance with the input relevant information. It is characterized by controlling to switch to the second partial image information to be associated. Here, the output of the partial image information and the audio information associated therewith is performed in synchronization. The voice information is text information, and the text information is output as voice by a voice rule synthesis device. Further, in addition to the related information, display information indicating a display method of the partial image information associated with the related information is input, and each of the partial image information is identifiably displayed corresponding to the display information.

Also, the information processing apparatus of the present invention is an information processing apparatus that outputs audio information and image information in association with each other, and creates a plurality of partial image information from the image information by specifying a range of the image information. Means for creating partial image information,
Related information input means for inputting related information for associating the sound information with each of the partial image information, and sound information output means for outputting the sound information associated with a plurality of the partial image information by a plurality of related information; In accordance with the output of the audio information by the audio information output means, the output target is associated with the new relevant information from the first partial image information associated with the previous relevant information in accordance with the input relevant information. Image information output means for switching to the second partial image information to be displayed. Here, there is further provided a synchronization control means for controlling output of the partial image information and audio information associated therewith in synchronization. Further, the voice information is text information, and the voice information output means includes a voice rule synthesizing device that outputs the text information as voice. Further, in addition to the related information, display information indicating a display method of partial image information associated with the related information is input from the related information input unit, and the image information output unit includes:
The respective partial image information is displayed in an identifiable manner in accordance with the display information.

Embodiment An embodiment of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the configuration of the audio explanation device of this embodiment.

In the figure, reference numeral 1 denotes text data input means for inputting audio data for explanation in the form of text, and inputs a character code from a keyboard, a word processor or the like. Reference numeral 2 denotes a text data storage memory for storing text data input from the text data input means 1. Reference numeral 3 denotes screen data input means for inputting screen data for explanation.
Input screen data created by graph / graphic drawing software. Reference numeral 4 denotes a screen display memory for storing one or more screen data inputted from the screen data input means 3.

Reference numeral 5 denotes an input data control unit, which stores data in the text data storage memory 2 and the screen display memory 4 and positions the screen information and the text data designated by the screen range designating means 6 and the text range designating means 7. The storage of information (offset addresses in the memories 2 and 4) in the position information storage memory 8 is controlled.

Reference numeral 10 denotes an output data control unit which performs transformation processing on a part of the data stored in the screen display memory 4 based on the position information stored in the position information storage memory 8 and sends it to the display unit 9. It sends the text data stored in the text data storage memory 2 to the audio output unit 11 and controls the output timing of audio data and screen display data.

Reference numeral 11 denotes an audio output unit that converts audio text data sent from 10 into audio and outputs the audio to the speaker 11a, and 12 denotes an input / output control unit that controls operations of the input data control unit 5 and the output data control unit 6. In the present example, this is performed by the switching switch 12a.

FIG. 2 is a diagram showing a hardware configuration of the screen explanation device of the present embodiment.

In the figure, 21 is a keyboard 21 for inputting control information for operation of the present apparatus, 22 is a data input section for inputting screen data or text data, and 23 is digital data for input information from the keyboard 21 and the data input section 22. As internal bus
It is an input interface to input to 30.

24 is a CPU for processing / calculation, 25 is a ROM for storing a control program, 26 is a RAM for storing and supporting data, and is a text data section 26a for storing text data and a screen data section 26b for storing screen data. And a position information section 26c for storing position information.

28 is a display CRT, 29 is an audio output unit that outputs audio to a speaker 29a, and 27 is an internal bus 30 connected to the CRT 28 and the audio output unit 29.
Is an output interface for outputting digital data. CPU24, ROM25, RAM26, input interface 23
And the output interface 27 are connected to the internal bus 30.

FIG. 3 is a flowchart showing the operation of the screen explanation device of this embodiment.

First, in step S2, the input / output switching switch 12a
Select input / output. In step S4, the process branches depending on whether it is an input or an output. In the case of an input, what input is determined in steps S6, S10, S14, and S18, and the respective storage processes in steps S8, S12, S16, S20 are performed. In this example, these input operations are performed by the keyboard 21
It can be switched by a further key input.

First, an operation at the time of input of the screen explanation device of the present embodiment will be described.

First, the procedure for inputting the voice text from step S6 to S8 will be described. After the operator specifies data input, the character code string of the voice output document created by the word processor or the like is input from the data input unit 22 and stored in the text data unit 26a of the RAM 26 in the form of a character code. . 4th
The figure shows an example of a voice output document.

Next, in steps S10 → S12, input of screen data is selected, and screen data is input from the data input unit 22. The screen is an image, a figure, a symbol, a character, or the like. In this example, screen data, such as that shown in FIG. 5, which has already been created as a bit image, is created using screen data input from a graphic or graph creation software (tool) or an image reader. It will be described using FIG. The input screen data is stored in the screen data section 26b of the RAM 26.

Next, the operator inputs a screen range in steps S14 to S16. In the input of the screen position information in step S16, first, the screen data is transferred from the screen data section 26b to the CRT 28, and the screen is displayed. The operator designates a screen range with a mouse while looking at the display, and inputs an identification symbol of the designated screen from the keyboard. In this way, the position information of the screen to be sequentially cut out is set, and the screen input in step S12 is divided into several screens. An identification symbol is also added and displayed in correspondence with the divided screen data.

This identification symbol includes information on the type of screen and the display method. FIG. 6 shows an example of screen division. In this embodiment, a case will be described in which one graph is described using five divided screens. The identification symbol in this example is
For example, two “a1” from the left of “a11” are identification numbers of the divided screen. The rightmost numeral "1" indicates the display method. “1” is a control code for performing a normal display, “2” is a control code for performing a blink display of the portion, and “3” is a control code for performing a color change display. The screen "a42" shown in FIG. 6 is used for blink display, and the screen "a53" is used for discoloration display.

Next, the operator specifies the range of the text in steps S18 → S20. In storing the text position information in step S20, first, text data is transferred from the text data section 26a to the CRT 28, and characters are displayed on the screen. The operator uses a keyboard, a mouse, or the like to separate the text data for voice output by associating the divided screens with the identification symbols. After separating the text data, the above-mentioned screen identification symbol is pushed into the separated portion. This is shown in FIG. Split screen data and text data are separated by the "/" symbol.

At the time when the text data range designation processing is completed, the address value of a memory in which the divided text data and the divided screen data are stored (this is referred to as position information) is stored in the position information section 26c. ing. Location information section 2
FIG. 8 shows the contents of 6c. The position information section 26c has an identification code 81 for identifying whether it is a screen or a text. "11" is a text, and "00" is a screen. Both of them include the position information 82 of the divided data for explanation.

As described above, the text data and the screen data are respectively divided, and the position information 82 of each data is stored in the position information section 26c together with the identification code 81 and the display selection code 83. An identification code 81 indicating the end of the description or the end of the display screen is added as “01”.

At step S22, the end of the input is checked, and if not, steps S6 to S22 are repeated.

Next, an operation at the time of output of the screen explanation device of the present embodiment will be described.

When the output is selected by the input / output switching switch 12a in step S2, the process proceeds from step S4 to step S24, and outputs the screen and the corresponding sound in the order stored in the position information section 26c. The identification code 81 is checked, and if it is text data, the text data is transferred from the text data unit 26a to the audio output unit 29 according to the position information, and the audio is output from the speaker 29a. On the other hand, if the data is screen data, the designated data of the display method is checked, and the deformed display is performed in the designated method. Here, in order to synchronize with the audio output, the audio output unit 29 checks whether or not the audio synthesis has ended, and after confirming the end, displays the next screen. This is performed by the audio output unit 29 setting an end flag when the synthesis of the text data of one section ends.

When the explanation of one paragraph is completed, the operator is asked in step S26 whether or not the explanation is necessary. If the explanation is repeated, the operation returns to step S24, and the explanation is made in the same manner from the beginning of the original paragraph. Check the end of the output with, and if not, return to step S24,
Steps S24 to S28 are repeated.

In this way, an appropriate explanation can be made by voice while displaying the screen (or the modified display) in the order of the divided screen display → the explanatory audio output → the divided screen display (modified) → the explanatory audio output → the divided screen display (modified). .

In the above-described embodiment, the explanation speech uses a rule synthesizer that converts character information input in text form into speech. However, a speech input unit is provided to directly input speech instead of text data. By using a voice data storage memory in place of the text data storage memory 2, voice information is also displayed by symbol labels as in the case of the divided screen data, and the output order of the screen and voice is controlled in the same manner as in the above embodiment. The same effect can be obtained.

In the embodiment, the screen data, the text data, and the like are prepared by dividing the data created in advance into units such as explanations by the screen range designating unit 6 and the text range designating unit 7. Although the symbols (labels) have been added, data may be input while attaching labels at the input stage.

As the voice information, the natural voice waveform may be stored in the memory as it is, but the voice waveform is ADPCM (Aduptive differ).
Storing the data in a memory using a waveform compression means such as a pulsative pulse code modulation has better memory efficiency. In this case, an encoder for performing compression and a decoder for performing reproduction are required.

The memory used in the present invention may be a magnetic disk, a magneto-optical disk, or the like, in addition to the semiconductor memory. When a portable medium such as a floppy disk is used, a display unit 9, an output data control unit 10, and an audio output unit 1, which are control units on the output side in FIG.
Output (screen + sound) can be performed by using only one component.

The method of synchronizing the output of the screen data displayed on the screen and the data for audio output is not limited to this example, and various other methods are conceivable.

For example, as shown in FIG. 9, the data is divided at the time of transition from the audio data to the screen data (indicated by 、 １), one block is set as a set of explanation data, and after the audio explanation is completed in one block data, the next When control is performed so as to output blocks, audio and screen display can be synchronized. Alternatively, the data of the next block may be output after the output of the screen data of one block or the audio output, whichever is later, is completed.

In the above example, the block boundary is determined based on the boundary from the voice to the screen data. However, the block position is also stored in the position information storage memory 8 as control information, and as shown by ↑ in FIG. Waiting for the end of the output, and outputting the next block at the end of the output, the synchronization can be established in any method. Further, when time information is inserted at the block boundary as shown in FIG. 10 and after a preset time interval has elapsed after the output of the screen or audio, the data of the next block is output. A more efficient explanation can be provided.

As described above, even when a large number of screens are displayed in one screen, or when a specific position on the screen is to be described, it is possible to explain by voice while instructing the place just described. It is easy to understand and efficient for humans (explainees). In addition, as the audio information used for the description, it is possible to have the conversational description using a demonstrative pronoun such as "as shown in this figure" or "this part is ...", so that the description is simplified and only the essential points are described. Explain efficiently.

[Effects of the Invention] According to the present invention, it is possible to automatically switch from partial image information to other partial image information by using identification information as a trigger along with output of audio information. The image information to be changed can be output without the user's instruction operation every time the image information changes.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of the audio explanation device of the present embodiment, FIG. 2 is a diagram showing the hardware configuration of the screen explanation device of the present embodiment, and FIG. 3 is the operation of the screen explanation device of the present embodiment. FIG. 4 is a diagram showing an example of a document for audio output, FIG. 5 is a diagram showing an example of data for screen output, FIG. 6 is a diagram showing an example of division of an input screen, and FIG. FIG. 8 is a diagram showing an example of determining the output order of text and screen, FIG. 8 is a diagram showing identification codes and control data stored in a position information memory, and FIGS. 9 and 10 are voice data and screen division data. FIG. 8 is a diagram for explaining another output synchronization method. In the figure, 1 ... text data input means, 2 ... text data storage memory, 3 ... screen data input means, 4 ...
Screen display memory, 5 ... input data control unit, 6 ... image range designating means, 7 ... text range designating means, 8 ... position information storage memory, 9 ... display unit, 10 ... output data control unit, Reference numeral 11 denotes an audio output unit, 11a denotes a speaker, 12 denotes an input / output control unit, and 12a denotes an input / output control switch.

Claims (8)

(57) [Claims] 1. An information processing method using audio information and image information, wherein a plurality of partial image information is created from the image information by designating a range of the image information. Inputting related information for associating with each of the partial image information, outputting the audio information associated with a plurality of the partial image information by a plurality of relevant information, and outputting the audio information in association with the output of the audio information According to the information, the output target is controlled to be switched from the first partial image information associated with the previous related information to the second partial image information associated with the new related information. Information processing method. 2. The information processing method according to claim 1, wherein output of the partial image information and audio information associated therewith is performed in synchronization. 3. The information processing method according to claim 1, wherein said voice information is text information, and said text information is output as voice by a voice rule synthesizer. 4. In addition to the related information, display information indicating a display method of partial image information associated with the related information is input, and each of the partial image information is identifiably displayed in accordance with the display information. The information processing method according to claim 1, wherein: 5. An information processing apparatus for outputting audio information and image information in association with each other, wherein a plurality of partial image information is generated from the image information by designating a range of the image information. Means, related information input means for inputting related information for associating the sound information with each of the partial image information, and sound information for outputting the sound information associated with the plurality of partial image information by a plurality of related information Output means, according to the output of the audio information by the audio information output means, according to the input related information, a target to be output, from the first partial image information associated with the previous relevant information, a new association An image information output unit configured to switch to and display the second partial image information associated with the information. 6. The information processing apparatus according to claim 5, further comprising a synchronization control unit that controls output of the partial image information and audio information associated therewith in synchronization. 7. The information processing apparatus according to claim 5, wherein said voice information is text information, and said voice information output means includes a voice rule synthesizing device which outputs said text information as voice. 8. In addition to the related information, display information indicating a display method of partial image information associated with the related information is input from the related information input means, and the image information output means inputs the display information to the display information. The information processing apparatus according to claim 5, wherein each of the partial image information is displayed so as to be identifiable.