JP3048510B2 - Sound information processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound information processing apparatus, and more particularly, to various document processing apparatuses such as a Japanese word processor, various image processing apparatuses such as an optical file, and various information transmission functions. Communication terminal, PO
The present invention relates to a sound information processing apparatus having a transmission unit capable of associating sound information with an information source conventionally used in an S terminal, a cash register, and the like, and printing and recording the sound information on an object on a sheet.

[0002]

2. Description of the Related Art At present, as a means for exchanging and distributing information, a medium such as a printing paper or a floppy disk is mainly used, and all kinds of document information and image information are recorded on this medium. However, recently, instead of this medium, a printing paper described in two-dimensional code has been considered. The two-dimensional code notation is an optically readable mark (two-dimensional pattern) added to a virtual matrix-like (two-dimensional) cell, and the code description is text information or image information. It is coded and described.

If such printing paper is used for data exchange and distribution, distribution costs can be reduced as compared with the case where a floppy disk is used, and data can not be decrypted directly, which is useful for maintaining confidentiality. .

As described above, document information and its coded information are printed and used as disclosed in Japanese Patent Laid-Open No. 5-12 / 1993.
An information processing device described in Japanese Patent Publication No. 288 is known. This information processing apparatus converts document information into encoded information such as JIS code, creates a barcode, and prints it on a recording sheet. Then, if necessary, the recording paper is read and inversely converted to return to the document information for use. When recording paper is copied or transmitted by a facsimile machine, the quality of encoded information is prevented from deteriorating, and the occurrence of information loss and errors is reduced.

[0005] Further, it is common for other media such as a magnetic recording device to print sound information as a two-dimensional code on a sheet. Has been done. However, they were all handled alone and not in connection with other information.

As a method for printing sound information on a sheet using a two-dimensional code as described above, a sound recording method and apparatus and a sound reproducing method and apparatus described in JP-A-3-114821 are known. The audio recording method and apparatus and the audio reproducing method and apparatus digitize audio, convert the digital data into an image of a net pattern, and print the image on a print medium. Conversely, by reading the print medium on which the image of the mesh pattern is printed with an image reader, digitizing the read medium, and performing analog conversion, audio can be reproduced.
However, there is no relationship between the audio data and other information.

[0007]

That is, conventionally, although sound information can be printed on paper, it is not associated with other print information around the sound information.
The sound information recorded on a sheet of paper is merely reproduced as a group of sound information, and is not utilized any further.

[0008] The present invention has been made in view of such circumstances, and relates sound information to print information.
It is an object of the present invention to provide a sound information processing apparatus which outputs sound information required by print information as needed, reads print information in accordance with output sound information, and enables display of the content indicated by the sound. .

Another object is as follows. The two-dimensional code printed on the sheet can be output as sound information related to each of the document information, image, graphic information, etc. printed on the two-dimensional code, and each related information indicated by the sound information can be output as sound. Enables visual representation in conjunction with information output progress.

In addition, the position of the two-dimensional code of the relative information of the sound to be printed on the sheet can be read before other information without recognizing the position of the two-dimensional code by sophisticated means. Facilitate information handling.

Furthermore, if the relative information of the sound printed on the sheet is pointed out, the sound information can be immediately heard. Then, means for preventing misreading of the sound information printed on the sheet is provided to ensure the quality of the sound information.

[0012] In addition, when the sound information is printed out in a two-dimensional code, the print format can be determined according to the purpose at the time of reproduction. Furthermore, when reading a sheet containing sound information, the sound output of the sound information is suppressed, and the sound can be read in accordance with the situation at the time of reproduction by stopping the reproduction of the sound or suppressing the volume. I do.

When reading a sheet containing sound information, the display output of the sound-related information in two-dimensional code is wiped off, and only information other than the sound-related information is displayed, so that a display method suitable for visualization is provided. provide.

Further, the original is read first, separately from the original reading and the sound output timing, and thereafter, the sound is output while confirming the display information on the display unit. Further, in the process of reading a document, sound output is performed while sequentially extracting related sound information from the head.

By pointing out specific coordinates of the displayed document information, image, graphic information, or the like, related sound information related to the coordinates is output. Also, if the cursor is moved when outputting sound information, even if the cursor moves out of the target area of the sound information, the sound information currently being reproduced is not stopped, and the sound output can be performed until the end of the sound information Then, while listening to the current voice output, the cursor can be moved to the next task.

[0016]

FIG. 1 is a block diagram showing the configuration of the present invention. As shown in FIG. 1, the present invention provides a visual input device for inputting visual information such as character information, image information and graphic information. Information input means 101, visual information storage means 102 for storing input visual information, area specifying means 103 for specifying a part of the area of visual information stored in visual information storage means 102, area specifying means 103 Sound information input means 104 for inputting sound information corresponding to the visual information of the area specified by, and sound information storage means 105 for converting the input sound information into barcode data and storing the same.
And relative relation information storage means 106 for converting relative relation information in which visual information of an area designated by the area designation means 103 and sound information stored in the sound information storage means 105 are converted into barcode data and stored. This is a sound information processing apparatus provided.

In the above arrangement, printing means capable of printing visual information such as characters, images and figures and bar codes on paper, visual information stored in the visual information storing means 102, and sound information storing means Print control means for determining a print arrangement of the sound information stored in the storage section 105 and the relative relation information stored in the relative relation information storage section 106 on a sheet surface, wherein the print control section determines whether the sound information or the relative relation information It has a function to print only the information on the outer periphery of the visual information, so that when the printed sheet is read, the sound information or the relative related information can be read before the visual information. preferable.

Also, printing means capable of printing visual information such as characters, images and figures and bar codes on paper, visual information stored in the visual information storing means 102, and sound information storing means 105 Print control means for deciding a print arrangement on the paper surface of the recorded sound information and the relative relation information stored in the relative relation information storage means 106, wherein the print control means stores only the sound information or the relative relation information. It is preferable to have a function of printing in the vicinity of the visual information specified by the area specifying means so that the relationship between the visual information and the sound information can be easily determined.

Further, a reading means capable of reading visual information such as characters, images and figures printed on paper and a bar code, a reproducing condition specifying means for specifying a reproducing condition of sound information, and a reading means for reading. And a sound reproducing means for reproducing the sound information identified by the identifying means in accordance with the conditions specified by the reproducing condition specifying means. preferable.

A reading means capable of reading visual information such as characters, images and figures printed on paper and a bar code, and a display means capable of displaying the visual information and bar code read by the reading means. Display means, a barcode display condition designating means for designating barcode display conditions, and a barcode read by the reading means.
It is preferable that the display device further includes a display control unit for displaying on the display unit in accordance with the condition specified by the barcode display condition specifying unit.

A sound information reproduction designating means for designating an area of visual information associated with sound information and a sound information reproduction designating means by referring to the relative related information stored in the relative related information storage means 106. It is preferable to further include a reading unit that reads the sound information corresponding to the area specified by the reading unit from the sound information storage unit 105, and a sound reproducing unit that reproduces the sound information read by the reading unit.

In the present invention, the bar code includes
Although a normal barcode can be used, from the viewpoint of the amount of information, a two-dimensional pattern in which optically readable marks are respectively added to virtual matrix-shaped cells is preferable.

In the present invention, the visual information input means 101, the area specification means 103, the reproduction condition specification means, the bar code display condition specification means and the sound information reproduction specification means are various types such as a keyboard and a pen input device. An input device can be used.

As the sound information input means 104, various sound input devices including a microphone and a D / A converter for converting an analog sound into a digital sound can be used.

The print control means, identification means, readout means and display control means include a CPU, a ROM, a RAM,
It is convenient to use a microcomputer having an / O port, and the visual information storage means 102, the sound information storage means 105, and the relative related information storage means 106 are usually RAMs.

As the printing means, various types of dot printers and laser printers of thermal transfer type and impact type can be used. Various image sensors can be used as the reading unit.

As the sound reproducing means, various sound generating devices including a D / A converter for converting a digital sound into an analog sound and a speaker can be used. Display means include a CRT display device and an LCD (liquid crystal display device)
And other display devices.

[0028]

According to the present invention, the visual information input by the visual information input means 101 is stored in the visual information storage means 102, and the sound information corresponding to the visual information of the designated area input by the sound information input means 104 is The sound information is converted into barcode data and stored in the sound information input means 104, and the visual information of the area designated by the area designating means 103 is associated with the sound information stored in the sound information storage means 105 and converted into barcode data. Then, it is stored in the relative relation information storage means 106. Therefore, the visual information and the sound information are stored in association with each other, so that the processing in which the visual information and the sound information are associated can be performed.

A printing unit, and a printing control unit,
When the print control means has a function of printing only the sound information or the relative related information on the outer peripheral portion of the visual information,
When the printed sheet is read, the sound information or the relative related information is read before the visual information, so that it is easy to associate the sound information with the visual information.

[0030] The apparatus further comprises a printing means and a printing control means.
When the print control means is provided with a function of printing only sound information or relative related information near the visual information designated by the area designating means, the relationship between the visual information and the sound information can be easily determined.

When the apparatus further comprises a reading means, a reproducing condition specifying means, an identifying means and a sound reproducing means, the volume of the reproduced sound is reduced or the reproduction is stopped according to the surrounding circumstances at that time. be able to.

In the case where a reading means, a display means, a bar code display condition designating means and a display control means are further provided, when the bar code and the visual information are displayed together, the visual information becomes difficult to read. Since the bar code can be prevented from being displayed, the screen can be easily viewed.

Sound information reproduction designating means, reading means,
When a sound reproducing means is further provided, only sound information corresponding to the specified visual information can be reproduced.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the embodiments shown in the drawings. Note that the present invention is not limited to this.

FIG. 2 is a block diagram showing the configuration of an embodiment in which the present invention is applied to a data processing apparatus capable of printing and reading sound information. Reference numeral 1 denotes a pen capable of specifying coordinates from the display screen, 2a denotes a tablet that detects points on the display screen specified by the pen 1, and 2b denotes an LC having a display screen.
A display device such as D (liquid crystal display device), 3 is a coordinate detection circuit that calculates coordinates specified by the pen 1, and 4 is a barcode data output control circuit that controls suppression of two-dimensional barcode data to be displayed on the screen. Reference numeral 5 denotes a keyboard, and reference numeral 6 denotes a cursor position calculation circuit.

Reference numeral 7 denotes a designated area search circuit, which includes document information,
The positional relationship between the sound area in the image information and the graphic information and the cursor is searched. Reference numeral 8 denotes a bar code conversion circuit that converts the digitized sound information into a two-dimensional bar code, 9 denotes a bar code restoration circuit that converts the read two-dimensional bar code sound information into digital sound information, and 10 denotes a bar code restoration circuit. A speaker input / output control circuit for controlling an input or controlling an exit to a speaker, 11 is a speaker (speaker / microphone) also serving as a microphone for taking in sound, and 12 is a converter for converting an analog sound signal into a digital signal or a sound signal. These are A / D and D / A converters that convert digital signals into analog signals.

Reference numeral 13a denotes a sheet object feeding device for feeding a sheet on which information is printed to a scanner, and 13b denotes a scanner for reading a sheet on which information is printed. The image reader 14 is constituted by the sheet object feeding device 13a and the scanner 13b.

Reference numeral 15 denotes a sheet object feeding control circuit for controlling the sheet object feeding device 15, and reference numeral 16 denotes a barcode data range discriminating circuit for discriminating a range of information of a two-dimensional barcode printed on the sheet object.

A check sum calculation circuit 17 calculates a checksum for detecting a reading error, an error judgment circuit 18 judges an error by comparing the two-dimensional bar code with its checksum, 19 a print layout selection circuit, 20
Is a specific information calling circuit for controlling input / output of data to / from the RAM, 21 is a RAM, 22 is a printing device such as a printer, 23 is printed data, and 24 is a control device for controlling all circuits. The RAM 21 stores relative related information and sound information.

A method for inputting voice and a method for displaying and printing the data in the data processing apparatus will be described. First,
With keyboard 5, pen 1, image reader 14, etc.
Create document information, graphic information, image information, etc., and then input voice.

Hereinafter, the contents of the voice input processing in the data processing apparatus will be described with reference to the flowchart of FIG. In the voice input, first, a voice input mode is entered (step S201), and a portion to be sounded in the voice input mode is designated by a coordinate designating means such as a tablet 2a or a cursor integrated with the display device 12b, as shown in FIG. , P2, etc. (step S202), press a sound input key in the keyboard 5 from the speaker / microphone 11 (step S203), start inputting a voice (step S204), and press the sound input key again. (Step S205), the input is terminated.

The analog signal of the input voice is A / D
The digital signal is converted by the converter 12 (step S
206), two-dimensional barcode data is created by the barcode conversion circuit 8 (step S206). At this time, the coordinate information of the designated area of each of the document information, the graphic information, and the image information is associated with the sound information of the two-dimensional barcode (step S207), and a relative association can be grasped as to how the association is performed. The information is stored as a two-dimensional barcode (step S208). When there are a plurality of parts to be further sounded in the same document, the above operation is repeated similarly (step S209).

FIG. 5 shows an example of this related information. In the figure, the area A is relative related information, the area B is document information, the area C is image information, the area D is graphic information, and the area E is two-dimensional bar-coded sound information. Are recorded, and each information is associated with a reference number (n) added in the input order after each symbol.

By providing the area A of the relative related information in the header portion of the sheet, an advanced means for recognizing the position of the sound information is not required. Further, a method of the association will be described with reference to FIG.

For example, it is assumed that an area b in the document information, an area c in the image information, an area d in the graphic information, and an area e in the sound information are associated with each other. At this time, in the relative related information area A, the coordinates (x
bn, ybn), (xbn ′, ybn ′) and coordinates (xcn, ycn) of points cn, cn ′ representing the area c,
(Xcn ', ycn') and a point d representing the area d
n, dn 'coordinates (xdn, ydn), (xdn', y
dn ') and the coordinates (xen, yen) and (xen', yen ') of the points en, en' representing the area e are recorded.

For example, it is assumed that the text in the area b is to be converted into a sound, and the area indicated by the points bn and bn 'is designated by a pen or a cursor. Then, the area e in the area a
Are called, the coordinates of the points en, en ', etc.

The relative related information and the two-dimensional bar code of the sound information are read by the image reader 14 when a reading error occurs. A checksum is added to the end of each block. For example, the sum of the information sequence is added as a reference numeral to the end of each block of the two-dimensional barcode of the relative related information and the sound information.

FIG. 7 shows a flowchart of the checksum adding process. First, the relative related information and the two-dimensional barcode of the sound information are read (step S301), and the sum of the information sequence is calculated (step S302). Next, the barcode of the total sum result of the information sequence printed following the information sequence is read (step S303). The two obtained values are compared (step S304). If they are equal, there is no error, and if they are different, reading is performed again.

An example of a method of calculating the sum of data in the relative related information will be described with reference to the data structure diagram of the relative related information in FIG. Reference numbers 1 to n are assigned to the relative related information. For the reference number 1, coordinates (xa1, ya1), (xa1 ', ya) representing the respective areas a to e are provided.
1 ′) to (xe1, ye1), (xe1 ′, ye1 ′)
, And their sum S1, S1 'is S1 = (xa1 + xb1 + ... + xe1) + (ya1 + yb1 + ... + ye1) S1' = (xa1 '+ xb1' + ... + xe1 ') + (ya1 ′ + yb1 ′ +… + y
e1 '). These values are added to the end of the data. The same calculation is performed for the reference number 2 and subsequent numbers in the second row.

Further, as shown in FIG. 9, a barcode of relative related information and sound information can be added near document information or near image and graphic information. For example, when an area to be converted into sound in the document information is designated and the voice input is completed, the two-dimensional barcode of the relative related information and the sound information is added like an underline in the document. The underline starts from the beginning of the sounded sentence,
Described by vertical n dots and horizontal m dots. The dot numbers n and m are determined by the amount of data and the like.

As shown in FIG. 10, when it is desired to add audio guidance to certain graphic information or image information,
When the input of the voice is completed by designating them, a two-dimensional barcode of the relative related information and the sound information is added like a frame of the information or a corner in the area.

The frame-shaped sound information is recorded clockwise or counterclockwise starting from the upper left of the area, and the data width (the number of dots) is determined from the amount of data, the length of the outer periphery of the area, and the like. When sound information is added to a corner in an area, an appropriate number of vertical and horizontal dots is determined from the amount of data, the size of the area, and the like. In these cases, the sound information can be immediately heard by pointing out the relative related information printed on the sheet.

When printing the information stored as described above, the information can be printed in various print formats. FIG. 11 shows a flowchart when this print format is selected. When the document information, the image information, and the graphic information are input (step S501), a voice input mode is entered (step S502), and it is selected whether or not to add a checksum for sound conversion (step S503).

When a voice is input and the two-dimensional bar code conversion of the sound information and the relative related information is completed (step S50).
4) If a checksum is added, it is added (step S505); if not, a print format is selected (step S506), and printing on a sheet is performed (step S507).

FIG. 12 shows an example of a print format that can be selected in step S506. Area A represents relative related information, area B represents document information, area C represents image information, area D represents graphic information, and area E represents sound information. For example, (5-2-1)
Since the relative related information is in the header of the sheet, advanced means for recognizing the position of the relative related information is not required, and the processing time can be reduced.

In (5-2-5), since the relative information and the sound information are recorded in the document information in an underlined manner, the sound can be formed simultaneously with the reading of the document. (5-2
In the case of -6), the part in which the two-dimensional barcode is written is arranged at the bottom of the sheet, so that the user can easily read a document or the like. As described above, there are various features depending on the layout method, and many layout patterns other than those shown in the figure are conceivable, and selection according to the needs of the user can be made.

After selecting a layout, based on the layout, moving or enlarging document information, graphic information, and image information,
It is also conceivable to enable reduction using a pen or cursor.

When reading a sheet on which sound information is recorded, it may be necessary to suppress the reproduction sound, stop reproduction, or reduce the volume, depending on the surrounding circumstances at that time. To achieve this, it is possible to select reproduction sound suppression before reading.

FIG. 13 shows a flowchart of the reproduction sound suppression processing. First, a reading mode is entered (step S
601), a screen as shown in FIG. 14 is displayed, and by moving the cursor or instructing with a pen or the like on the display screen,
A selection of volume suppression is performed (step S602).

When the reproduction of the sound is turned off, the sheet is read (step S608), and it is displayed on the screen (step S609), and the setting is completed. If the volume adjustment is necessary, the volume is adjusted by moving the cursor or using a pen or the like on the display screen of FIG. 14 (step S603). If the sound reproduction is ON, the volume adjustment screen of step S603 is skipped and the sheet is adjusted. Start reading (step S60)
4). Since it is displayed on the display screen (step S6
05), and information required to be sounded is designated by a cursor or a pen (step S606), and the sound is sounded (step S606).
607).

When reading a sheet in which sound information is mixed,
Document information, image information, and graphic information displayed on the screen may be difficult to read. To avoid this, it is possible to select whether to display sound information before reading.

FIG. 15 shows a flowchart of the sound information display presence / absence selection processing. First, a reading mode is entered (step S701), and a selection screen as shown in FIG. 16 is displayed, and selection of presence / absence of sound information display is performed by moving a cursor or instructing with a pen or the like on the display screen (step S701). S702). If the sound information display is necessary, the sheet is read as it is and all the data is displayed (step S703). If the sound information display is unnecessary, the area of the relative related information itself and the area of the sound information are read from the relative related information after reading the sheet. Recognize, wipe them off and display them (step S
704).

FIGS. 17 and 18 show display examples of presence / absence of sound information display. Area A is relative related information, area B is document information, area C is image information, area D is graphic information, area E
Represents sound information. Although the display is as shown in FIG. 17 in the normal display, when the display OFF is selected in step S702 in FIG. 15, the display becomes as shown in FIG. 18 and the screen becomes easier to see.

Next, a case where a sheet is read and converted to sound will be described. If the sound information is to be sounded, the sound information of the two-dimensional barcode can be entirely stored in the internal memory and the sound can be called out if necessary, as long as it is within the range of the internal memory capacity of the information processing device to be used. .

FIG. 19 shows a flowchart of a process for reading and converting the sheet into a sound. The sheet on which the information is mixed is read by the scanner (step S
801), an image is displayed on the screen (step S8)
02).

The relative related information and the sound information of the two-dimensional barcode are recorded in the internal memory (step S803). As for this information, a memory at a specific address is called up as necessary (step S804), and is sonicated (step S80).
5).

As described above, the sound information is not only stored once in the memory and called up when necessary, but also by controlling the image reader 14 each time based on the relative related information, and the sound of the necessary part is obtained. It is also possible to read only information and convert it to sound.

FIG. 20 is a flowchart of a process of reading only specific sound information and converting it into sound. The sheet on which the information is mixed is read by the scanner (step S
901), an image is displayed on the screen (step S9)
02). However, the sound information of the two-dimensional barcode is not recorded in the internal memory. Here, when the specific document information and image information graphic information which are sonicated are selected by means for designating a specific position such as the pen 1 (step S903), the position of the specific sound information is determined from the relative related information. Is called (step S904), the reading device is controlled (step S905), only specific information is read (step S906), and the sound is converted (step S9).
07).

FIG. 21 shows a specific configuration of the pen 1 for designating a specific position of specific document information, image information, and graphic information which are sonicated. The reference numbers of the components are the same as those shown in FIG. .

This configuration consists of a pen 1 and a transparent tablet 2
a is used. The tablet 2a is overlaid on the screen of the display device 2b. A displayed on the screen of the display device 2b is relative related information, B is document information, C is image information, D is graphic information, and E is sound information.

For example, in order to convert a part of the text in the area B into a sound, the user instructs the area B on the tablet 2a with the pen 1 or operates the cursor to move into the specific area.
A specific area is specified by pressing a specification key or the like. Thus, the point coordinates and the like indicating the area E are called from the information of the area A in the RAM 21. The controller 24 converts the sheet feeding amount of the sheet object feeding device 13a from the coordinates, and reads the specific sound information by operating the sheet object feeding device 13a.

Here, the timing at which sound information is converted into sound will be described. For example, when sound information is input and related information is displayed on the screen, when a cursor is over an area where sound information is present, or when the area is pointed with a pen 1 or the like, sound information corresponding to the area is displayed. Can be output as a sound.

FIG. 22 is a flowchart of the sound output process based on the area instruction. A specific area in the document information designated at the time of voice input is indicated by, for example, inverting the area, and a specific area in the image information, graphic information is indicated by, for example, surrounding the area with a frame. In the relative information, numerical data of the area, for example, numerical values in the X direction and the Y direction based on the coordinates of one vertex are recorded.
A serial number or the like for determining which sound information is paired is recorded.

When the area is designated by the pen 1 or the cursor (step S1000), and the cursor is over the area (step S1001), or when the area is designated by the pen 1 or the like (step S1002), the data is recorded there. The relative serial number and the relative information
The corresponding sound information is called (step S1003), and the sound information is converted (step S1004).

A method of specifying specific information in document information, image information, and graphic information by moving a cursor will be described in detail. As shown in FIG. 23, specific document information b to be sounded
And its area is (xb, yb) (xb ', yb')
And the current cursor position can be represented by (xC, yC).

FIG. 24 shows a flowchart of the cursor moving process at this time. The cursor is moved in an arbitrary direction using the keyboard (step S1005). The coordinates representing the area of the document information b and the current position of the cursor are compared in the X coordinate direction (step S1006), and further compared in the Y coordinate direction (step S1007). (Step S1008).

When the sound information is converted into a sound by the above-described method, another region may be specified even though all the sound information in the region specified first is not converted into a sound.

That is, as shown in FIG. 25, when there are specific vocalized documents b1 and b2 in the document information area B, the document b1 is specified first, and Document b2 may be specified. At this time, with regard to the conversion of the document b2, a certain blank time is provided after the reproduction of the sound information of the document b1 is completed.
The second sound information is reproduced.

FIG. 26 is a timing chart of the sound reproduction waiting, and FIG. 27 is a flowchart of the sound reproduction waiting process. First, a document b1 in the document information area B is specified (step S1101), and sound information of the document b1 is reproduced (step S1102). If the document b2 is specified during the reproduction (step S1103), it is determined whether the document b1 is being reproduced (step S1104). If the document b1 is being reproduced, the process is continued (step S1107). A predetermined blank time is provided (step S1105), and the sound information of the document b2 is reproduced (step S1106).

In this way, by recording the sound information on the sheet in association with the print information other than the sound information, the contents of the document information, image, and graphic information on the sheet can be enhanced.

For example, as sound information accompanying document information, images, and graphic information, sound images that are difficult to express in sentences,
It is possible to print emotional remarks and the like, and to add supplementary information related to them as sound information.

By recording this sound information in association with other information, it is possible to create various information utilization scenes which have not been realized until now. In other words, when converting the sound information of the document information, image, and graphic information into a sound, the specific area of each piece of the document information, image, and graphic information is designated by a pointing device such as a mouse or a pen, or by moving a cursor. ,
It becomes possible to output related sound information related to the specific area in a sonic form.

Further, when sound information is output as sound, it is possible to display a specific area of the related document information, image and graphic information in a display form in which the specific area can be determined. In the former (output sound information from other than sound information),
It enables the transmission of sound information that is difficult to express in words and emotions, and makes it possible to create a dictionary from printed expressions other than sound information to sound information, and to reach people's eyes with letters and documents. It is possible to record and manage confidential information that you do not want as sound information, and to make it possible to read document information and the like simply by moving the cursor without looking at the display.

In the latter case (the area other than the sound information is guided on the display surface from the sound information output), information other than the sound information can be viewed in association with the flow of the sound information.
Document information can be read without looking at the display without moving the cursor, and information other than sound information related to sound information can be visualized, enabling map guidance to the target object, sound information, etc. Or a dictionary to information other than sound information.

Further, since sound information can be complementarily managed in a sheet state, management and retrieval of information becomes easy. In other words, filing of information can be easily performed, and information on a sheet can be copied by a copying machine or the like, so that information can be easily distributed to a large number of people. Since it is possible to issue a printed matter including the information, means for distributing sound information on an inexpensive sheet can be provided, and it can be used as various information transmissions.

These can be realized by printing the sound information and its related information on the sheet by various methods. For example, the sound information can be printed on the header of the sheet, the document information, the image, the graphic information, or the like. Select the format according to the purpose of the information written on the sheet at the time of printing while printing in the area such as etc. and adding supplementary information of these two-dimensional barcodes to eliminate reading errors By doing so, it is possible to consider various printed materials.

At the time of reproduction, the sound output of the sound information on the sheet is turned off or suppressed, and the displayed sound information and sound-related information are erased. it can.

Further, in the case of continuously requesting the designation of a sound from the display by using a cursor or the like, it is designed so that the next sound cannot be heard until one sound output is completed. Make it possible to hear without overlapping.

Further, it is also possible to read sheets in a lump and try sound output later, or to try sound output each time while reading the sheet, because it is possible to use the method according to the needs. There is a strong possibility that usage will be born in various ways.

[0090]

According to the present invention, since the visual information and the sound information of the area specified by the area specifying means can be stored in association with each other, it is possible to perform processing in which the visual information and the sound information are associated with each other. Become.

When only the sound information or the relative related information is printed on the outer peripheral portion of the visual information, it is easy to associate the sound information with the visual information. Further, when only the sound information or the relative related information is printed near the visual information specified by the area specifying means, the relationship between the visual information and the sound information can be easily determined.

If the apparatus further comprises a reading means, a reproducing condition specifying means, an identifying means, and a sound reproducing means, the sound volume of the reproduced sound can be reduced, You can stop sound playback.

[0093] Further, in a case where reading means, display means, bar code display condition designating means and display control means are further provided, when the bar code and the visual information are displayed together, the visual information is difficult to read. In some cases, the bar code can be prevented from being displayed, so that the visual information can be easily viewed.

Further, when the apparatus further comprises sound information reproduction designating means, reading means, and sound reproducing means, only sound information corresponding to the specified visual information can be reproduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is a block diagram showing a configuration of an embodiment in which the present invention is applied to a data processing device capable of printing and reading sound information.

FIG. 3 is a flowchart showing the content of a voice input process in the data processing device of the present invention.

FIG. 4 is an explanatory diagram showing a method for specifying an area of visual information according to the present invention;

FIG. 5 is an explanatory diagram showing an example of associating coordinate information of a designated area of each of document information, graphic information, and image information with sound information of a two-dimensional barcode according to the present invention.

FIG. 6 is an explanatory diagram showing a method of associating coordinate information of a designated area of each of document information, graphic information, and image information with sound information of a two-dimensional barcode according to the present invention.

FIG. 7 is a flowchart showing a checksum adding process according to the present invention.

FIG. 8 is an explanatory diagram showing a data structure of relative relation information according to the present invention.

FIG. 9 is an explanatory diagram showing an example of adding barcodes of relative related information and sound information to the lower part of document information according to the present invention.

FIG. 10 is an explanatory diagram showing an example in which barcodes of relative related information and sound information according to the present invention are added around image information and a part of graphic information.

FIG. 11 is a flowchart showing processing when a print format is selected according to the present invention.

FIG. 12 is an explanatory diagram illustrating an example of a selectable print format according to the present invention.

FIG. 13 is a flowchart showing a reproduction sound suppressing process according to the present invention.

FIG. 14 is an explanatory diagram showing a display example of a screen when adjusting the volume according to the present invention.

FIG. 15 is a flowchart showing a selection process of sound information display presence / absence according to the present invention.

FIG. 16 is an explanatory diagram showing an example of a selection screen for the presence or absence of sound information display according to the present invention.

FIG. 17 is an explanatory diagram showing a display example with sound information display according to the present invention.

FIG. 18 is an explanatory diagram showing a display example without sound information display according to the present invention.

FIG. 19 is a flowchart showing processing when reading a sheet and converting it into sound according to the present invention.

FIG. 20 is a flowchart showing processing for reading only specific sound information and converting it into sound according to the present invention.

FIG. 21 is an explanatory diagram showing a specific configuration of a pen for specifying a specific position of document information, image information, and graphic information according to the present invention.

FIG. 22 is a flowchart showing a sound output process based on an area instruction according to the present invention.

FIG. 23 is an explanatory diagram showing a method of designating specific information in document information, image information, and graphic information by moving a cursor according to the present invention.

FIG. 24 is a flowchart showing a cursor moving process at the time of specifying specific information according to the present invention.

FIG. 25 is an explanatory diagram showing an example of a case where instructions for reproducing audio information are continuous according to the present invention.

FIG. 26 is a time chart showing the timing of waiting for reproduction of sound according to the present invention.

FIG. 27 is a flowchart showing a sound reproduction waiting process according to the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 pen 2 a tablet 2 b display device 3 coordinate detection circuit 4 barcode data output control circuit 5 keyboard 6 cursor position calculation circuit 7 designated area search circuit 8 barcode conversion circuit 9 barcode restoration circuit 10 speaker input / output control circuit 11 speaker / microphone 12 A / D, D / A converter 13a Sheet object feeder 13b Scanner 14 Image reader 15 Sheet object feed control circuit 16 Barcode data range discriminating circuit 17 Check sum calculating circuit 18 Error discriminating circuit 19 Print layout selecting circuit 20 Calling specific information Circuit 21 RAM 22 Printing device 23 Printed data 24 Control device

Claims (6)

(57) [Claims] 1. Visual information input means for inputting visual information such as character information, image information, and graphic information, visual information storage means for storing input visual information, and visual information stored in the visual information storage means Area specifying means for specifying a part of the area, sound information input means for inputting sound information corresponding to visual information of the area specified by the area specifying means, and converting the input sound information into barcode data. Information storage means for storing and storing relative information in which visual information of an area designated by the area designating means and sound information stored in the sound information storage means are converted into barcode data and stored. A sound information processing device comprising information storage means. 2. A printing means capable of printing visual information such as characters, images and figures and a barcode on paper, visual information stored in a visual information storage means, and visual information stored in a sound information storage means. Print control means for deciding a print arrangement of the sound information and the relative relation information stored in the relative relation information storage means on a sheet surface, wherein the print control means converts only the sound information or the relative relation information into visual information. Has the function of printing on the outer periphery,
2. The sound information processing apparatus according to claim 1, wherein, when the printed sheet is read, the sound information or the relative related information is read before the visual information. 3. A printing means capable of printing visual information such as characters, images and figures and a barcode on paper, visual information stored in a visual information storing means, and visual information stored in a sound information storing means. Print control means for determining a print arrangement of the sound information and the relative relation information stored in the relative relation information storage means on a sheet surface, wherein the print control means specifies only the sound information or the relative relation information as an area designating means 2. The sound information processing apparatus according to claim 1, wherein the sound information processing apparatus has a function of printing near the visual information specified by the function, whereby the relationship between the visual information and the sound information can be easily determined. 4. A reading means capable of reading visual information and a bar code such as characters, images and figures printed on paper, a reproducing condition specifying means for specifying a reproducing condition of sound information, and reading by the reading means. 2. The apparatus according to claim 1, further comprising: identification means for identifying the sound information from the barcodes provided; and sound reproduction means for reproducing the sound information identified by the identification means in accordance with the conditions specified by the reproduction condition specification means. Sound information processing device. 5. A reading means capable of reading visual information and a barcode such as characters, images and graphics printed on a sheet, and displaying the visual information and the barcode read by the reading means. Display means, bar code display condition designating means for designating bar code display conditions, and display control means for displaying the bar code read by the reading means on the display means in accordance with the conditions designated by the bar code display condition designating means. The sound information processing apparatus according to claim 1, further comprising: 6. A sound information reproduction designating means for designating an area of visual information associated with sound information, and a sound information reproduction designating means by referring to relative relative information stored in a relative relevant information storage means. 2. The sound information processing apparatus according to claim 1, further comprising: a reading unit that reads sound information corresponding to the designated area from the sound information storage unit; and a sound reproducing unit that reproduces the sound information read by the reading unit.