JP4388142B2 - Information processing system and recording medium storing program for causing computer to perform processing in this system - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
  The present invention relates to an information processing system for processing a document including uncoded non-coded data such as image data and audio data, and more specifically, information enabling search processing based on such uncoded data. Regarding processing system.
[0002]
[Prior art]
In an information processing system that performs processing such as document creation, editing, and search, encoded coded data such as characters and symbol strings and uncoded non-coded data such as images and sounds are processed. . The search process in such a system searches for a specific character or symbol (sequence) from a coded character or symbol string constituting the document in a document, or a specific code from an accumulated document group. A document including the converted character, symbol (column) (keyword) is searched.
[0003]
[Problems to be solved by the invention]
However, the conventional information processing system cannot perform a search process based on uncoded data such as image data and audio data. For example, it is not possible to search for a specific image or sound from a certain document, or to search for a document containing a specific image or sound from an accumulated document group.
[0004]
  Therefore, the present inventionSectionThe title is to provide an information processing system capable of performing search processing based on uncoded data such as image data and audio data..
[0005]
[Means for Solving the Problems]
  UpLessonIn order to solve the problem, the present inventionIn an information processing system for processing a document containing unencoded audio data, when a search command is issued, an extraction unit that extracts audio data included in the specified document, and the extraction unit Label information generating means for generating label information including main frequency components representing the characteristics of the audio data from the audio data, main frequency component information as label information as a search key provided together with the search command, and the label Determining means for determining whether or not the main frequency component information which is the label information generated by the information generating means is substantially the same, and the determining means is the main frequency component information of the both label information Are determined to be substantially the same, a search result based on the determination result is obtained.
[0012]
  In such an information processing system, a search command was issued.CaseFrom the specified documentvoiceData was extracted and extractedvoiceFrom the dataContains the main frequency component representing the characteristics of the audio dataLabel information is generated. When it is determined that the generated label information is substantially the same as the label information provided as the search key, a search result based on the determination result is obtained.
[0013]
  The search result based on this determination result is a search result reasonably obtained from the determination result. For example, the source of label information that is substantially the same as the label information provided as a search keyAudio dataAnd thatAudio dataThe designated document including “” is obtained as a search result.
[0016]
  Furthermore, it is described in a predetermined area of the document.voiceFrom the standpoint that documents classified based on data can be searched, the present invention provides that the extraction means is described in a predetermined area of the document.voiceConfigured to extract data.
[0017]
  In such an information processing system, always from a predetermined area of the documentvoiceData is extracted. Its extractedvoiceLabel information is generated from the data. When the label information is substantially the same as the label information provided as a search key, the document can be obtained as a search result. Therefore, it is the same as the predetermined area of documents belonging to the same classificationvoiceIf data is described, documents belonging to the same classification can be obtained as search results.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
An information processing system according to an embodiment of the present invention is constructed by a computer system, and its functional configuration is, for example, as shown in FIG. In this information processing system, a document including media data composed of at least one of image data and audio data is processed (created, edited, searched, etc.). In FIG. 1, an information processing system 1 includes a document management unit 10, a management unit 11, a display unit 12, an audio output unit 13, an input unit 14, an input / output control unit 15, a feature extraction unit 16, a compression unit 17, and a display control unit. 18 and a search unit 19.
[0020]
The document management unit 10 manages documents (including media data) handled in the system. The management unit 11 manages the relationship between the document managed by the document management unit 10 and label information as will be described later. The display unit 12 displays a document managed by the document management unit 10. When the media data included in the document managed by the document management unit 10 is composed of sound data, the sound output unit 13 outputs sound based on the sound data. The input unit 14 inputs a document including media data and data serving as a search key to the system.
[0021]
The input / output control unit 15 performs interface processing among the display unit 12, the audio output unit 13 and the input unit 14. A document managed by the document management unit 10 is provided to the display unit 12 via the input / output control unit 15, and the document is displayed on the display unit 12. Audio data (media data) included in a document managed by the document management unit 10 is provided to the audio output unit 13 via the input / output control unit 15, and audio based on the audio data is output from the audio output unit 13. Is done. A document input by the input unit 14 is provided to the document management unit 10 via the input / output control unit 15, and the provided document is managed by the document management unit 10.
[0022]
The feature extraction unit 16 extracts media data from the document managed by the document management unit 10, and further extracts feature data representing the feature of the media data from the media data. The feature data is provided to the management unit 11 as label information. For example, in the case of media data having a certain pattern, the frequency component amount that the pattern has is calculated as the feature data. For example, when the media data is composed of image data, the number of blocks defined by the number of image chunks (characters, symbols, etc.) included in the image is extracted as feature data. Furthermore, for example, when the media data is composed of audio data, the main frequency component value (for example, 8 kHz) of the audio data is extracted as the feature data.
[0023]
When the media data extracted from the document managed by the document management unit 10 is composed of image data, the compression unit 17 reduces a part or all of the extracted image data (to a predetermined size). Normalized reduced data is generated, or frame data representing simplified image data is calculated. This frame data represents, for example, a triangle, a quadrangle, a circle, etc., which are simplified image data. The reduced data or the frame data generated or calculated by the compression unit 17 is provided to the management unit 11 as label information.
[0024]
The management unit 11 that receives the label information from the feature extraction unit 16 or the compression unit 17 manages the correspondence between the document including the media data that is the label information and the label information. That is, when label information is designated, a document corresponding to the designated label information is specified based on the correspondence relationship managed by the management unit 11.
[0025]
The display control unit 18 creates a list of label information whose relationship with each document is managed by the management unit 11 and causes the display unit 12 to display the list of label information. The list of label information displayed on the display unit 12 is used for selecting a search key in the search process.
[0026]
The search unit 19 searches the document group managed by the document management unit 10 for a document corresponding to the search key input from the input unit 14. Specifically, the label information substantially the same as the search key is searched from the label information whose correspondence with the document is managed by the management unit 11, and the document having the correspondence with the searched label information is specified. To do. Then, the identified document is obtained as a search result.
[0027]
As described above, based on the control of the display control unit 18, the display unit 12 may display a list of label information whose correspondence with the document is managed in the management unit 11. In this case, when the input unit 12 performs an operation of selecting one label information from the list, the label information is provided to the search unit 19. The search unit 19 performs the same process as described above using the provided label information as a search key.
[0028]
As described above, the management unit 11 that manages the correspondence between the document managed by the document management unit 10 and the label information may not be provided. In this case, the search unit 19 specifies a document managed by the document management unit 10, and uses label information generated by the feature extraction unit 16 or the compression unit 17 from media data extracted from the specified document. obtain. Then, the search unit 19 further determines whether or not the label information and the input search key are substantially the same.
[0029]
If they are substantially the same, the search unit 19 can output the media data that is the basis of this label information as a search result. This process corresponds to a process of searching for media data matching the search key from the specified document. In addition, a document including media data that is the source of the label information can be output as a search result. This processing corresponds to processing for searching a document including media data that is the source of label information matching the search key from documents managed by the document management unit 10.
[0030]
According to the information processing system having the above-described functions, label information representing the characteristics of uncoded media data is associated with a document including the media data, so that search processing based on the label information is possible. It becomes. As a result, a search process based on the media data (uncoded data) that is the source of the label information becomes substantially possible.
[0031]
Next, a more specific system will be described.
FIG. 2 shows a hardware configuration of the information processing system. This system is configured, for example, in a portable computer terminal device capable of handwriting input. In FIG. 2, the system includes a CPU (Central Processing Unit) 20, a ROM (Read Only Memory) 21, a memory unit 22, a handwriting input device 23, a keyboard device 24, a display device 25, and a file storage unit connected to a bus. 26.
[0032]
Various programs are stored in the ROM 21. The program stored in the ROM 21 includes a program for performing processing in the system according to the present invention. That is, the ROM 21 corresponds to the recording medium according to the present invention. The CPU 21 executes various processes such as a document creation process, a document edit process, a document management process, and a document search process in accordance with a program stored in the ROM 20.
[0033]
The memory unit 22 stores data obtained by processing in the CPU 20 and is used as a working memory. The handwriting input device 23 is used for inputting handwritten line images, handwritten characters, and the like using a touch pen. The keyboard device 24 is used for inputting characters and symbols by key operation. The display device 25 is composed of, for example, a liquid crystal display panel, and displays a document or the like created and edited in the system. The file storage unit 26 is composed of, for example, an EEPROM, a RAM with a backup, or the like, and stores documents, tables and the like created in the system.
[0034]
In the system as described above, a document is created and edited using handwritten line images and handwritten characters input from the handwriting input device 23, and characters and symbols input from the keyboard device 24. Further, a mark is described in a specific area of the created and edited document using the handwriting input device 23. The specific area of the document is determined in advance at the beginning of the document, for example, as shown in FIG. This mark can be used, for example, for rough classification of documents (e.g., importance rank), and as shown in FIG. 5, one mark (FIG. 5 (a)) and two marks It is represented by a mark (Fig. 5 (b)), three circles (Fig. 5 (c)), and the like.
[0035]
When the document creation, editing, and mark entry are completed as described above, the CPU 20 performs document storage processing (including image data, hereinafter referred to as an image data document) according to the procedure shown in FIG.
In FIG. 4, the CPU 20 extracts image data in a specific area of the image data document (S1), and performs normalization processing of the image data so that the display image has a predetermined size (S2). Then, the CPU 20 further extracts features from the normalized image data (S3) (corresponding to the feature extraction means 16 shown in FIG. 1). For example, the number of blocks included in the image represented by the normalized image data, that is, the number of blocks is extracted as a feature.
[0036]
When the marks shown in FIGS. 5A, 5B, and 5C are described, the numbers of extracted blocks are “1”, “2”, and “3”, respectively. Then, the CPU 20 generates the number of blocks as label information corresponding to the image data document (S4). Thereafter, the CPU 20 stores the created and edited image data document in the file storage unit 26 (S5). A table representing the relationship between the image data document (document name) and the storage position is constructed in the file storage unit 26, and the relationship between the two is managed in the system (corresponding to the document management means 10 shown in FIG. 1).
[0037]
For example, as shown in FIG. 7, the relationship between label information (in this example, the number of blocks (a, b,...)) And an image data document (document name, AAA, BBB, etc.) is described. A label management table is built in the file storage unit 26. When the label information is created as described above, the created label information is described in the label management table so as to correspond to the created and edited image data document (S6). The relationship between the image data document and the label information is managed using this label management table (corresponding to the management means 11 shown in FIG. 1).
[0038]
As described above, every time an image data document is created and edited, label information is created based on the image data (mark) described in the specific area, and the image data document is stored in the file storage unit 26. In addition, the relationship between the image data document and the label information is described in the label management table.
[0039]
For example, in order to search for three marked documents (see FIG. 5C) that the user has determined as the most important document, the number of blocks “3” is used as a search key. 24 or when input from the handwriting input device 23, the CPU 20 executes a search process according to the procedure shown in FIG. 8, for example (corresponding to the search means 19 shown in FIG. 1).
[0040]
After initializing internal coefficients and the like (S11), the CPU 20 acquires the input search key (number of blocks “3”) (S12). Further, the CPU 20 increments the counter value i initialized to “0” (i = 1) (S13), and stores the label information corresponding to the i-th (first) document in the label management table (FIG. 7). Read from (see) (S14).
[0041]
Then, the CPU 20 compares the acquired search key with the read label information (S15), and determines whether or not they match (S16). If they match, the CPU 20 sets the ID (document number) of the i-th (first) image data document in a predetermined register (S17). If the retrieved key information does not match the read label information, or after setting the ID of the image data document in the register, the CPU 20 counts the number i of all documents in which the counter value i is stored in the file storage unit 26.₀(I ≧ i)₀(S18), if not reached, the label information corresponding to the next document is compared with the search key.
[0042]
Thereafter, the counter value i is the total number of documents i.₀Until the counter value i is equal to the total document number i.₀(When the comparison between the label information corresponding to all documents and the search key is completed), the CPU 20 displays the IDs of the image data documents set in the register at that time on the display device 25 as a list. (S19). In other words, for example, a list of documents having the number of blocks “3” specified by the search key (documents in which three circles are described in a specific area) is displayed on the display device 25.
[0043]
When the user designates one document from the document group in the list by using the handwriting input device 23, the designated document is read from the file storage unit 26 and displayed on the display device 25.
In the system as described above, label information is created from the characteristics of image data (handwritten circles) included in the document, and the correspondence between the label information and the document is managed. Then, when the label information is provided as a search key, the document is searched based on the managed correspondence. As a result, a document search process based on image data (handwritten circles) can be performed.
[0044]
When extracting the block number from the acquired image data as a feature, various images can be used in the same system as long as a block can be extracted. For example, as shown in FIGS. 6A, 6B, and 6C, one Δ mark, two Δ marks, and three Δ marks can be used. In this case, the number of blocks is “1”, “2”, and “3”, as in the case of the circles.
[0045]
It is also possible to create label information by extracting the following features of image data. That is, two-dimensional Fourier analysis is performed on the image data of the specific area, and the frequency component amount obtained at that time, the color component of the image data of the specific area, etc. are extracted as the characteristics of the image data. Can be created.
[0046]
In the above example, the label information is created by extracting features from the acquired image data. However, a reduced image obtained by reducing the image data can be used as the label information.
As an image described in the specific area, an example in which ◯, ×, Δ, □ as shown in FIG. 9 is used will be described.
[0047]
At the end of the creation and editing of the document, any one of the four types of marks is entered in the specific area (first area) of the document by the handwritten character input device 23. In this case, the CPU 20 executes processing in accordance with substantially the same procedure as that shown in FIG. That is, after image data (one of ○, ×, Δ, □) in a specific area is taken out (S1), normalization processing of the image data is executed (S2), and image data of a predetermined size is obtained. (Generally reduced image data) is generated (thumbnail processing) (corresponding to the compression unit 17 shown in FIG. 1). Then, label image information is created based on the normalized image data (S4). The created and edited document is stored in the file storage unit 26 (S5).
[0048]
The relationship between the label image information and the document is managed as follows.
The created label image information (reduced image data) itself is stored in the file storage unit 26. Then, the storage position (pointer value) of the label image information is described in a label storage position table representing the relationship between the label image information and the storage position, for example, as shown in FIG. This label storage position table is constructed in the file storage unit 26.
[0049]
As described above, after the storage position of the label image information is described in the label storage table, the relationship between the document (AAA, BBB, etc.) created and edited and the label image information is as shown in FIG. Is described in the label management table (S6 shown in FIG. 4).
[0050]
The search processing of the document stored in the file storage unit 26 while being associated with the label image information as described above is performed, for example, according to the procedure shown in FIG.
For example, when a user performs handwriting input using the handwriting input device 23 as a search key in order to search for a document with any mark of ○, ×, Δ, □, the CPU 20 After the coefficient values are initialized (S21), the image data of the input mark is acquired (S22). Then, the CPU 20 normalizes the acquired image data (S23), and generates search data corresponding to the label image information to be searched from the normalized image data obtained as a result (S24).
[0051]
When the search data is generated in this way, the CPU 20 reads the label image information from the file storage unit 26 with reference to the label storage position table (see FIG. 10) (S25). Then, the CPU 20 compares the search data generated as described above with the read label image information (S26), and determines whether or not they match (S27). This comparison and coincidence determination is performed by a general image recognition process, and label image information having a predetermined similarity (substantially the same) to the search data is the label image information that matches the search data. Determined.
[0052]
If they do not match, the label image information managed in the label storage position table is repeatedly compared with the search data and matched in the same manner as described above (S36, S25, S26, S27). When there is label image information that matches the search data, the CPU 20 sets the ID of the label image information in a predetermined register as the ID of the label image information (search label) to be searched (S28).
[0053]
Thereafter, the CPU 20 increments the initialized counter value i (i = 0) (i = 1) (S29), and the ID of the label image information corresponding to the i-th (first) document is managed by the label management. Read from the table (see FIG. 11) (S30). Then, the CPU 20 compares the ID of the search label set in the predetermined register with the ID of the read label image information (S31), and determines whether or not they match (S32). If they match, the CPU 20 sets the ID (document number, document name, etc.) of the i-th (first) image data document in a predetermined register (S33). If the ID of the search label does not match the ID of the read label image information, or after setting the ID of the image data document in the register, the CPU 20 stores all the counter values i stored in the file storage unit 26. Number of documents i₀(I ≧ i)₀(S34), if not reached, the ID of the label image information corresponding to the next document is compared with the ID of the search label.
[0054]
Thereafter, the counter value i is the total number of documents i.₀Until the counter value i is equal to the total document number i.₀(When the comparison between the label image information corresponding to all documents and the search label is completed), the CPU 20 displays the IDs of the image data documents set in the register at that time on the display device 25 as a list. (S35). That is, on the display device 25, for example, a list of documents in which a circle is described in a specific area is displayed.
[0055]
When the user designates one document from the document group in the list by using the handwriting input device 23, the designated document is read from the file storage unit 26 and displayed on the display device 25.
In the process according to the procedure shown in FIG. 12, if all the label image information does not match the search data (S36, YES), the CPU 20 displays a message that the corresponding image data document does not exist. After being displayed on the device 25 (S37), the search process is terminated.
[0056]
In the above system, label image information obtained by reducing image data (handwritten ○ mark, × mark, Δ mark, □ mark) included in a document is created, and the correspondence between the label image information and the document is managed. The Then, when handwritten image data is provided as a search key, a document corresponding to the image data is searched based on the managed correspondence.
[0057]
In the above example, the user directly inputs image data (such as a mark) described in the document to be searched. However, as the types of image data increase, it becomes difficult for the user to remember the image data. In such a case, the CPU 20 performs, for example, a process according to the procedure shown in FIG. 13 before performing the search process as described above so that the image data can be easily specified.
[0058]
In FIG. 13, the CPU 20 refers to the label storage position table (see FIG. 10), reads each label image information from the file storage unit 26, and displays a list of each label image information (reduced image) on the display device 25. It is displayed (S41). When the user performs an operation of designating one label image information from the displayed label image information group using the handwriting input device 23, the CPU 20 acquires the designated label image information (S42). Thereafter, the CPU 20 uses the acquired label image information as search data, and performs a search process P according to a procedure similar to the procedure shown in FIG.₁₀₀Execute.
[0059]
In each of the above-described examples, the head portion of the image data document is used as the specific area, and the label information is created from the image data described in that portion. However, the present invention is not limited to this. For example, as shown in FIG. 3B, the head line of the image data document or the head part of the actual image data included in the image data document as shown in FIG. May be.
[0060]
Furthermore, in each of the above-described examples, the image that is the basis of the label information is a mark that the user has purposely described for use in, for example, document classification, but is not limited thereto. For example, image data (predetermined area portion of the document data developed as an image in the memory unit 22) included in a predetermined area of the document itself created by the user is reduced, or image data (for one page) representing the entire document (for one page) It is also possible to create label information by reducing the document data developed in the memory unit 22 as an image.
[0061]
In such a case, since various label information is created according to the created document, it is difficult to specify search data corresponding to the label information when searching for a document. In order to solve this, the process according to the procedure shown in FIG. 13 as described above is executed before the search process. That is, a list of created label information (reduced images) is displayed on the display device 25, and a corresponding document is searched using the label information selected from the label information group of the list as search data.
[0062]
In each of the above examples, one label information is created corresponding to one document, but the present invention is not limited to this. It is also possible to create a plurality of label information for one document and manage the label information so as to be associated with one document.
For example, as shown in FIG. 14, in the created image data document (AAA), the image data portion α₁, Α₂,. . . α_nIs specified, the specified image data part is reduced and normalized (thumbnail processing), and the corresponding label information a₁, A₂,. . . , A_nIs created. The created label information is stored in the file storage unit 26 together with the created document. For example, the relationship between the image data document (AAA, BBB, etc.) and the label information is described in the label management table as shown in FIG.
[0063]
Also in this case, according to the same process as described above (see FIG. 12), when certain label information is designated as search data, the corresponding image data document is searched with reference to the label management table.
In each of the above examples, the document search is performed using the label information. Next, search processing for image data in each document will be described.
[0064]
Such a search process is performed, for example, according to the procedure shown in FIG.
In FIG. 16, when the user inputs image data (α) as a search key using the handwriting input device 23, the CPU 20 initializes various coefficients (S51), and then inputs the input image data (α ) Is acquired (S52). Then, the CPU 20 normalizes the acquired image data (α) (S53), and generates search data corresponding to label image information to be searched from the normalized image data (reduced image) obtained as a result. (S54).
[0065]
Thereafter, the CPU 20 reads the image data document from the file storage unit 26 and develops it in the memory unit 22 (S55). Then, the CPU 20 scans the image data document developed in the memory unit 22 (S56) and determines whether or not an image data portion exists (S57). If it is determined that the image data portion exists before reaching the final end of the read document (S58), the CPU 20 extracts the image data portion (S59).
[0066]
When the image data portion is extracted in this way, the CPU 20 normalizes the extracted image data portion (S60), and label information (a) (reduced image) is obtained from the normalized image data obtained as a result. Is created (S61). Then, the CPU 20 determines whether or not the search data generated from the handwritten input image as described above matches the label information (a) (S62). In this matching determination process, as in the above-described example (see FIG. 12), the label information having a predetermined similarity (substantially the same) as the search data is determined to match the search data.
[0067]
If it is determined that the search data matches the label information (a), the CPU 20 causes the display device 25 to display that the extracted image data portion corresponds to the search data (S64). Thereafter, the CPU 20 determines whether or not there is an operation input for interrupting the process (S65). If there is no operation input for such processing interruption, scanning of the image data document is started again, and processing for extracting the image data portion and whether or not the extracted image data portion matches the search data. Is continued (S56 to S65).
[0068]
If the extracted image data portion does not match the search data, scanning of the image data document is continued as it is (S63 → S56).
When the above processing is repeated and scanning is completed up to the final end of the read document, the CPU 20 further determines whether or not there is an image data portion that matches the search data in the above-described process (S66). If the matching image data does not exist in the read document, the CPU 20 displays a message indicating that the corresponding document data portion does not exist on the display device 25 (S67).
[0069]
Thereafter, the CPU 20 determines whether or not a request for processing for the next document has been made. If such a request has been made, the next image data document is used as a new search target document. The data is read from the file storage unit 26 and expanded in the memory unit 22 (S68 → S55). Thereafter, the same processing as described above is performed on the new image data document.
[0070]
If the display device 25 displays that the extracted image data portion corresponds to the search data and determines that an interruption operation has been performed, the CPU 20 ends the search process for the document, and the next document It is determined whether or not the search process should be performed (S68).
[0071]
According to the search processing as described above, image data that matches the image data input from the handwriting input device 23 is searched from each image data document.
In the system that performs the document search process based on the label information described above, the search process is performed using the label management table that describes the relationship between the label information and the image data document. However, the present invention is not limited to this. In particular, every time search data is given without using a label management table, the image data of a predetermined area of the document is read according to the same procedure as shown in FIG. 16, and the read image data and search data The comparison process and the coincidence determination process may be performed.
[0072]
As described above, for example, as shown in FIG. 17, when the image data represents “Mt. Fuji”, as shown in FIG. 18A, the reduced data of “Mt. Fuji” is used as label information. As shown in (b), it is also possible to register a trapezoidal line drawing, which is the frame data of “Mt. Fuji”, as label information (described in a label management table).
[0073]
Moreover, although each Example mentioned above produced label information from image data, it is not limited to this. Label information can also be created from audio data, composite data of audio data and image data, or the like.
Furthermore, although the example which the system based on this invention constructed | assembled in the portable computer terminal device was demonstrated, it is not restricted to this. It can also be built in a normal desktop or laptop personal computer. In this case, the file storage unit 26 is composed of, for example, a hard disk device. Various programs can be provided to the system by a recording medium such as a CD-ROM, a floppy disk, and various memory cards.
[0074]
In each of the above examples, the processing in step S1 shown in FIG. 4 corresponds to the extraction means (extraction program means), and the processing in steps S2 to S4 corresponds to the label information generation means (label information generation program means). 7, 10 and 11, and the tables shown in FIG. 15 respectively correspond to management means. The processing according to each procedure in FIGS. 8 and 12 corresponds to search means (search program means).
[0075]
  Also, the processing of steps S56 to S59 shown in FIG.)Correspondingly, the processing in steps S60 and S61 shown in FIG.)Correspondingly, the processing of steps S62 and S63 shown in FIG.)Correspond.
[0076]
Further, the process of step S6 shown in FIG. 4 corresponds to a program means for providing the management means with the correspondence between the document and the label information, and the process of the procedure shown in FIG. 13 corresponds to the label list display control means.
[0077]
【The invention's effect】
  As described above, claims 1 to2According to the described invention of this application,voiceSince label information was created from data and document search was performed based on the correspondence between the label information and the document,voiceA system capable of performing a search process based on data can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a functional configuration of a system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an example of a hardware configuration of a system according to the embodiment of the present invention.
FIG. 3 is a diagram illustrating an example of a specific area of an image data document from which image data is to be extracted.
FIG. 4 is a flowchart illustrating an example of a processing procedure when a created document is stored.
FIG. 5 is a diagram illustrating an example of an image to be described in a specific area of a document.
FIG. 6 is a diagram illustrating another example of an image to be described in a specific area of a document.
FIG. 7 is a diagram showing a label management table describing a relationship between a document and label information.
FIG. 8 is a flowchart illustrating an example of a procedure of document search processing.
FIG. 9 is a diagram showing still another example of an image to be described in a special case area of a document.
FIG. 10 is a diagram showing a label storage position table describing the relationship between label information and its storage position.
FIG. 11 is a diagram showing a label management table describing the correspondence between label information and documents.
FIG. 12 is a flowchart illustrating an example of a procedure of document search processing.
FIG. 13 is a flowchart illustrating an example of a processing procedure for displaying a list of label information.
FIG. 14 is a diagram showing a correspondence between image data in a created and edited image data document and label information.
FIG. 15 is a diagram showing a label management table describing the correspondence between documents and label information.
FIG. 16 is a flowchart illustrating an example of a procedure of image data search processing;
FIG. 17 is a diagram illustrating an example of image data.
FIG. 18 is a diagram illustrating an example of label information.
[Explanation of symbols]
1 Information processing system
10 Document management unit
11 Management unit
12 Display unit
13 Audio output unit
14 Input unit
15 Input / output control unit
16 Feature extraction unit
17 Compression unit
18 Display control unit
19 Search unit
20 CPU
21 ROM
22 Memory unit
23 Handwriting input device
24 keyboard
25 Display device
26 File storage unit

Claims (2)

In an information processing system that processes documents containing unencoded audio data,
An extraction means for extracting audio data included in a designated document when a search command is issued;
Label information generating means for generating label information including a main frequency component representing the characteristics of the audio data from the audio data extracted by the extracting means;
Whether main frequency component information, which is label information as a search key provided together with the search command, and main frequency component information, which is label information generated by the label information generating means, are substantially the same. Determination means for determining whether or not
An information processing system configured to obtain a search result based on the determination result when the determination unit determines that the main frequency component information of both the label information is substantially the same. The information processing system according to claim 1, wherein the extraction unit extracts voice data recorded in a predetermined area of the document.

Images