JP4964080B2 - Image processing system, image processing method, and image processing program - Google Patents

Description

  The present invention relates to a technique for extracting an object included in an image of image data and giving meaning to the extracted object in the layout of the image data.

  Conventionally, a paper document is scanned into electronic data, an area analysis is performed on the digitized document image information to extract an object such as a character area, and text information of the character area is obtained by OCR processing. An image processing system that performs a predetermined process based on text information is known.

  In the conventional image processing system described above, the meaning (title, header, body paragraph, etc.) in the layout of a certain character area is extracted, and the text attribute of the text information in the character area (character string “2006/01/25”) [Date], etc.) (for example, see Patent Document 1).

Further, there is also known a technique that makes it possible to correct the meaning determined for an object extracted from an image of image data and the attribute determined for text using an interface such as a GUI (for example, (See Patent Document 2).
Japanese Patent Laid-Open No. 11-250041 JP 2001-344562 A

  In the above conventional image processing system, only a single meaning finally determined for an object included in an image can be notified to the user. Therefore, the user must determine whether or not the semantic determination process automatically performed by the image processing system is appropriate based only on the finally determined single meaning. There wasn't.

  In addition, when it is desired to correct the result of the semantic determination automatically performed on an object included in the image (for example, when an object whose meaning is determined is not appropriate, it is changed to an appropriate object, etc.) ), The user has to determine a correction candidate that he / she thinks appropriate, which is a burden of operation.

  The present invention has been made to solve the above-described problems, and contributes to reducing the operation burden when managing the processing results of the semantic determination processing performed on the objects included in the image. It aims at providing the technology that can do.

In order to solve the above-described problem, an image processing system according to an aspect of the present invention includes an image data acquisition unit that acquires image data, and an object included in an image of the image data acquired by the image data acquisition unit. An object extracting unit for extracting the object, a meaning determining unit for determining the meaning of the object extracted by the object extracting unit in the layout of the image data based on a predetermined determination rule, and a plurality of the meaning determining unit When it is determined that the objects have the same meaning, a notification unit that displays a list of information related to the plurality of objects determined to have the meaning in association with the information related to the meaning, and accepts a user operation input An operation input unit and an object selected by a user operation input received by the operation input unit, It includes a meaning setting unit that sets as having a meaning that is determined by the meaning determining part for an object, and a history information acquisition unit that acquires the selection history of the object by the operation input unit, the means setting section, When it is determined that the plurality of objects have the same meaning in the meaning determination unit, based on the history information acquired by the history information acquisition unit, of the objects displayed as a list by the notification unit The object having the highest frequency selected in the past is set as having the meaning determined by the meaning determination unit for the object .

The image processing method according to one aspect of the present invention acquires image data, extracts an object included in an image of the acquired image data, and extracts the extracted object based on a predetermined determination rule. When the meaning of the image data in the layout is determined and it is determined that the plurality of objects have the same meaning, the information on the plurality of objects determined to have the meaning is associated with the information on the meaning Display the list, accept the user's operation input, set the object selected by the accepted user's operation input as having the determined meaning for the object, obtain the selection history of the accepted object, If it is determined that the objects have the same meaning, Based on the gravel information, the highest object is selected frequencies in the past of the objects to be listed, it may be characterized by setting as having a meaning that is determined for the object.

An image processing program according to an aspect of the present invention acquires image data, extracts an object included in an image of the acquired image data, and extracts the extracted object based on a predetermined determination rule. When the meaning of the image data in the layout is determined and it is determined that the plurality of objects have the same meaning, the information on the plurality of objects determined to have the meaning is associated with the information on the meaning Display the list, set the object selected by the user's operation input as having the meaning determined for the object, obtain the selection history of the object selected by the user's operation input , a plurality of objects When it is determined that they have the same meaning, the history information of the acquired object Zui, it is possible, characterized in that the highest object selected frequency in the past of the objects to be listed, to execute processing of setting as having a meaning that is determined for the object in the computer .

  As described above in detail, according to the present invention, there is provided a technique capable of contributing to the reduction of the operation burden when managing the processing result of the semantic determination process performed on the object included in the image. can do.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described.

  FIG. 1 is a functional block diagram for explaining an image processing system S1 according to the first embodiment of the present invention.

  The image processing system S1 according to the present embodiment includes an image data acquisition unit 101, a document region analysis module 102, a region meaning analysis module 104, a region meaning presentation module 108, a region meaning setting module 110, a display unit 111, an operation input unit 112, A character recognition module 113, a semantic information storage module 114, a text semantic analysis module 115, a CPU 801, and a memory 802 are provided. The image processing system S1 can be realized by, for example, an MFP (Multi Function Peripheral) having these configurations.

  The image data acquisition unit 101 acquires image data to be processed in the image processing system S1. Hereinafter, as an example, a case where image data obtained by scanning from a document is acquired will be described as an example. However, the present invention is not limited thereto, and as a result, if the image data is digitized, data in the image data acquisition unit 101 is used. It becomes the object of acquisition. Also,

  A document area analysis module (corresponding to an object extraction unit) 102 extracts an object included in an image of image data acquired by the image data acquisition unit 101. Specifically, the document area analysis module 102 has, for example, the position and size of the rectangular area surrounding the character string in the image of the image data (or the upper left position coordinates and the lower right position coordinates of the rectangular area). Analysis is performed, and an object such as a character area included in the image of the image data is extracted.

  An area semantic analysis module (equivalent to a semantic determination unit) 104 is in the process of laying out image data of an object extracted by the object extraction unit 102 based on a predetermined determination rule stored in a storage area such as the memory 802, for example. Is analyzed (determined) (for example, “title”, “header”, “footer”, “paragraph”, etc.). The analysis result (meaning of each object) in the area semantic analysis module 104 is stored in the semantic information storage module 114 in association with each object.

  When the area meaning presenting module (corresponding to the notification unit) 108 determines that a plurality of objects have the same meaning in the area meaning analyzing module 104 based on various types of information stored in the semantic information storage module 114. (For example, when it is determined that a plurality of objects are “titles”), information on the plurality of objects determined to have the meaning is displayed as a list on the display unit 111 in association with the information on the meaning Let Accordingly, the user can easily grasp what meaning (analysis result) the object included in the image is determined to have.

  The area meaning setting module (corresponding to the candidate selection section and the meaning setting section) 110 is an object that is displayed in a list on the display section 111 in accordance with a command from the area meaning presentation module 108 based on an operation input to the operation input section 112 by the user. Select one of. Then, the area meaning setting module 110 sets the object selected as described above as having the meaning determined by the area meaning analysis module 104 for the object.

  The display unit 111 includes, for example, a liquid crystal display or a CRT display, and has a role of displaying a processing result or the like in the image processing system S1 on the screen. Specifically, the display unit 111 can be realized by a GUI screen of an MFP control panel, a GUI screen of a PC, or the like.

  The operation input unit 112 includes, for example, a keyboard, a mouse, and the like, and has a role of receiving user operation inputs. Of course, the functions of the display unit 111 and the operation input unit 112 may be integrally realized by, for example, a touch panel display.

  The character recognition module 113 has a function of recognizing characters as text information from a character image included in the image data image.

  The semantic information storage module 114 is composed of a storage device such as an HDD, and has a role of integrating and storing the processing results in the document area analysis module 102, the area meaning analysis module 104, and the character recognition module 113. . In addition, the other storage modules described below are similarly configured from a storage device such as an HDD, and have a role as a database.

  The text semantic analysis module 115 determines the character or character string based on the text information obtained by the character recognition module 113 for the character or character string in the character area extracted by the document area analysis module 102. The text meaning (that is, attributes such as “date”, “company name”, “person name”) is analyzed.

  Specifically, the text semantic analysis module 115 analyzes that the attribute of the character string “2006/01/25” included in the text information of a certain character region extracted from the image is “date”. I do.

  The CPU 801 has a role of performing various processes in the image processing system according to the present embodiment, and also has a role of realizing various functions by executing a program stored in the memory 802. The memory 802 includes, for example, a ROM, a RAM, and the like, and has a role of storing various information and programs used in the image processing system.

  Details of the processing by the image processing system S1 according to this embodiment will be described below. Here, as an example, a case where a predetermined process is performed on image data obtained by scanning a document as shown in FIG. 2 will be described. FIG. 3 is a diagram for explaining the flow of processing in the first embodiment of the present invention.

  First, the document area analysis module 102 creates a character area such as “2006/01/25” or a character area such as “application guidance” in a document image in which objects such as character strings are arranged in the layout shown in FIG. Are extracted as character area 1 and character area 2, respectively.

  Specifically, when extracting an object such as a character area, for example, the document area analysis module 102 uses the upper left corner of the document image as the origin of coordinates, and the character area 1 is changed from coordinates (80, 10) to coordinates (120, 20). The process of recognizing the rectangular area is performed. Note that the object extraction process from the image by the document area analysis module 102 only needs to be able to specify the position of the object in the document image. For example, an expression method such as “the upper left coordinate of the rectangular area surrounding the character area 1 is (80, 10), the rectangular width is 40, and the height is 10” may be employed.

  Next, the meaning of the character area extracted by the document area analysis module 102 is analyzed by the area meaning analysis module 104.

  For example, the area semantic analysis module 104 performs analysis such that the character area 1 is a “header” and the character area 2 is a “title” in the document image to be processed.

  Next, the character recognition module 113 analyzes the image of the character area extracted by the document area analysis module 102 and acquires text information. For example, the character recognition module 113 analyzes that the text information of the character area 1 is “2006/01/25”.

  As described above, various information and analysis results acquired by the document area analysis module 102, the area meaning analysis module 104, and the character recognition module 113 are stored in the semantic information storage module 114.

  Here, as shown in FIG. 3, in addition to the position of the rectangular area surrounding the character area, the meaning of the area, and the text information, the text semantic analysis module 115 obtains the text meaning from the character string in the text information. If extraction is possible, the position of the rectangular area of the character string, the text meaning of the character string, and the text information of the character string are retained. Note that the semantic information storage module 114 can retain all pieces of information as long as several character strings included in one character area have text meanings individually. FIG. 4 is a diagram showing the relationship between various pieces of information stored in an integrated manner in the semantic information storage module 114.

  In this way, the user can set an object having a certain meaning by selecting from among the objects displayed in the list, which can greatly reduce the burden of inputting correction candidates for the object. it can.

  In addition, by modifying the meaning of an object in an image through a semantic determination process using a predetermined algorithm in advance, a significant change is made to a program for executing an existing semantic determination process. Without any changes, it is possible to make flexible changes to the meaning of objects.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  FIG. 5 is a functional block diagram for explaining an image processing system S2 according to the second embodiment of the present invention.

  The present embodiment is a modification of the above-described first embodiment. Hereinafter, parts having the same functions as those already described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. The present embodiment and the first embodiment are different in the display method of the association between the object and the meaning on the display unit 111.

  The image processing system S2 according to this embodiment includes an area meaning attribute storage module 116, an area meaning attribute presentation module 117, and an area meaning attribute setting module 118 in addition to the image processing system S1 according to the first embodiment. In addition to the functions shown in the first embodiment, the area semantic analysis module 104 according to the present embodiment has a certainty of object semantic analysis processing (semantic determination processing) performed by the region semantic analysis module 104 ( (Reliability) determination function (corresponding to the function of the reliability determination unit).

  The area semantic attribute storage module 116 has a role of acquiring attribute information such as reliability of the result of the area meaning determination process executed by the area semantic analysis module 104 and storing the information.

  The area semantic attribute presentation module 117 presents information stored in the semantic information storage module 114 in accordance with the attribute value stored in the area semantic attribute storage module 116. Therefore, in this embodiment, the area meaning presentation module 108 and the area meaning attribute presentation module 117 have a function corresponding to a notification unit.

  The area meaning presentation module 108 and the area meaning attribute presentation module 117 in the present embodiment cooperate with each other, and when the area meaning analysis module 104 determines that a plurality of objects have the same meaning, the meaning A plurality of objects that are determined to have, are associated with the meaning and displayed in a list on the display unit 111 in descending order of reliability determined by the area semantic analysis module 104.

  The area semantic attribute setting module 118 gives the user a means for manually setting the value of the attribute stored in the area semantic attribute storage module 116. Therefore, in this embodiment, the region meaning setting module 110 and the region meaning attribute setting module 118 have functions corresponding to a candidate selection unit and a meaning setting unit.

  Details of processing in the image processing system S2 according to the present embodiment will be described below.

  Here, text information of a character area whose area meaning is “title” is extracted from image data of a document to be processed in the image processing system S2, and the image of the document to be processed is based on the text information. Processing for generating a file name when saving data in a predetermined storage device will be described.

  For example, when image data of a document as shown in FIG. 2 is input to the image processing system S2 as processing target data, text information extracted as “title” (for example, “application guide”, “submission destination”, etc.) A file name such as “application guide-submission destination.pdf” is generated from a character string or the like and presented to the user. Then, the user prompts for confirmation on the file name generated as described above, and when the correction is made by the user, the corrected file name is assigned to the image data file and stored. FIG. 6 is a flowchart for explaining the flow of processing in the second embodiment of the present invention.

  First, the image data acquisition unit 101 acquires image data of a document to be processed in the document processing system S2 (S101).

  Next, the document area analysis module 102 analyzes a character area in the document image, and acquires rectangular coordinates (position and size, etc.) surrounding the character area. Information acquired by the document area analysis module 102 is notified to the semantic information storage module 114 and stored (S102).

  Next, the area meaning of the character area extracted by the document area analysis module 102 (extracted object) based on a predetermined determination rule stored in a storage area such as the memory 802 by the area semantic analysis module 104, for example. Of the image data in the layout). The area semantic analysis module 104 also determines the reliability of the semantic determination process as well as the semantic determination process of the object included in the image data.

  Specifically, the determination process of the reliability (probability) of the meaning determination process in the area semantic analysis module 104 is, for example, a determination of a character area whose meaning is “title”. A score of 10 points is assigned when the character area is close, and 20 points are assigned when the character area is large, and the total score obtained as a result of the scoring is used as an index of the certainty of the meaning determination process. The total score obtained as a result of the above scoring is stored in the semantic information storage module 114.

  Information on the area meaning acquired by the area semantic analysis module 104 in this way is notified to the semantic information storage module 114 and stored. At this time, the area semantic analysis module 104 obtains the candidate rank of each character area based on the probability (reliability) that the analysis result is correct, and notifies the area semantic attribute storage module 116 of this candidate rank as an attribute. And store it (S103).

  For the image data of the document illustrated in FIG. 2, the area semantic analysis module 104 determines the area meanings of the character area 2 having text information “application guidance” and the character area 3 having text information “submission destination”, for example. When interpreted as “title”, the analysis result is notified to the semantic information storage module 114 and stored. Specifically, the area semantic analysis module 104 has a high possibility that the character area 2 obtained when analyzing the area meaning is the most “title”, and the character area 3 may be the second “title”. The probability ranking of the analysis result that is high is notified to the area meaning attribute storage module 116 and stored in the form of a data table or the like as shown in FIG. 7 (S103).

  Next, the character recognition module 113 acquires text information of the character area analyzed by the document area analysis module 102. The text information acquired by the character recognition module 113 is notified to the semantic information storage module 114 and stored (S104). For example, when the text recognition module 113 acquires the text information of the character area 2 of “application guidance”, the text recognition module 113 notifies the semantic information storage module 114 to store the text information.

  Next, the area semantic attribute presentation module 117 arranges the character areas stored in the semantic information storage module 114 in the order of the probability of the analysis results stored in the area semantic attribute storage module 116, and displays the display unit 111. Is displayed on the screen and presented to the user (S105). Such information presentation by the region meaning attribute presentation module 117 is performed on a GUI screen as shown in FIG. 8, for example.

  On the GUI screen displayed on the display unit 111, in the window 201, which is a preview window, a rectangle surrounding the character area extracted by the document area analysis module 102 is displayed in addition to the input document image. Further, the rectangular frame surrounding the character area analyzed by the area semantic analysis module 104 as having the area meaning “title” is highlighted in red. In the list box 202, text information of a plurality of character areas interpreted as having the meaning of “title” is arranged in the order of the probability (reliability) when analyzed by the area semantic analysis module 104. Listed in association with the meaning of “title”. Specifically, in the example shown in FIG. 8, the character area “application guidance” is analyzed to be the first “title”, and the character area “submission destination” is analyzed to be the second “title”. In this order (in descending order of reliability), the list box 202 displays a list.

  Next, when there is a candidate that the user determines to be not “title” in the list of character areas analyzed as “title” as the area meaning presented to the user (S106, Yes), The area meaning setting module 110 excludes the candidate that is not the “title” from the list based on the user's operation input to the operation input unit 112 (S107).

  At this time, the semantic information storage module 114 is notified of information that the area meaning of the character area deleted from the list is not [title]. The area meaning setting module 110 provides a function of giving an operation interface to the user and notifying the semantic information storage module 114 of the operation result. For example, if there is a candidate that the user determines that the area meaning is not “title” from the list of character areas displayed in the list box 202 on the GUI screen shown in FIG. By selecting the candidate item and pressing the command button 205 which is a delete button, the selected item can be deleted from the list.

  Next, in the character area list box 202 in which the area meanings presented to the user on the display unit 111 are arranged in the order of “title”, the user determines and arranges the character areas in the order of “title”. If it is determined that it has not been performed (S108, Yes), the area meaning attribute setting module 118 corrects the ranking of the candidate character areas in the list based on the user's operation input to the operation input unit 112 (S109).

  Information relating to the arrangement order of the character areas after the change of the list whose order has been changed in this way (order that seems to be [title]) is notified to the area meaning attribute storage module 116 by the area meaning attribute setting module 118. A function of giving an operation interface to the user and notifying the result of the operation to the area meaning attribute storage module 116 is provided by the area meaning attribute setting module 118.

  For example, when an item is selected from the list of character areas determined as “title” displayed in the list box 202 on the GUI screen shown in FIG. When the command button 204 is pressed when the order of the item one higher is switched, the order is switched with the item one lower than the selected item.

  Further, the command box 205 and the list box 202 edited by the command buttons 203 and 204 can be returned to the initially presented state by pressing the command button 206.

  Next, in the order of the character areas arranged in the order of “title”, the text information is connected with “-” (hyphen) or the like, and an extension is given to generate a file name (S110). In FIG. 8, as an example, the file name generated by concatenating the text information with “-” in order of the items displayed in the list box 1-2 and giving the extension “.pdf” is the text box 207. Is displayed.

  Lastly, the image data to be processed is stored in a storage device (for example, the memory 802) with the file name generated in S110. As a specific operation, for example, when the save button of the command button 209 is pressed, the text information displayed in the text box 207 is set as the file name and the image data is saved.

  In the GUI screen shown in FIG. 8, text information of the character area extracted as “title” is displayed in the text box 207, but the recognition result of the character recognition module 113 is incorrect, and the displayed text information is displayed. May become wrong. In such a case, the user can correct misrecognition of the character recognition module 113 by directly inputting the text into the text box 207 by the operation input to the operation input unit 112.

  In the present embodiment, the file name is generated using only the text information of the character area whose area meaning is “title”. However, the present invention is not limited to this, and the character whose area meaning is “header”. It is possible to use text information of various other meanings, such as using text information of a region as a file name. In addition, as an example of a predetermined process for image data to be processed, a file name is generated based on text information having a specific area meaning. In addition, the area meaning and text information modified by the user are also shown. Can be used as a search keyword for image data of a document.

  As described above, with the configuration of the present embodiment, the user can select an object that has a specific meaning as an object ranked higher among a plurality of objects displayed in the list (the meaning of the object). It is possible to confirm from an object that is highly likely to have). As a result, the user can perform an operation of searching for an object that most closely matches the meaning among a group of objects determined to have a certain meaning with a small burden.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  FIG. 9 is a functional block diagram for explaining an image processing system S3 according to the third embodiment of the present invention.

  This embodiment is a modification of the above-described first and second embodiments. Hereinafter, parts having the same functions as those already described in the first and second embodiments are denoted by the same reference numerals, and description thereof is omitted. The image processing system S3 according to the present embodiment is different from the above-described embodiments in a method for correcting meanings for object regions extracted from image data.

  Specifically, the image processing system S3 according to the present embodiment includes an area meaning setting tendency analysis module (corresponding to a history information acquisition unit and an identification information acquisition unit) 119, in addition to the image processing system S2 according to the second embodiment. An area meaning document tendency comparison module (corresponding to an image type determination unit) 120, an area meaning setting tendency storage module 121, and an area meaning fixed setting module (corresponding to a meaning setting unit, a history information acquisition unit, and an identification information acquisition unit) 122 are provided. It becomes the composition which becomes.

  An area meaning setting trend analysis module (corresponding to a history information acquisition unit and an identification information acquisition unit) 119 performs an area meaning setting module 110 and an area meaning attribute setting module 118 on the document information stored in the semantic information storage module 114. The information (selection history) related to the user's correction (object selection) operation set via the user ID is acquired, and whether there is a tendency in the user's correction method is analyzed. Here, the area meaning setting tendency analysis module 119 is a new processing target in order to specify the content of the correction operation performed through the area meaning setting module 110 and the area meaning attribute setting module 118 for each user. Information for identifying a user who operates the region meaning setting module 110 and the region meaning attribute setting module 118 using the operation input unit 112 for the document (ID information, password, biometrics input at the time of password authentication) Information on fingerprints, pupils, voiceprints, etc. acquired at the time of authentication). The user identification information acquired in this manner includes the semantic information storage module 114 in association with the processing contents (selection, setting, correction, etc.) including the case where the correction operation is performed on the document that has been processed in the past. Stored in

  The area meaning setting trend storage module 121 stores the correction contents of the user made on a certain document analyzed by the area meaning setting tendency analysis module 119 in association with the document.

  The area meaning document tendency comparison module (image type determination unit) 120 converts the image of the document to be processed acquired by the image data acquisition unit 101 into the image of the document stored in the area meaning setting trend storage module 121. By comparing whether or not they are similar, the type of the image data to be processed is determined.

  An area semantic fixed setting module (equivalent to a meaning setting unit, a history information acquisition unit, and an identification information acquisition unit) 122 uses an area semantic document tendency comparison module 120 to store image data similar to the image data of a document to be processed. When stored in the semantic setting trend storage module 121, the area semantic setting trend analysis module 119 obtains information regarding past correction operations (such as object selection history) performed by the user on a document having the tendency. Is acquired from the region meaning setting trend storage module 121 based on the user identification information, and the same content correction as the correction operation is applied to the information stored in the semantic information storage module 114 or the region meaning attribute storage module 116. As described above, the area meaning fixed form setting module 122 acquires information regarding the object selection history in the area meaning setting module 110 for the type of image data determined by the area meaning document tendency comparison module 120.

  By adopting such a configuration, it is possible to grasp from the history information what kind of meaning is often given to an object in a certain type of image (for example, a document image of a certain format), and its meaning The tendency of attaching can be reflected in the meaning given to the object.

  In the present embodiment, the tendency of the user's past correction operation with respect to the analysis result of the area meaning and the analysis result of the attribute of the area meaning of each object included in the input image data is analyzed and newly input If the document is similar to a document input in the past, a correction operation performed by the user in the past is performed in advance on the input document, and then the result is presented to the user.

  In the image processing system S3 according to this embodiment, when image data of a document as illustrated in FIG. 2 is input and there are two character areas whose area meaning is determined to be “title”, the “title” The text information of the character area determined to be is displayed as a list on the display unit 111 in the order of the probability. Here, based on the result presented on the display unit 111, the user determines that the second candidate of the character area listed as the “title” candidate is not “title”, and the area meaning setting module 110. Suppose you have performed an operation to delete from candidates.

  At this time, the information on the input image data and the user's correction operation are recorded in the area meaning setting tendency storage module 121. Next, when image data to be newly processed is input, if the image data processed in the past is similar in layout, etc., the user's correction operation performed in the past on the image data Are automatically applied to newly input image data. In other words, in this case, the second candidate is deleted from the two character areas determined as candidates for the “title”. In this way, by presenting the result of the deletion of the second candidate to the user, the user performed the second character area candidate determined as “title”, which was performed on similar image data in the past. The delete operation can be omitted for newly input similar documents.

  FIG. 10 is a flowchart for explaining a process of acquiring a tendency of a correction operation performed by the user on the document image data input to the document processing system S3.

  First, the image data acquisition unit 101 acquires an image of a document input to the document processing system 13 (S201).

  Next, the document image is analyzed. The document area analysis module 102 analyzes the character area in the document image, information on the coordinates (position, size, etc.) of the rectangle surrounding the character area, and the area meaning of the character area acquired by the area semantic analysis module 104 The text information acquired by the character recognition module 113 is notified to the semantic information storage module 114 and stored (S202).

  Next, the region semantic attribute storage module 116 is notified of the rank of the reliability (high reliability) of the region semantic analysis result analyzed by the region semantic analysis module 104 and stored (S203).

  Next, the information stored in the semantic information storage module 114 and the regional meaning attribute storage module 116 is presented to the user by the regional meaning attribute presentation module 117 (S204).

  Next, the user confirms the analysis result presented on the display unit 111, and if necessary, corrects the area meaning or the area meaning attribute if necessary, by the area meaning setting module 110 or the area meaning attribute setting module 118, respectively. (S205).

  If the user makes corrections using the region meaning setting module 110 or the region meaning attribute setting module 118 in the processing in S205 (S205, Yes), the region meaning setting trend analysis module 119 causes the region meaning setting module 110 to perform the correction. Alternatively, the user's correction operation instructed to the area meaning attribute setting module 118, information on the input document (information stored in the semantic information storage module 114), and the user who performed the correction operation are identified. And the area meaning setting tendency storage module 121 is notified and stored (S206).

  For example, as information related to image data of an input document, a user has performed in the past on an image similar to the document layout of the image data and the coordinates of all character regions analyzed by the document region analysis module 102. Then, the region meaning setting tendency storage module 121 stores them in association with the correction work of excluding the second character region candidate determined as “title” from the candidates. FIG. 11 shows an area meaning setting tendency storage module 121 in which the coordinates of each character area included in the image data of the document are associated with the contents of the correction operation performed by the user in the past on the document image similar to the document. It is a figure which shows the condition currently stored in.

  FIG. 12 is a flowchart for explaining details of processing in the image processing system S3 according to the present embodiment.

  In the image processing system S3 according to the present embodiment, when the image data of the input document is similar to the layout of the document stored (previously input) in the area meaning setting tendency storage module 121, The user's correction operation performed on the previous document is performed on the image data of the newly input document and presented to the user.

  First, the image data acquisition unit 101 acquires an image of a document input to the document processing system 13 (S301).

  Next, the input document image is analyzed. Information on the coordinates (position, size, etc.) of the rectangle surrounding the character area obtained by analyzing the character area in the document image by the document area analysis module 102, and the area meaning of the character area acquired by the area semantic analysis module 104 Then, the text information acquired by the character recognition module 113 is associated with the semantic information storage module 114 and stored (S302).

  Next, the order of the probability of the analysis result of the area meaning analyzed by the area semantic analysis module 104 is notified to the area meaning attribute storage module 116 and stored (S303).

  Next, whether or not a document having a layout similar to the input document is stored in the region meaning setting tendency storage module 121 is compared and determined by the region meaning document tendency comparison module 120 (S304). As for comparison of documents, for example, as shown in FIG. 11, when coordinate information of all character areas extracted from a document previously input in the area meaning setting tendency storage module 121 is stored, the document area Compared with the coordinates of the character area extracted from the image data of the document newly acquired by the analysis module 102, if the difference between the values is small, it is determined that the two documents are similar.

  Here, an example is shown in which the similarity of a document is determined using only the coordinates of a character area included in the document image, but the present invention is not limited to this. For example, the text information extracted from the document image is used and stored in the semantic information storage module 114 and the region semantic attribute storage module 116 such that when there are many common words in two documents, it is determined that they are similar. A variety of information can be used to compare documents.

  Next, when a document similar to the input document is stored in the area meaning setting tendency storage module 121 (Yes in S304), the area meaning setting tendency storage module 121 is added to the image data of the newly input document. The region semantic fixed setting module 122 executes correction operations previously performed by the user on similar documents stored in (S305).

  For example, if the area-meaning document tendency comparison module 120 determines that the input document is similar to the document A in which information is stored in the area meaning setting tendency storage module 121 as shown in FIG. The semantic fixed form setting module 122 performs a correction operation performed on the document A stored in the area meaning setting tendency storage module 121 by the user, an operation of “excluding the second [title] character area candidate”. Execute. That is, the area meaning setting module 110 is instructed to correct the area meaning of the character area that has been analyzed that the area meaning seems to be the second [title].

  Next, in S304, when a document similar to the input document is stored in the area meaning setting tendency storage module 121 (S304, Yes), there is no similar document as a result of correction in S305. In the case (S304, No), the analysis results in S302 and S303 are presented to the user by the region meaning attribute presentation module 117 (S306).

  Next, when the user corrects the presented content (S307, Yes), in S308, similarly to S206 shown in FIG. 10, the input document and user are input by the area meaning setting trend analysis module 119. Is stored in the area meaning setting tendency storage module 121 (S308), and the process is terminated. On the other hand, if the user has not corrected the presented content (S307, No), the process is terminated.

  In this embodiment, when there is processed image data similar to the image data to be processed, a configuration in which the same correction as that added to the processed image data is shown. It is not limited to. For example, when there are a plurality of processed image data having a layout similar to that of the image data to be processed, the region semantic fixed setting module 122 causes the region semantic analysis module 104 to have the same meaning for the plurality of objects. Objects that have been selected in the past among the objects displayed in a list by the area meaning presentation module 108 based on the history information acquired by the area meaning setting trend analysis module 119 when it is determined that May be set as having the meaning determined by the area semantic analysis module 104 for the object. In this way, by automatically reflecting the past selection operation history (trend) in the semantic setting process for the object included in the image, the user does not need to make the same correction to a similar document each time. It can contribute to the improvement of convenience.

  In the present embodiment, the configuration in which the correction processing based on the past correction history is performed based on whether or not there is processed image data similar to the image data to be processed is shown, but the present invention is not limited thereto. It is not a thing. For example, the region meaning setting tendency analysis module 119 acquires identification information for identifying a user who selects an object, and acquires information related to an object selection history by the user corresponding to the acquired identification information. Also good. By adopting such a configuration, it is possible to grasp from a history information what kind of meaning a user gives to an object in the image, and to give meaning to the object how to select the meaning. Can be reflected.

  Further, a correction operation to be acquired in the case of a configuration in which correction contents and the like made in the past for a document having a certain layout are automatically reflected to a new input document having a layout similar to a document input in the past. The history information is not necessarily limited to that of a single user, but the operation history of a plurality of users belonging to a predetermined group can be acquired or a document with a similar document layout can be obtained in the past regardless of the user. It is also possible to acquire all the history of corrective operations performed.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.

  FIG. 13 is a functional block diagram showing a configuration of an image processing system S4 according to the fourth embodiment of the present invention. This embodiment is a modification of the first embodiment. In the image processing system S4 according to the present embodiment, the area meaning presentation module 108 and the area meaning setting module 110 in the image processing system S1 according to the first embodiment are changed to a text meaning presentation module 123 and a text meaning setting module 124. It has a configuration.

  The text meaning presentation module 123 presents the text meaning stored in the semantic information storage module 114 to the user by causing the display unit 111 to display the screen.

  The text meaning setting module 124 gives the user means for manually setting the text meaning stored in the semantic information storage module 114 by an operation input to the operation input unit 112.

  With such a configuration, the analysis result of the text meaning stored in the semantic information storage module 114 is presented to the user by the text meaning presentation module 123. The user can confirm the presented analysis result by himself / herself, and can confirm a location that has been erroneously analyzed.

  Further, the text semantic information setting module 124 allows the user to set or correct the text semantic analysis result output by the text semantic analysis module 115.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described.

  FIG. 14 is a functional block diagram showing a configuration of an image processing system S5 according to the fifth embodiment of the present invention. The present embodiment is a modification of the above-described fourth embodiment. The image processing system S5 according to the present embodiment includes a text semantic attribute storage module 125, a text semantic attribute presentation module 126, and a text semantic attribute setting module 127 in addition to the configuration of the image processing system S4 according to the fourth embodiment. It becomes the composition which becomes.

  The text meaning attribute storage module 125 acquires attributes such as the probability that the text meaning result analyzed by the text meaning analysis module (corresponding to the character attribute determination unit) 115 is correct, and stores the information. When the object extracted by the document area analysis module 102 is a character area, the text semantic analysis module 115 determines the attribute of the character included in the character area. Information regarding the character attributes determined by the text semantic analysis module 115 is stored in the text semantic attribute storage module 125.

  The determination of the probability that the text meaning result in the text meaning analysis module 115 is correct is, for example, whether or not it is “person name”, in the vicinity of the text information to be determined, If there is a character such as “sama”, a score of 10 is assigned according to a predetermined rule, and the higher the score obtained as a result of this scoring, the more likely it is to be a “person name”. Predetermined rules for determining the likelihood that the result of the text meaning determination in the text meaning analysis module 115 is correct are stored in the memory 802, for example.

  The text semantic attribute presentation module (input candidate display unit) 126 displays information stored in the semantic information storage module 114 according to the value of the text attribute stored in the text semantic attribute storage module 125. Display on the screen.

  The text meaning attribute setting module 127 gives the user a means for manually setting the value of the attribute stored in the text meaning attribute storage module 125 based on the operation input to the operation input unit 112.

  In the present embodiment, text information of a character region whose text meaning is “person name” is extracted from the image data of the document input to the image processing system S5. When multiple “person names” are detected from the image data of the input document, the “person names” are ordered in a higher order as the user's desired order or the text meaning determination result is more likely to be correct. ”Is generated. In accordance with the items and order listed in the “person name” list, the transmission destination for saving the input document image changes.

  For example, when the text information of “person name” “Tokyo Taro” is acquired from the document, the document is transmitted to the destination A, and when the list of “Tokyo Taro” and “Jiro Osaka” is acquired, the document is transmitted. If the order is “Jiro Osaka” and “Taro Tokyo” in this order, the document can be sent to the destination C.

  FIG. 15 is a flowchart for explaining details of processing of the image processing system S5 according to the fifth embodiment of the present invention. Hereinafter, as an example, a description will be given of a process in which a user appropriately corrects a result obtained automatically from a document and transmits the document according to a “person name” list generated based on the corrected information. In the generation process of the “person name” list here, a list arranged in the order desired by the user is created.

  First, the image data acquisition unit 101 acquires image data of a document input to the image processing system S5 (S401).

  Next, the document area analysis module 102 analyzes the character area in the document image, and obtains the coordinates (position and size, etc.) of the rectangle surrounding the character area (S402). The information acquired in this way is notified to the semantic information storage module 114 and stored.

  Next, the text recognition module 113 acquires text information of the character area. The acquired text information is stored in the semantic information storage module 114. For example, when the text information “application guidance” is acquired for the character area 2, the text information is stored in the semantic information storage module 114 (S403).

  Next, the text meaning analysis module 115 acquires the text meaning of the text information in the extracted character area. Then, the semantic information storage module 114 stores the acquired text meaning information. Furthermore, at this time, the ranking is assigned to all text information to which the text meaning is given in the order of the text information desired by the user. This ranking is notified to the text meaning attribute storage module 125 as an attribute and stored (S404).

  For example, in the image data of the document with the layout as illustrated in FIG. 2, the text meaning of the text information “Taro Tokyo” or “Jiro Osaka” is analyzed as “person name”, and the analysis result is sent to the semantic information storage module 114. Stored. Furthermore, “Taro Tokyo” is the “person name” desired by the first user and “Jiro Osaka” is the second desired “person name” for the text information assigned the text meaning by the text semantic analysis module 115. Is stored in the text semantic attribute storage module 125 as an attribute. FIG. 16 is an example of a data table showing text semantic attributes stored in the text semantic attribute storage module 125.

  Next, the text semantic attribute presentation module 126 arranges the text information stored in the semantic information storage module 114 in the order desired by the user stored in the text semantic attribute storage module 125, and the display unit 111 The screen is displayed (S405). FIG. 17 is a diagram illustrating a display example of a GUI displayed on the display unit 111 by the text meaning attribute presentation module 126. As described above, when the extracted object is a character, the text meaning attribute presentation module 126 determines the attribute of the character, and inputs the character with the determined attribute for the character with the determined attribute. Are displayed as input character candidates in a predetermined process.

  In the GUI screen shown in FIG. 17, in a window 301 that is a preview window, in addition to the input document image, a rectangle surrounding the character area extracted by the document area analysis module 102 is displayed. Further, the rectangular border of the character area whose text meaning is analyzed as “person name” by the text meaning analysis module 115 is highlighted in red. In the list box 302, the text information analyzed by the area semantic analysis module 104 as having the meaning “title” is arranged in a desired order given by the text semantic analysis module 115 and displayed in a list. Yes.

  For example, in the example shown in FIG. 17, the character area “Taro Tokyo” is analyzed as the “person name” desired by the user first, and the character area “Jiro Osaka” is the second “person name” desired by the user. Therefore, the list is displayed in the list box 302 in this order.

  Next, in the list of text information whose text meaning presented to the user on the display unit 111 is analyzed as “person name”, there is a candidate that seems not to be the desired “person name” as judged by the user. In the case (S406, Yes), the text meaning setting module 124 excludes it based on the operation input to the operation input unit 112.

  At this time, the text meaning setting module 124 notifies the meaning information storage module 114 of information that the text meaning of the text information deleted from the list is not “person name” (S407).

  For example, if there is a candidate whose text meaning is not “person name” from the list displayed in the list box 302 on the GUI screen (S408, Yes), the candidate item is displayed in the list box 302. And the command button 305 as a delete button is pressed. As a result, the selected item is deleted from the list.

  Next, from the list of text information whose text meaning is “person name” arranged in the order desired by the user, the operation input unit 112 determines the ranking of candidates that are judged not to be in the order of “person name” desired by the user. Based on the operation input to, the text semantic attribute setting module 127 performs a correction operation (S409).

  At this time, the text semantic attribute storage module 125 is notified of information related to the change in the ranking of the “person name” desired by the user.

  For example, when an arbitrary item is selected from the “person name” list displayed in the list box 302 on the GUI screen and the command button 303 is pressed, the order is switched to the item one higher than the selected item, When the command button 304 is pressed, the order is switched to an item one level lower than the selected item.

  Further, the command box 305 and the list box 302 edited by the command buttons 303 and 304 can be returned to the initially displayed state by pressing the command button 306.

  Next, the transmission destination of the document is determined based on the list arranged in the order of “person names” desired by the user (S410). The transmission destination is defined in advance, and the transmission destination is determined according to the contents of the list.

  Finally, the input document is transmitted to the transmission destination determined in S410 (S411). For example, when a transmission button of the command button 307 is pressed, document transmission is started.

  In the GUI screen shown in FIG. 17, the text information of the character area extracted as “person name” is displayed in the text box 302, but the recognition result of the character recognition module 113 is incorrect and the displayed text information is displayed. May become wrong. In such a case, recognition errors in the character recognition module 113 can be corrected by the user directly entering text in the text box 302 using the operation input unit 112.

  In the present embodiment, an example is shown in which text information whose text meaning is “person name” is used to select a transmission destination of document data. However, the present invention is not limited to this. For example, it is possible to use text information having various text meanings such as text information whose text meaning is “date”.

  In addition, the method of using the acquired text meaning information is not limited to the selection of the transmission destination, for example, the user is prompted to correct the acquired text meaning, and the corrected text meaning information is used as the text meaning. It can also be used to search for documents that have been extracted.

  As described above, the text meaning attribute presentation module 126 according to the present exemplary embodiment uses the character whose attribute has been determined by the text meaning analysis module 115 as an input in a predetermined process in which the character having the attribute determined for the character is to be input. Display as a character candidate. By doing in this way, the user selects a character input in a process in which a character having a certain attribute (for example, a person name, a telephone number, an address, etc.) is input as a character having the attribute from the input character candidates. The burden of character input can be greatly reduced.

(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described.

  The present embodiment is a modification of the above-described fifth embodiment. Hereinafter, parts having the same functions as those already described in the fifth embodiment are denoted by the same reference numerals, and description thereof is omitted. The image processing system S6 according to the present embodiment differs from the above-described fifth embodiment in a method for correcting an attribute added to text information extracted from image data. FIG. 18 is a functional block diagram showing a configuration of an image processing system S6 according to the sixth embodiment of the present invention.

  Specifically, the image processing system S6 according to this embodiment includes a text meaning setting tendency analysis module (corresponding to a history information acquisition unit and an identification information acquisition unit) 128 in addition to the image processing system S5 according to the fifth embodiment. A text meaning document tendency comparison module (corresponding to an image type determination unit) 130, a text meaning setting tendency storage module 129, and a text meaning fixed setting module (corresponding to a meaning setting unit, a history information acquisition unit, and an identification information acquisition unit) 131 are provided. It becomes the composition which becomes.

  The text meaning setting tendency analysis module 128 has a tendency to the user's correction contents set via the text meaning setting module 124 and the text meaning attribute setting module 127 with respect to the document information stored in the meaning information storage module 114. Analyze if there is.

  The text meaning setting tendency storage module 129 stores the contents of the user's correction operation performed on a certain document analyzed by the text meaning setting tendency analysis module 128 and information for identifying the document as a set. To do.

  The text meaning document tendency comparison module 130 compares whether the document of the image data to be processed is similar to the document layout stored in the text meaning setting tendency storage module 129.

  When the text meaning document setting comparison module 130 stores a document similar to the document to be processed in the text meaning setting trend storage module 129, the text meaning setting trend storage module 129 stores the text meaning setting tendency storage module 129. Thus, information on the past correction operation performed by the user on the document having the tendency is acquired, and information on the processing target document stored in the semantic information storage module 114 or the text semantic attribute storage module 125 is acquired. To reflect.

  In this embodiment, the tendency of the user's correction operation is analyzed for the text semantic analysis result and the text semantic attribute analysis result for the input document to be processed. If the layout of a newly input document is similar to a document that has been processed in the past, a correction operation performed by the user on the previously input document image is newly input. After the document is automatically applied to the document, the result of the automatic correction is presented to the user.

  For example, in the image processing system S6 according to the present embodiment, when a document image including an object arranged in the layout illustrated in FIG. 2 is acquired, text information of two character areas whose text meaning is “person name” is obtained. The list is displayed on the display unit 111 in the order desired by the user. Here, from the result of the list displayed on the display unit 111, the user determines that the second item among the items listed as “person name” is not “person name” in the text meaning, It is assumed that an operation for deleting from the list has been performed. At this time, the input document image 1 information and the user's correction operation are recorded as history information. Next, when image data of a document to be processed is newly input to the image processing system S6, if the image data of the document to be processed in the past is similar to the layout or the like, the processing is performed in the past. The user's correction operation performed on the document image is automatically reflected in advance on the newly input document image data. That is, in this case, the second item of the items in the “person name” list obtained as a result of the analysis of the image data of the newly input document is automatically deleted. By presenting the “person name” list from which the second item has been deleted in this way to the user, the user can newly perform a correction operation performed on image data of a similar document in the past. Then it can be omitted.

  FIG. 19 is a flowchart showing details of processing in the image processing system S6 according to the sixth embodiment of the present invention.

  First, a process for obtaining a tendency of a correction operation performed by the user for a document input to the image processing system S6 will be described.

  First, the image data acquisition unit 101 acquires image data of a document input to the document processing system 23 (S501).

  Next, the document area analysis module 102 analyzes the character area in the document image and acquires information on the coordinates (position, size, etc.) of the rectangle surrounding the character area. Further, the character recognition module 113 recognizes the text information of the character area extracted by the document area analysis module 102. Then, the text meaning analysis module 115 recognizes the text meaning of the text information recognized by the character recognition module 113. In this way, various types of information acquired by the document area analysis module 102, the character recognition module 113, and the text semantic analysis module 115 are stored in the semantic information storage module 114 (S502).

  Next, the user's desired display order in the text meaning list display analyzed by the text meaning analysis module 115 is stored in the text meaning attribute storage module 125 as an attribute (S503).

  Next, the information stored in the semantic information storage module 114 and the text semantic attribute storage module 125 is displayed on the display unit 111 by the text semantic attribute presentation module 126 and presented to the user (S504).

  Next, as a result of confirming the analysis result presented on the display unit 111 by the user, when the text meaning setting module 124 or the text meaning attribute setting module 127 is used to correct the text meaning or the text meaning attribute (S505). Yes), the process of S506 is executed. Specifically, the text semantic setting module or text semantic attribute setting is performed by the information related to the image data of the input document (that is, information stored in the semantic information storage module 114) and the text semantic setting trend analysis module 128. The text meaning setting trend storage module 129 stores the information in association with the information indicating the content of the user's correction operation performed on the document via the module 127 (S506).

  For example, as input document information, a user has performed in the past on information indicating the positions (coordinates, etc.) of all character regions analyzed by the document region analysis module 102 and a document having a layout similar to the document. The information regarding the contents of the correction work is stored in the text meaning setting trend storage module 129 in association with each other.

  In FIG. 20, the contents of the correction operation performed by the user in the past on the coordinate information in the document of the object such as the character area included in the image of the document image data and the attribute information determined on the character area. And shows a situation where the text meaning setting trend storage module 129 is stored.

  Next, the layout of the document newly input as the processing target in the image processing system S6 according to the present embodiment is similar to the layout of the document stored in the text meaning setting trend storage module 129 (input in the past). If it is, the same correction as the user's correction operation performed on the past input document is automatically applied to the newly input document and presented to the user. Processing to be performed will be described. FIG. 21 is a flowchart for explaining the details of the reflection processing of the past correction operation to the newly input document in the sixth embodiment of the present invention.

  First, the image data acquisition unit 101 acquires image data of a document input to the document processing system S6 (S601).

  Next, the document area analysis module 102 analyzes an object such as a character area in the image of the document image data, and acquires information on the coordinates (position, size, etc.) of the rectangle surrounding each character area. In addition, the character recognition module 113 recognizes text information from the character area extracted from the image by the document area analysis module 102. Then, the text meaning analysis module 115 recognizes the text meaning of the text information recognized by the character recognition module 113. In this way, various types of information acquired by the document area analysis module 102, the character recognition module 113, and the text semantic analysis module 115 are stored in the semantic information storage module 114 (S602).

  Next, the user's desired display order in the list display of the text meaning analyzed by the text meaning analysis module 115 is stored in the text meaning attribute storage module 125 as an attribute (S603).

  Next, whether or not information related to a document having a layout similar to a document newly input as a processing target is stored in the text meaning setting trend storage module 129 is compared with the text meaning document trend comparison module 128. This is performed (S604).

  As for comparison of documents in the text meaning document tendency comparison module 128, as shown in FIG. 20, for example, information on the coordinate values of all character areas extracted from documents input in the past to the text meaning setting tendency storage module 129 is obtained. If the difference is small when compared with the coordinate value of the character area extracted from the image data of the newly input document acquired by the document area analysis module 102, the two documents are stored. Are judged to be similar.

  Here, the configuration in which the similarity of the document is determined using only the coordinates of the character area included in the document image (information regarding the layout of the object in the document) is illustrated, but the present invention is not limited to this. For example, using text information of a character area included in a document image, if there are many words included in common between two documents to be compared, the semantic information is determined to be similar. Documents can also be compared using various information stored in the storage module 114 and the area meaning attribute storage module 125.

  Next, when information regarding a document having a layout similar to the document input to the image processing system S6 is stored in the text meaning setting trend storage module 129 (S604, Yes), image data of the newly input document On the other hand, the text semantic fixed form is subjected to correction processing having the same content as the correction operation previously performed by the user on the document similar to the newly input document stored in the text meaning setting trend storage module 129. Automatically performed by the setting module 131.

  Specifically, in S604, the text meaning document tendency comparison module 130 stores the information newly stored in the text meaning setting tendency storage module 129 as shown in FIG. 20 in the image processing system S6. If it is determined that the document A is similar to the existing document A (S604, Yes), the text semantic fixed setting module 131 corrects the document A stored in the text semantic setting trend storage module 129 by the user. A correction process having the same contents as the operation is performed on the image data of the newly input document. In other words, here, the text information that has been analyzed that the text meaning is “person name” and the order desired by the user is the second text information is corrected so that the text meaning is not “person name”. An instruction is issued to the meaning setting module 124.

  The text meaning attribute presentation module 126 corrects the information in the text meaning setting tendency storage module 129 in step S605 if the text meaning setting tendency storage module 129 has information on a document whose layout is similar to the document newly input to the image processing system S6. The display unit 111 presents the result of performing (S605), and if there is no information regarding a similar document (S604, No), the display unit 111 presents the analysis result in S602 and S603 ( S606).

  Next, when the user corrects the content of the text meaning or the text meaning attribute presented on the display unit 111 (S607, Yes), the text meaning setting trend analysis module 128 determines the content of determination of the meaning and attribute. The corrected new input document and the user's correction operation are associated with each other and stored in the text meaning setting trend storage module 129 (S608), and the process ends. On the other hand, if the user has not corrected the content presented on the display unit 111 (S607, No), the process is terminated.

  In each of the above-described embodiments, an example in which an object extracted from an image of document image data is mainly a character area has been described. However, the present invention is not limited to this, and can be included in a normal document. Needless to say, objects such as figures and photographic images can also be extracted.

  In each of the above-described embodiments, the configuration in which notification is given to the user by displaying the object and the semantic determination result about the object on the screen is given as an example. However, the present invention is not limited to this. For example, You may make it perform notification of the content similar to the above by an audio | voice.

  In the above-described embodiment, the scanned image data is subjected to semantic determination processing based on a predetermined rule, and the processing result of the determination processing is selected by the candidate selection unit or the user or document by the semantic setting unit. The configuration for performing the semantic setting according to the type is illustrated, but the configuration is not limited to this. For example, the semantic determination is performed based on a rule determined in advance according to the user or the document type with respect to the scanned image data. Processing may be performed.

  By adopting such a processing algorithm, it is possible to reduce processing steps for selecting an appropriate meaning for an object in the above-described embodiment.

  Each step in the processing in the above-described image processing system is realized by causing the CPU 801 to execute an image processing program stored in the memory 802.

  In this embodiment, the function for implementing the invention is recorded in advance in the apparatus. However, the present invention is not limited to this, and the same function may be downloaded from the network to the apparatus, and the same function is recorded. What is stored in the medium may be installed in the apparatus. The recording medium may be any form as long as the recording medium can store the program and can be read by the apparatus, such as a CD-ROM. Further, the function obtained by installing or downloading in advance may be realized in cooperation with an OS (operating system) or the like inside the apparatus.

  Although the present invention has been described in detail according to particular embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.

It is a functional block diagram for demonstrating image processing system S1 by the 1st Embodiment of this invention. It is a figure which shows an example of the document used as a process target. It is a figure for demonstrating the flow of the process in the 1st Embodiment of this invention. It is a figure which shows the relationship between each of the various information stored integrally by the semantic information storage module 114. FIG. It is a functional block diagram for demonstrating image processing system S2 by the 2nd Embodiment of this invention. It is a flowchart for demonstrating the flow of the process in the 2nd Embodiment of this invention. It is a figure which shows an example of the data table stored in the area | region meaning attribute storage module. 6 is a diagram illustrating an example of a GUI screen displayed on the display unit 111. FIG. It is a functional block diagram for demonstrating image processing system S3 by the 3rd Embodiment of this invention. It is a flowchart for demonstrating the process which acquires the tendency of correction operation which a user performs with respect to the image data of the input document. It is a figure which shows the condition where the coordinate of each character area of a document, and the content of the correction operation which the user performed in the past with respect to the document image similar to the said document are matched and stored. It is a flowchart for demonstrating the detail of the process in image processing system S3 by this Embodiment. It is a functional block diagram which shows the structure of image processing system S4 by the 4th Embodiment of this invention. It is a functional block diagram which shows the structure of image processing system S5 by the 5th Embodiment of this invention. It is a flowchart for demonstrating the detail of a process of image processing system S5 by the 5th Embodiment of this invention. It is an example of the data table which shows the text meaning attribute stored in the text meaning attribute storage module 125. It is a figure which shows the example of a display of GUI displayed on the display part 111 by the text meaning attribute presentation module 126. FIG. It is a functional block diagram which shows the structure of image processing system S6 by the 6th Embodiment of this invention. It is a flowchart which shows the detail of the process in image processing system S6 by the 6th Embodiment of this invention. The figure which shows the situation where the coordinate information in the document of objects, such as a character area, and the content of the correction operation which the user performed in the past with respect to the attribute information etc. which were determined about these character areas are made into a set, and are stored. is there. It is a flowchart for demonstrating the detail of the reflection process of the past correction operation to the document newly input in the 6th Embodiment of this invention.

Explanation of symbols

S1 to S6 Image processing system, 101 Image data acquisition unit, 102 Document area analysis module, 104 Area meaning analysis module, 108 Area meaning presentation module, 110 Area meaning setting module, 111 Display section, 112 Operation input section, 113 Character recognition module , 114 semantic information storage module, 115 text semantic analysis module, 116 area semantic attribute storage module, 117 area semantic attribute presentation module, 118 area semantic attribute setting module, 119 area semantic setting trend analysis module, 120 area semantic document trend comparison module, 121 area meaning setting tendency storage module, 122 area meaning fixed form setting module, 123 text meaning presentation module, 124 text meaning setting module, 125 text Semantic attribute storage module, 126 text semantic attribute presentation module, 127 text semantic attribute setting module, 128 text semantic setting trend analysis module, 130 text semantic document trend comparison module, 129 text semantic setting trend storage module, 131 text semantic fixed setting module , 801 CPU, 802 memory.

Claims (14)

An image data acquisition unit for acquiring image data;
An object extraction unit that extracts an object included in an image of the image data acquired by the image data acquisition unit;
A semantic determination unit that determines the meaning of the object extracted by the object extraction unit in the layout of the image data based on a predetermined determination rule;
A notification unit configured to display a list of information related to a plurality of objects determined to have the meaning in association with the information related to the meaning when the meaning determination unit determines that the plurality of objects have the same meaning; ,
An operation input unit for receiving user operation inputs;
A meaning setting unit that sets the object selected by the user's operation input received by the operation input unit as having the meaning determined by the meaning determination unit for the object;
A history information acquisition unit that acquires an object selection history by the operation input unit ,
The meaning setting unit displays a list by the notification unit based on history information acquired by the history information acquisition unit when the meaning determination unit determines that a plurality of objects have the same meaning. An image processing system that sets an object having the highest frequency of selection in the past as having the meaning determined by the meaning determination unit for the object . An image data acquisition unit for acquiring image data;
An object extraction unit that extracts an object included in an image of the image data acquired by the image data acquisition unit;
A semantic determination unit that determines the meaning of the object extracted by the object extraction unit in the layout of the image data based on a predetermined determination rule;
A reliability determination unit that determines the rank of each object determined to have the meaning when the meaning determination unit determines that the plurality of objects have the same meaning;
When the meaning determination unit determines that the plurality of objects have the same meaning, the reliability determination unit associates information about the plurality of objects determined to have the meaning with the meaning. A notification unit for displaying a list in descending order of determination;
An operation input unit for receiving user operation inputs;
The object selected by the user's operation input received by the operation input unit is set as having the meaning determined by the meaning determination unit for the object, and the object received by the operation input unit A semantic setting unit for changing the order of objects by operation input;
A history information acquisition unit for acquiring an object selection history and an object ranking history by the operation input unit,
The notification unit reflects the object selection history and the object ranking history acquired by the history information acquisition unit when the meaning determination unit determines that the plurality of objects have the same meaning. An image processing system that displays objects as a list. An image data acquisition unit for acquiring image data;
An object extraction unit that extracts an object included in an image of the image data acquired by the image data acquisition unit;
A semantic determination unit that determines the meaning of the object extracted by the object extraction unit in the layout of the image data based on a predetermined determination rule;
When the plurality of objects are determined to have the same meaning by the meaning determination unit, the rank of each object determined to have the meaning is determined based on the position of each object in the layout of the image data. A reliability determination unit for determining;
When the meaning determination unit determines that the plurality of objects have the same meaning, the reliability determination unit associates information on the plurality of objects determined to have the meaning with the information on the meaning. A notification unit for displaying a list in descending order of determination in
An image processing system comprising: An image data acquisition unit for acquiring image data;
An extraction unit that extracts text information included in an image of the image data acquired by the image data acquisition unit;
Based on a predetermined determination rule, a meaning determination unit that determines whether there is text information of a person name in the text information extracted by the extraction unit;
  A reliability determination unit that determines, based on the type of additional characters, whether or not there is a predetermined additional character in the vicinity of the text information, and the ranking of the text information determined by the meaning determining unit as a person name;
When there are a plurality of text information determined as the text information of the name of the person in the meaning determination section, the notification section that displays the list of the text information of the name of the person in descending order of the order determined by the reliability determination section;
An image processing system comprising: The image processing system according to claim 1,
A reliability determination unit that determines the reliability of the meaning determination process in the meaning determination unit;
When the meaning determining unit determines that the plurality of objects have the same meaning, the notification unit associates the plurality of objects determined to have the meaning with the meaning, and determines the reliability. An image processing system that causes a list to be displayed in descending order of reliability. The image processing system according to claim 3 .
A meaning setting unit that sets the object selected by the user's operation input received by the operation input unit as having the meaning determined by the meaning determination unit for the object;
Includes a history information acquisition unit that acquires the selection history of the object by the operation input unit,
The meaning setting unit displays a list by the notification unit based on history information acquired by the history information acquisition unit when the meaning determination unit determines that a plurality of objects have the same meaning. An image processing system that sets an object having the highest frequency of selection in the past as having the meaning determined by the meaning determination unit for the object. The image processing system according to claim 1, wherein:
An identification information acquisition unit for acquiring identification information for identifying a user who selects an object by the operation input unit ;
The history information acquisition unit is an image processing system that acquires information related to an object selection history received by the operation input unit by a user corresponding to the identification information acquired by the identification information acquisition unit. In the image processing system according to any one of claims 1 to 3 ,
A character attribute determination unit that determines an attribute of the character when the object extracted by the object extraction unit is a character;
An input candidate display unit that displays a character whose attribute is determined by the character attribute determination unit as an input character candidate in a predetermined process in which the character having the attribute determined for the character is to be input. system. Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When a plurality of objects are determined to have the same meaning, information related to the plurality of objects determined to have the meaning is displayed in a list in association with the information related to the meaning .
Accept user input,
The object selected by the received user operation input is set as having the determined meaning for the object,
Get selection history of accepted objects ,
When it is determined that a plurality of objects have the same meaning, an object with the highest frequency selected in the past among the objects displayed in the list is selected for the object based on the history information of the acquired object. An image processing method set as having a determined meaning . Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, the order of each object determined to have the meaning is determined,
When it is determined that a plurality of objects have the same meaning, information on the plurality of objects determined to have the meaning is associated with the meaning and displayed in a list in descending order of the determined order.
Accept user input,
The object selected by the received user operation input is set as having the determined meaning for the object, and the order of the objects is changed by the received user operation input,
Get object selection history and object ranking history,
An image processing method for displaying an object in a list by reflecting an acquired object selection history and an object ranking history when a plurality of objects are determined to have the same meaning. Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, the order of each object determined to have the meaning is determined based on the position of each object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, information relating to the plurality of objects determined to have the meaning is associated with the information related to the meaning and displayed as a list in descending order of the determined order. Processing method. Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When a plurality of objects are determined to have the same meaning, information related to the plurality of objects determined to have the meaning is displayed in a list in association with the information related to the meaning .
The object selected by the user's operation input is set as having the determined meaning for the object,
Get selection history of objects selected by user operation input ,
When it is determined that a plurality of objects have the same meaning, an object with the highest frequency selected in the past among the objects displayed in the list is selected for the object based on the history information of the acquired object. An image processing program for causing a computer to execute a process of setting as having a determined meaning . Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, the order of each object determined to have the meaning is determined,
When it is determined that a plurality of objects have the same meaning, information on the plurality of objects determined to have the meaning is associated with the meaning and displayed in a list in descending order of the determined order.
The object selected by the user's operation input is set as having the determined meaning for the object, and the order of the objects is changed by the received user operation input,
Get object selection history and object ranking history,
When a plurality of objects are determined to have the same meaning, the object selection list and the object order history are reflected to display a list of objects.
An image processing program that causes a computer to execute processing. Get image data,
Extracting an object contained in the image of the acquired image data;
Based on a predetermined determination rule, determine the meaning of the extracted object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, the order of each object determined to have the meaning is determined based on the position of each object in the layout of the image data,
When it is determined that a plurality of objects have the same meaning, information on the plurality of objects determined to have the meaning is associated with the information on the meaning and displayed in a list in descending order of the determined order.
An image processing program that causes a computer to execute processing.

Images