JP7707705B2 - Character recognition system and character recognition method - Google Patents

Description

The present invention relates to a character recognition system and a character recognition method.

Conventionally, in character recognition systems, character images contained in an original image are converted into character data and stored, allowing for character-based search and classification, improving the storability and reusability of the original. For this reason, various character recognition technologies have been proposed (see, for example, Patent Document 1).

Patent Document 1 discloses an information processing device that performs optical character recognition (OCR) processing on image data and converts it into document data. In the information processing device disclosed in Patent Document 1, a character string with a special shape, such as a company logo, is pre-registered as a specific pattern. The information processing device then matches the pre-registered specific pattern with an image containing the specific pattern, thereby eliminating false detection of character strings with a special shape, such as a company logo, and improving the accuracy of OCR analysis results.

JP 2019-149073 A

However, when the manuscript is, for example, a memo, images of special symbols and special characters handwritten on the memo may be included in the manuscript image. Such handwritten special symbols and special characters are usually not registered in a character recognition system, so they cannot be recognized by the character recognition system or are recognized as different characters. In this case, problems occur in that unrecognizable characters such as handwritten special symbols and special characters cannot be searched or classified by character, and they cannot be used as digital data. In addition, since the writing style and shape of handwritten special symbols and special characters vary depending on the writer, it is difficult to register these handwritten symbols and characters in a character recognition system in advance. Note that the technology disclosed in the above-mentioned Patent Document 1 is a character recognition technology that matches a character string of a special shape such as a company logo registered in advance with a specific pattern and an image containing the specific pattern, so it cannot solve the above problem. Therefore, there has been a demand for the development of a character recognition system that enables the extraction and digitization of handwritten special symbols and special characters even if the manuscript image contains handwritten special symbols and special characters.

The present invention has been made to solve the above problems. The object of the present invention is to provide a character recognition system and a character recognition method that can extract and digitize handwritten special symbols and characters even if they are included in an original image, and can efficiently search, classify, and otherwise process such symbols and characters.

The character recognition system of the present invention comprises an image reading unit that reads an image of a document, a character recognition unit that extracts characters contained in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a memory unit that stores the results of character recognition on the image of the document, wherein the character recognition unit extracts unrecognized characters that cannot be recognized by character recognition, links similar characters to the unrecognized characters, compares the similar characters linked to each unrecognized character among the multiple unrecognized characters, groups at least a portion of the multiple unrecognized characters based on the comparison results, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters in the memory unit.
In addition, the character recognition system of the present invention includes an image reading unit that reads an image of a document, a character recognition unit that extracts characters contained in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a memory unit that stores the results of the character recognition on the image of the document, and when there are multiple images of the document read by the image reading unit, the character recognition unit performs character recognition on each image of the document, extracts unrecognized characters that cannot be recognized by character recognition, groups the unrecognized characters extracted in a specified image of the document among the multiple images of the document and the unrecognized characters extracted in an image of a document other than the specified image of the document among the multiple images of the document, assigns the same specified character code to each of the grouped unrecognized characters, and outputs and stores the data in which the same specified character code is assigned to each of the grouped unrecognized characters in the memory unit.

In addition, the character recognition method of the present invention is a character recognition method in a character recognition system that includes an image reading unit that reads an image of a document, a character recognition unit that extracts characters included in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a memory unit that stores the results of the character recognition on the image of the document, in which the character recognition unit extracts unrecognized characters that cannot be recognized by character recognition, links similar characters to the unrecognized characters, compares the similar characters linked to each unrecognized character among the multiple unrecognized characters, groups at least a portion of the multiple unrecognized characters based on the comparison results, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters in the memory unit.
In addition, the character recognition method of the present invention is a character recognition method in a character recognition system that includes an image reading unit that reads an image of a document, a character recognition unit that extracts characters contained in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a memory unit that stores the results of the character recognition for the image of the document, in which when there are multiple images of the document read by the image reading unit, the character recognition unit performs character recognition on each image of the document, extracts unrecognized characters that cannot be recognized by character recognition, groups the unrecognized characters extracted in a specified image of the document among the multiple images of the document and the unrecognized characters extracted in an image of a document other than the specified image of the document among the multiple images of the document, assigns the same specified character code to each of the grouped unrecognized characters, and outputs and stores the data in which the same specified character code is assigned to each of the grouped unrecognized characters to the memory unit.

According to the present invention configured as above, even if the document image contains special handwritten symbols or characters, it is possible to extract and digitize these symbols and characters, and to efficiently search, classify, etc. these symbols and characters.

FIG. 1 is a diagram showing an example of an image of a document including handwritten special symbols. 1 is a block diagram showing a configuration of a character recognition system according to an embodiment of the present invention. 1 is a diagram showing an example of an original image read by a character recognition system according to an embodiment of the present invention; FIG. 2 is a diagram showing an example of the layout of a document image analyzed in a character recognition system according to an embodiment of the present invention. 4A to 4C are diagrams for explaining a character string segmentation process performed in a character recognition system according to an embodiment of the present invention. 4A to 4C are diagrams for explaining a character segmentation process performed in a character recognition system according to an embodiment of the present invention. 4A and 4B are diagrams for explaining character normalization processing performed in a character recognition system according to an embodiment of the present invention. 1 is a diagram showing a relationship between an unrecognized character, a similar character, and the degree of match between the two in a character recognition system according to an embodiment of the present invention; 3 is a diagram showing an example of a recognition result (text data) of a character recognition process output in the character recognition system according to the embodiment of the present invention. FIG. 1 is a diagram showing an example (output example 1) of output data showing the correspondence between special symbols and character codes assigned thereto, outputted in a character recognition system according to one embodiment of the present invention; FIG. A figure showing an example (output example 3) of another output data showing the correspondence between special symbols and character codes assigned thereto, output in a character recognition system according to one embodiment of the present invention. FIG. 11 is a diagram for explaining an example (output example 3) of another output data showing the correspondence between special symbols and character codes assigned thereto, output in a character recognition system according to one embodiment of the present invention. 4 is a flowchart showing the procedure of character recognition processing for one document image, which is performed in the character recognition system according to the embodiment of the present invention. 4 is a flowchart showing a procedure of character recognition processing for a plurality of document images, which is performed in the character recognition system according to the embodiment of the present invention.

The configuration of a character recognition system and a character recognition method according to one embodiment of the present invention will be specifically described below with reference to the drawings. Note that the present invention is not limited to the following example.

[An example of a document image containing special symbols that can be subject to character recognition]
First, before describing the configuration of the character recognition system and the character recognition method of the present embodiment, an example of an image of a document including handwritten special symbols to be recognized in the character recognition system of the present embodiment will be described. Fig. 1 is a diagram showing an example of an image of a document (memo) including handwritten special symbols.

In the example shown in FIG. 1, one original image N includes images of four memos N1 to N4. Also, in the example shown in FIG. 1, text information TX1 to TX4 indicating the contents of the memo is shown in the center of each image of memos N1 to N4, and handwritten special symbols SC1 to SC4 are shown above the text information TX1 to TX4. The special symbols SC1 to SC3 shown in each of memos N1 to N3 are special symbols that enclose the handwritten character "重" in a circle, and the special symbol SC4 shown in memo N4 is a special symbol that resembles the shape of a moon.

Handwritten special symbols such as special symbols SC1 to SC4 are symbols written to help organize the memory of the person who created the note or for later classification, and are usually difficult to register in a character recognition system in advance. However, the character recognition system of this embodiment extracts (identifies) special symbols SC1 to SC4 as unrecognized characters in the character recognition process, and assigns (links) a specific character code to the unrecognized characters, thereby converting the unrecognized characters into digital data.

For example, the special symbols SC1 to SC3 have different symbol shapes, but the meanings of the symbols are the same. Therefore, if multiple special symbols with the same meaning are recognized as different special symbols (character codes), these multiple special symbols cannot be managed together. The character recognition system of this embodiment assigns the same character code to multiple special symbols with the same meaning and manages them.

[Character Recognition System Configuration]
Fig. 2 is a block diagram showing the configuration of the character recognition system of this embodiment. As shown in Fig. 2, the character recognition system 1 includes an image forming device 10 and an information processing device 20. The image forming device 10 and the information processing device 20 are capable of transmitting and receiving information data to and from each other via a network 30. Note that Fig. 2 shows an example in which the character recognition system 1 includes one image forming device 10, but the present invention is not limited to this, and the character recognition system 1 may include a plurality of image forming devices.

When character recognition is performed by the character recognition system 1, the image forming device 10 first reads an image of the original document that is to be subjected to character recognition, and generates image data of the original document. Next, the image forming device 10 transmits the generated image data of the original document (hereinafter referred to as the "original image") to the information processing device 20 via the network 30. Next, the information processing device 20 performs character recognition processing on the original document image received via the network 30, and outputs the results of the character recognition processing.

In this embodiment, an example is described in which the character recognition processing functional unit (character recognition device unit 22 described below) is provided in the information processing device 20, but the present invention is not limited to this. For example, the character recognition processing functional unit may be provided inside the image forming device 10. In this case, the character recognition system 1 is composed of only the image forming device 10.

[Configuration of Image Forming Apparatus]
The image forming apparatus 10 is a multi-function peripheral (MFP) equipped with multiple functions such as a scanner function, a copy function, a facsimile function, a network function, a data box function, etc. The image forming apparatus 10 includes an operation display unit 11, an auto document feeder (ADF) 12, an image reading unit 13, a printer unit 14, a central processing unit (CPU) 15, a read only memory (ROM) 16, a random access memory (RAM) 17, a storage unit 18, a communication unit 19, and a bus 101. The bus 101 electrically connects the components together and is a signal path through which signals are input and output between the components.

The operation display unit 11 is composed of a display unit consisting of a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-luminescence) display, and an operation unit consisting of a touch sensor. The display unit and operation unit are integrally formed, for example, as a touch panel. The operation display unit 11 generates an operation signal representing the operation content input by the operator to the operation unit, and supplies the operation signal to the CPU 15. The operation display unit 11 also displays the operation content by the operator, setting information, etc. on the display unit based on the display signal supplied from the CPU 15. It is also possible to configure the operation unit as a mouse, tablet, etc., and to configure it separately from the display unit.

The automatic document feeder 12 is composed of a loading tray on which the recording medium is placed, a mechanism for transporting the recording medium, transport rollers, etc., and transports the recording medium to a specified transport path.

The image reading unit 13 optically reads the image of the document fed by the automatic document feeder 12, and performs A/D (Analog to Digital) conversion of the read image to generate image data (document image). The document image read by the image reading unit 13 is transmitted to the information processing device 20 via the communication unit 19 and the network 30. The image reading unit 13 can also read an image from the document on the platen glass.

The printer unit 14 is composed of components necessary for image formation, and prints a specified image on a recording medium based on the specified information of a print job. Specifically, the printer unit 14 forms an electrostatic latent image on the recording medium by irradiating a photosensitive drum charged by a charging device with light corresponding to the image from an exposure device, and develops the image by attaching charged toner in a developing device. The printer unit 14 then performs a process of primarily transferring the developed toner image to a transfer belt, secondary transferring the toner image from the transfer belt to the recording medium, and further fixing the toner image on the recording medium with a fixing device.

The CPU 15 controls the operation of each section within the image forming device 10. For example, the CPU 15 controls the document image reading process of the image reading section 13, the image forming process of the printer section 14 based on a print instruction from the user via the operation display section 11, and the transmission and reception process of data information between the information processing device 20 and the communication section 19.

The ROM 16 is composed of a storage medium such as a non-volatile memory, and stores programs and data that are executed and referenced by the CPU 15.

RAM 17 is composed of a storage medium such as a volatile memory, and temporarily stores information (data) required for each process performed by CPU 15.

The storage unit 18 is composed of a computer-readable non-transient recording medium that stores the program executed by the CPU 15, and is composed of a storage device such as an HDD (Hard Disk Drive). The storage unit 18 stores programs and data, such as the program for the CPU 15 to control each part, the OS (Operating System), and the controller. Note that some of the programs and data stored in the storage unit 18 may be stored in the ROM 16. In addition, the computer-readable non-transient recording medium that stores the program executed by the CPU 15 is not limited to an HDD, and may be, for example, a recording medium such as an SSD (Solid State Drive), a CD (Compact Disc)-ROM, or a DVD (Digital Versatile Disc)-ROM.

The communication unit 19 transmits and receives various data information to and from an external device connected via the network 30. The communication unit 19 is, for example, a communication interface having a communication IC (Integrated Circuit) and a communication connector, and is capable of transmitting and receiving various data information to and from the information processing device 20 connected via the network 30 using a predetermined communication protocol under the control of the CPU 15. The communication unit 19 also has, for example, an antenna, a demodulation circuit, a signal processing circuit, and the like, and is capable of transmitting and receiving various data information wirelessly to and from the information processing device 20 connected via the network 30 using a wireless communication method such as Wi-Fi (registered trademark).

[Configuration of information processing device]
The information processing device 20 receives an original image read by the image reading unit 13 of the image forming device 10 via the network 30, and performs character recognition processing on the received original image. As shown in Fig. 2, the information processing device 20 has an operation display unit 21, a character recognition device unit 22, a CPU 23, a ROM 24, a RAM 25, a storage unit 26, a communication unit 27, and a bus 201. The bus 201 is a signal path that electrically connects each component unit and inputs and outputs signals between the components.

The operation display unit 21 is composed of a display unit consisting of a display device such as an LCD or an organic EL display, and an operation unit consisting of a touch sensor. The display unit and the operation unit are integrally formed, for example, as a touch panel. The operation display unit 21 generates an operation signal representing the operation content input by the operator to the operation unit, and supplies the operation signal to the CPU 23. The operation display unit 21 also displays the operation content by the operator and setting information on the display unit based on the display signal supplied from the CPU 23. The display unit can also display the recognition results of the character recognition process (such as text information shown in FIG. 11) for the document image, which is output from the character recognition device unit 22 and will be described later. It is also possible to configure the operation unit as a mouse, tablet, or the like, and to configure it separately from the display unit.

The character recognition device 22 (character recognition unit) performs a predetermined character recognition process on the document image to be subjected to character recognition, received from the image forming device 10 via the network 30, under the control of the CPU 23. The character recognition process of this embodiment employs a processing method used in OCR, that is, a method in which the document image is subjected to layout analysis, line segmentation, character segmentation, character recognition, and result output in this order. However, the present invention is not limited to this, and any method can be employed for the character recognition process as long as it is a method capable of extracting various handwritten special symbols, etc., as described in FIG. 1, from the document image. Note that, in this embodiment, the function of the character recognition device 22 is configured by software, but the character recognition device 22 may be configured by a processing device (hardware) dedicated to character recognition, in which case the character recognition process is controlled within the processing device dedicated to character recognition. The configuration and function of the character recognition device 22 will be described in detail later.

The CPU 23 controls the operation of each unit in the information processing device 20. Therefore, the CPU 23 controls the operation of the character recognition processing performed by the character recognition device unit 22. Specifically, the CPU 23 controls the processing of analyzing the layout of the document image by the layout analysis unit 221 described below, the character segmentation processing by the character segmentation unit 222 described below, and the grouping and character code assignment processing by the character recognition processing unit 223 described below.

The ROM 24 is composed of a storage medium such as a non-volatile memory, and stores programs and data that are executed and referenced by the CPU 23.

The RAM 25 is composed of a storage medium such as a volatile memory, and temporarily stores information (data) necessary for each process performed by the CPU 23. The RAM 25 also temporarily stores, for example, unrecognized characters extracted by the character recognition processing unit 223 (see FIG. 8 described below).

The storage unit 26 is composed of a computer-readable non-transient recording medium storing the program executed by the CPU 23, and is composed of a storage device such as an HDD. The storage unit 26 stores programs and data for the CPU 23 to control each part, an OS (Operating System), a controller, etc. The storage unit 26 also stores various data generated by various processes related to character recognition performed by the character recognition device unit 22. Specifically, the storage unit 26 stores text data (see FIG. 11 described later) and output data related to special symbols (see FIGS. 9, 10, 12, and 13 described later) obtained as the recognition result of the character recognition process performed by the character recognition device unit 22. Note that some of the programs and data stored in the storage unit 26 may be stored in the ROM 24. The computer-readable non-transient recording medium storing the program executed by the CPU 23 is not limited to an HDD, and may be, for example, an SSD, a CD-ROM, a DVD-ROM, or other recording medium.

The communication unit 27 transmits and receives various data information to and from external devices connected via the network 30. The communication unit 27 is, for example, a communication interface having a communication IC and a communication connector, and is capable of transmitting and receiving various data information to and from the image forming device 10 connected via the network 30 using a predetermined communication protocol under the control of the CPU 23. The communication unit 27 also has components such as an antenna, a demodulation circuit, and a signal processing circuit, and is also capable of transmitting and receiving various data information wirelessly to and from the image forming device 10 connected via the network 30 using a wireless communication method such as Wi-Fi (registered trademark).

[Configuration and Functions of Character Recognition Device]
As shown in FIG. 2, the character recognition device unit 22 (character recognition unit) has a layout analysis unit 221, a character segmentation unit 222, and a character recognition processing unit 223.

(1) Layout Analysis Unit The layout analysis unit 221 analyzes the layout (arrangement) of areas such as image groups and character groups in the original image based on the original image input from the image forming apparatus 10. Specifically, the layout analysis unit 221 first separates each component in the original image, classifies each separated component according to the type of content (characters, drawings, tables, photographs, lines, etc.), and analyzes the layout of each area such as the character area and the figure area in the original image.

Figure 3 is a diagram showing an example of an original image 100 read by the image forming device 10 of the character recognition system 1. In the example shown in Figure 3, the original image 100 has, as its components, a graph drawing portion 110 arranged in the left half area of the original image 100, and a text portion 120 and a character portion 130 arranged in the upper right and lower right areas of the original image 100, respectively. Furthermore, the original image 100 also has, as its components, a ruled line 140 between the text portion 120 and the character portion 130. Note that the components of the original image 100 are not limited to these, and may include other components such as tables and photographs.

Figure 4 is a diagram showing an example of the results of the layout analysis of the original image 100 performed by the layout analysis unit 221. For ease of explanation, in Figure 4, only the layout areas of the drawing portion 110, text portion 120, character portion 130, and ruled lines 140 constituting the original image 100 are shown by dashed lines, and the contents of each component shown in Figure 3 are omitted. In Figure 4, the layout areas (drawing area 210, character area 220, and character area 230) of the drawing portion 110, text portion 120, and character portion 130 analyzed by the layout analysis unit 221 are each shown by a rectangular frame. Also, in Figure 4, the layout area (line area 240) of the ruled lines 140 analyzed by the layout analysis unit 221 is shown by a straight line.

Next, the layout analysis unit 221 extracts character areas from the layout areas (drawing area 210, character area 220, character area 230, and line area 240) of each component in the analyzed (classified) original image 100. In the example shown in FIG. 4, character area 220 and character area 230 are extracted. Then, data of the character areas (character area 220 and character area 230) extracted by the layout analysis unit 221 is output to the character extraction unit 222.

(2) Character Segmentation Unit The character segmentation unit 222 separates the character region into lines (in the case of horizontal writing) or columns (in the case of vertical writing) from the data of the character region in the original image 100 input from the layout analysis unit 221. In other words, the character segmentation unit 222 segments data of character strings of each line or column constituting the character region from the character region.

Figure 5 is a diagram showing an example of a character string extraction operation (decomposition operation) performed by the character extraction unit 222. The example shown in Figure 5 is the result when each character string constituting the character portion 130 is extracted from the contents of the character area 230 shown in Figure 4, i.e., the character portion 130 shown in Figure 3. The area surrounded by a dashed line in Figure 5 indicates the area of the character string of each line to be extracted. For example, in the third line from the right of the character portion 130, the character string 231 "Business partner company on a specified date" is extracted by the character extraction unit 222.

The character extraction unit 222 also breaks down the extracted character strings of each row or column into individual characters. That is, the character extraction unit 222 extracts data of each character constituting the character string from the extracted character strings of each row or column.

Figure 6 is a diagram showing an example of the method of extraction (decomposition) of each character performed by the character extraction unit 222 and the extraction results. Note that the example shown in Figure 6 shows the extraction process of each character performed on the character string 231 "business partner company on a fixed date" extracted from the character portion 130 shown in Figure 5.

In the process of extracting each character, the character extraction unit 222 first moves the scanning line L1 in the direction from the beginning to the end of the character string 231 to be processed (the direction of the arrow L2 in FIG. 6). At this time, the character extraction unit 222 counts the number of intersections (number of intersection dots) between the scanning line L1 and the constituent parts of each character. FIG. 6(a) is a diagram showing the change in the number of intersections with respect to the scanning direction of the scanning line L1. As shown in FIG. 6(a), in the section between adjacent characters, the number of intersections is "0" consecutively.

When the interval where the intersection number is "0" consecutively exceeds a predetermined threshold, the character extraction unit 222 determines that the interval where the intersection number is "0" consecutively is a character interval, breaks down the character string into each character interval, and extracts each character. As a result, as shown in FIG. 6(b), the character string 231 "te kette na hi tou tsu na hi tou tsu na hi" is broken down into 12 characters (area surrounded by dashed lines in the figure) "te", "kette", "ma", "tsu", "ta", "hi", "ni", "tori", "biki", "saki", "ki" and "gyo".

The character extraction unit 222 performs the above-mentioned process of extracting a character string of one row or one column and extracting each character from the character string for all character regions constituting the original image 100. Then, each character image data extracted by the character extraction unit 222 is output to the character recognition processing unit 223.

(3) Character Recognition Processing Unit The character recognition processing unit 223 first normalizes the size of each character image input from the character segmentation unit 222 to the size (predetermined size) of a character for matching registered in the character recognition system 1. Then, the character recognition processing unit 223 performs character recognition by matching each normalized character image with characters already registered in the character recognition system 1 (hereinafter referred to as "registered characters").

Figure 7 is a diagram for explaining the character normalization process performed in the character recognition processing unit 223. Character image T1 shown in Figure 7 is, for example, an image of the character "業" before normalization input from the character segmentation unit 222, and character image T2 is an image of the character "業" after normalization. Figure 7 shows an example in which character image T1 input from the character segmentation unit 222 is normalized to character image T2 with a height of "a" and a width of "a". Note that the size (predetermined size) of character image T2 after normalization is the size of the character registered in character recognition system 1, and therefore changes depending on, for example, the specifications of character recognition system 1.

After character normalization, the character recognition processing unit 223 compares the normalized character image with registered characters. If the result of the comparison shows that the degree of match between the normalized character image and a specific registered character exceeds a predetermined threshold, the character recognition processing unit 223 determines that character recognition is successful and converts the normalized character image into the specific registered character. In other words, the character recognition processing unit 223 converts the character image extracted from the original image 100 into digital data. Note that the method of comparing the normalized character image with a registered character, i.e., the method of calculating the degree of match between the two, can be, for example, a matching method used in conventional OCR.

On the other hand, if the character matching result shows that the normalized character image matches all registered characters at or below a predetermined threshold, or matches multiple registered characters at or above a predetermined threshold, the character recognition processing unit 223 determines the normalized character image as an unrecognized character. Note that in the character recognition system 1 of this embodiment, the handwritten special symbols SC1 to SC4 in the document image N shown in FIG. 1 do not normally match any registered characters that exceed a predetermined threshold, i.e., there are no registered characters that are recognized as identical, so in character recognition by the character recognition processing unit 223, the symbols are recognized as unrecognized characters and determined as such. The unrecognized characters are then temporarily stored in RAM 25.

After temporarily storing the unrecognized character in the RAM 25, the character recognition processing unit 223 searches for registered characters similar to the unrecognized character (hereinafter referred to as "similar characters") based on the degree of match obtained in the matching process between the unrecognized character and the registered character. The character recognition processing unit 223 then links a similar character having a degree of match exceeding the above-mentioned predetermined threshold to the unrecognized character, for example. At this time, only one similar character may be linked to the unrecognized character, or multiple similar characters may be linked to the unrecognized character. In the former case, only the similar character having the highest degree of match is linked to the unrecognized character. On the other hand, in the latter case, a specific number of similar characters may be linked to the unrecognized character in descending order of degree of match, or all similar characters exceeding a specific threshold may be linked to the unrecognized character. Furthermore, characters having a degree of match that does not exceed a specific threshold may be linked to the unrecognized character.

In addition, the character recognition processing unit 223 generates a data set (hereinafter referred to as a "similar character data set") that combines unrecognized characters, similar characters or similar character groups associated with the unrecognized characters, and the degree of match of each similar character. The generated similar character data set is stored in the storage unit 26. FIG. 8 is a diagram showing an example of a similar character data set generated for each of the handwritten special symbols SC1 to SC4 (unrecognized characters) in the document image N shown in FIG. 1. FIG. 8 shows an example of the configuration of a similar character data set when three similar characters are associated with each special symbol that is an unrecognized character. For example, in the similar character data set for the special symbol SC1, three similar characters, "重", "動" and "動", associated with the special symbol SC1, are associated with the degrees of match of the similar characters ("0.75", "0.61" and "0.55"). Note that similar character data sets are configured in a similar manner for the special symbols SC2 to SC4, so the configuration of the special symbols SC2 to SC4 will not be described here. In addition, the configuration of the similar character data set shown in FIG. 8 shows an example in which similar characters are arranged in order of the degree of match, but the order in which similar characters are arranged can be arbitrary.

When the character recognition processing unit 223 extracts an unrecognized character in the character recognition processing described above, it performs a process of assigning a specific character code to the extracted unrecognized character. Then, in the process of outputting the character recognition results described below, the character recognition processing unit 223 outputs a data set linking the extracted unrecognized character with the assigned character code to the storage unit 26. In other words, the data set linking the unrecognized character with the assigned character code is stored and registered in the storage unit 26. As a result, the unrecognized character extracted by character recognition is converted into digital data.

In the character code assignment process, the character recognition processing unit 223 first determines whether a character code corresponding to an unrecognized character (unrecognized character group) having a similar meaning to the currently extracted unrecognized character (unrecognized character group) has already been registered in the character recognition system 1. If a character code corresponding to an unrecognized character (unrecognized character group) having a similar meaning to the currently extracted unrecognized character (unrecognized character group) has already been registered in the character recognition system 1, the character recognition processing unit 223 assigns the specific registered character code to the currently extracted unrecognized character (unrecognized character group). In other words, the currently extracted unrecognized character (unrecognized character group) and previously registered unrecognized characters are grouped together.

On the other hand, if the group of unrecognized characters extracted this time contains one unrecognized character that does not correspond to a registered unrecognized character, the character recognition processing unit 223 assigns a new character code to the unrecognized character extracted this time. If the group of unrecognized characters extracted this time contains multiple unrecognized characters that do not correspond to a registered unrecognized character, the character recognition processing unit 223 performs a grouping process on the multiple unrecognized characters extracted this time. Then, if at least some of the unrecognized characters among the multiple unrecognized characters are grouped, the character recognition processing unit 223 assigns the same new character code to the grouped unrecognized characters. If there is an unrecognized character that is not grouped by this grouping process, the character recognition processing unit 223 assigns a different new character code to the unrecognized character.

In this embodiment, in the process of assigning character codes to unrecognized characters, as described above, the same character code is assigned to multiple unrecognized characters having the same meaning (content), and the multiple unrecognized characters are grouped to generate a group of unrecognized characters. Then, in the grouping process for the multiple unrecognized characters, similar characters associated with each unrecognized character are compared, and unrecognized characters that have overlapping similar characters are grouped together.

Therefore, for example, in the example shown in FIG. 8, the special symbols SC1 to SC4 included in the document image N are extracted as unrecognized characters, but in the process of assigning character codes to each unrecognized character, the special symbols SC1 to SC3, which overlap similar characters "重" and "動", are grouped as unrecognized characters having the same meaning, and a specific character code is assigned to the group of unrecognized characters. On the other hand, the three similar characters linked to the special symbol SC4 do not include any characters that overlap with the similar characters linked to the special symbols SC1 to SC3, so a new character code different from the specific character code assigned to the special symbols SC1 to SC3 is assigned to the special symbol SC4. When multiple unrecognized characters are grouped, a data set linking the group of unrecognized characters to the character codes assigned thereto is output to and stored in the memory unit 26 in the process of outputting the results of character recognition described later. Note that the method of grouping multiple unrecognized characters is arbitrary.

The character recognition processing unit 223 converts all character images in the character area of the document image, including special symbols, into character codes by the character recognition processing described above, and then outputs the results of the character recognition processing. In this output processing, the character recognition processing unit 223 outputs the data of the result of the character recognition of the document image to the memory unit 26 and stores it. In addition, in this output processing, the character recognition processing unit 223 may output the data of the result of the character recognition of the document image to the memory unit 26 and also to the operation display unit 21, so that the result of the character recognition is displayed on the display unit of the operation display unit 21. In addition, the character recognition processing unit 223 may output the data of the result of the character recognition of the document image to the memory unit 26 and also to a printing unit (not shown) so that the result of the character recognition is printed on a printing sheet.

Figure 9 is a diagram showing an example of text data, which is data resulting from character recognition output from the character recognition processing unit 223. Figure 9 shows data resulting from character recognition output when character recognition is performed on the original image N shown in Figure 1. As shown in Figure 9, the data resulting from character recognition output from the character recognition processing unit 223 is text data. In the example shown in Figure 9, character recognition is performed in the following order in the original image N: memo N1, memo N2, memo N3, and memo N4.

The text group beginning with the name <Block 1> in FIG. 9 corresponds to the result of character recognition of memo N1 in the document image N shown in FIG. 1. In addition, <Paragraph 1> in <Block 1> shows the recognition result of the special symbol SC1 in memo N1, and <Paragraph 2> shows the recognition result of the text information TX1 in memo N1. In this example, it is assumed that the character code "code a" is assigned to the special symbol SC1 in memo N1, which is extracted as an unrecognized character. Therefore, in <Paragraph 1> in <Block 1>, the character code "code a" assigned to the special symbol SC1 is shown as the recognition result. In addition, in <Paragraph 2> in <Block 1>, the same sentence (character string) as the content of the text information TX1 in memo N1 (see FIG. 1) is shown as the recognition result.

The text group beginning with the name <Block 2> in FIG. 9 corresponds to the result of character recognition of memo N2 in the document image N shown in FIG. 1. In addition, <Paragraph 1> in <Block 2> shows the result of recognition of the special symbol SC2 in memo N2, and <Paragraph 2> shows the result of recognition of the text information TX2 in memo N2. In this example, the special symbol SC2 in memo N2 extracted as an unrecognized character has the same meaning as the special symbol SC1 in memo N1, so the two are grouped together and assigned the same character code. Therefore, in <Paragraph 1> in <Block 2>, the character code "Code a" is shown as the recognition result. In addition, in <Paragraph 2> in <Block 2>, the same sentence (character string) as the content of the text information TX2 in memo N2 (see FIG. 1) is shown as the recognition result.

The text group beginning with the name <Block 3> in FIG. 9 corresponds to the result of character recognition of memo N3 in the original image N shown in FIG. 1. Then, in <Paragraph 1> in <Block 3>, the recognition result of the special symbol SC3 in memo N3 is shown, and in <Paragraph 2>, the recognition result of the text information TX3 in memo N3 is shown. In this example, the special symbol SC3 in memo N3 extracted as an unrecognized character has the same meaning as the special symbol SC1 in memo N1 and the special symbol SC2 in memo N2, so these special symbols are grouped and assigned the same character code. Therefore, in <Paragraph 1> in <Block 3>, the character code "Code a" is shown as the recognition result. Additionally, in <Paragraph 2> in <Block 3>, a character string that is the same as the content of text information TX3 in memo N3 (see Figure 1) is displayed as the recognition result.

The text group beginning with the name <Block 4> in Figure 9 corresponds to the result of character recognition of memo N4 in the original image N shown in Figure 1. <Paragraph 1> in <Block 4> shows the recognition result of special symbol SC4 in memo N4, and <Paragraph 2> shows the recognition result of text information TX4 in memo N4. In this example, special symbol SC4 in memo N4, extracted as an unrecognized character, is recognized as a special symbol different from special symbols SC1 to SC3 in notes N1 to N3. Therefore, <Paragraph 1> in <Block 4> shows the character code "Code b" as the recognition result, which is different from "Code a" assigned to special symbols SC1 to SC3. Additionally, in <Paragraph 2> in <Block 4>, the same sentence (character string) as the content of text information TX4 (see Figure 1) in memo N4 is displayed as the recognition result.

Furthermore, when an unrecognized character is extracted in the character recognition process, the character recognition processing unit 223 outputs and stores data linking the image data of the unrecognized character with the character code assigned thereto, that is, the correspondence data between the unrecognized character and the character code, to the storage unit 26. This allows the character recognition system 1 to manage unrecognized characters as well. Furthermore, in this output process, the character recognition processing unit 223 may output the correspondence data between the unrecognized character and the character code to the storage unit 26 and also to the operation display unit 21, so that the correspondence relationship between the unrecognized character and the character code is displayed on the display unit of the operation display unit 21. Furthermore, the character recognition processing unit 223 may output the correspondence data between the unrecognized character and the character code to the storage unit 26 and also to a printing unit (not shown) so that the correspondence data is printed on a printing sheet.

The configuration (mode) of the correspondence data between unrecognized characters and character codes output from the character recognition processing unit 223 is arbitrary. Various modes (output example 1 to output example 2) of the correspondence data between unrecognized characters and character codes will now be described with reference to Figs. 10 to 12. Figs. 10 to 12 are diagrams showing various modes (output examples) of the correspondence data between unrecognized characters and character codes, and are diagrams showing configuration examples of the correspondence data between unrecognized characters and character codes output when character recognition is performed on the document image N shown in Fig. 1.

The correspondence data of output example 1 shown in FIG. 10 has a configuration in which a character code is linked separately to each of the special symbols SC1 to SC4 in the document image N. In other words, the correspondence data of output example 1 shown in FIG. 10 is table-like correspondence data that lists, for each special symbol, a data set of each special symbol and its associated character code. Therefore, in the example shown in FIG. 10, the correspondence data of unrecognized characters and character codes output from the character recognition processing unit 223 includes a data set of special symbol SC1 and its associated character code ("code a"), a data set of special symbol SC2 and its associated character code ("code a"), a data set of special symbol SC3 and its associated character code ("code a"), and a data set of special symbol SC4 and its associated character code ("code b"). In the example shown in FIG. 10, special symbols SC1 to SC4 are extracted (confirmed) as unrecognized characters in the document image N in that order, so in the correspondence table, the data for special symbol SC1 is placed at the top and the data for special symbol SC4 is placed at the bottom.

The correspondence data of the output example 2 shown in FIG. 11 is the correspondence data in which the grouped special symbol group (group I) in the correspondence data of the output example 1 shown in FIG. 10 is taken into consideration. In the correspondence data of the output example 2 shown in FIG. 11, for the grouped special symbols, one special symbol is selected from the multiple special symbols, and only the data set of the selected special symbol and the character code associated with it is included in the correspondence data. Note that the method of selecting one special symbol from the multiple special symbols is arbitrary, but for example, the special character that is first recognized as an unrecognized character among the multiple special symbols may be selected, or the special character that is last recognized as an unrecognized character may be selected. In the example shown in FIG. 11, an example is shown in which the special symbol SC1 that is first recognized as an unrecognized character is selected from the grouped special symbols SC1 to SC3 (group I). Therefore, in the example shown in FIG. 11, the correspondence data between unrecognized characters and character codes output from the character recognition processing unit 223 includes a data set of the special symbol SC1 and its associated character code ("code a"), and a data set of the special symbol SC4 and its associated character code ("code b").

As an example of the output of the corresponding data shown in FIG. 10 and FIG. 11, information for identifying an area in the document image that is recognized as an unrecognized character (e.g., a placement area, etc.) and a character code associated therewith may be included in the corresponding data as a data set. FIG. 12 is a diagram showing the correspondence relationship (output example 3) between the placement areas of special symbols SC1 to SC4 extracted as unrecognized characters in the document image 100 and the character codes associated with each special symbol. In FIG. 12, the placement areas of the special symbols and the character codes associated therewith are shown connected by a dashed line. Although not shown in FIG. 12, the configuration of the corresponding data of output example 3 actually stored in the storage unit 26 is table-like correspondence data that lists the data set of the unrecognized character, the information for identifying the placement area, and the character code associated therewith for each special symbol, as shown in FIG. 10. However, for example, when the corresponding data of output example 3 is displayed on the display unit of the operation display unit 21 or printed on a print sheet, it is controlled to be displayed or printed in the display mode shown in FIG. 12. The corresponding data of output example 3 that is actually stored in storage unit 26 may be configured as a table listing a data set of information for identifying the unrecognized character and its placement area, the character code associated with the unrecognized character, and image data of the unrecognized character.

The output mode of the correspondence data showing the correspondence between the special symbols and the character codes assigned thereto is not limited to the output mode described in Figures 10 to 12, and any mode can be applied as long as it can clearly show the correspondence between the special symbols and the character codes assigned thereto.

[Procedure for character recognition processing for one document image]
Next, a description will be given of the character recognition process for one document image performed in the character recognition system 1. Fig. 13 is a flowchart showing the procedure of the character recognition process for one document image performed in the character recognition system 1 of this embodiment. The process described below is started when one document to be subjected to character recognition is fed to the image reading unit 13 of the image forming apparatus 10.

First, the image reading unit 13 of the image forming device 10 reads an image of one document that is to be subjected to character recognition (step S101). In this process, the image reading unit 13 reads the image of the one document and generates image data (document image) of the read document. The generated document image is then transmitted to the information processing device 20 via the network 30.

Next, the layout analysis unit 221 of the character recognition device unit 22 of the information processing device 20 performs a layout analysis process on the original image received from the image forming device 10 (step S102: see Figures 3 and 4). In this process, the layout analysis unit 221 extracts data of character areas contained in the original image. Also, in this process, the layout analysis unit 221 outputs the extracted character areas to the character extraction unit 222 of the character recognition device unit 22.

Next, the character extraction unit 222 extracts the characters contained in the character area input from the layout analysis unit 221 by row or column (step S103: see Figure 5).

Next, the character segmentation unit 222 performs a process of segmenting each character for each row or column of character strings segmented from the character region (step S104: see FIG. 6). In this process, the character segmentation unit 222 also outputs an image of each segmented character to the character recognition processing unit 223.

Next, the character recognition processing unit 223 normalizes the size of the character image input from the character segmentation unit 222 to a predetermined size (step S105: see FIG. 7). Note that the processes from step S105 to step S111 described below are performed for each character.

Next, the character recognition processing unit 223 compares the normalized character image with the registered characters stored in the memory unit 26 (step S106). In this process, the character recognition processing unit 223 calculates the degree of match between the normalized character image and the registered characters.

Next, the character recognition processing unit 223 determines whether the degree of match between the normalized character image and a specific (one) registered character exceeds a predetermined threshold based on the comparison result (degree of match) between the normalized character image and the registered character (step S107).

If the character recognition processing unit 223 determines in the process of step S107 that the degree of match between the normalized character image and the specific character exceeds a predetermined threshold (if step S107 is determined to be YES), the character recognition processing unit 223 converts the normalized character image into the specific character and converts it into data (step S108). Then, after the process of step S108, the character recognition processing unit 223 performs the process of step S111 described below.

On the other hand, in the process of step S107, if the character recognition processing unit 223 determines that the degree of match between the normalized character image and the specific character does not exceed a predetermined threshold (if step S107 returns a NO determination), the character recognition processing unit 223 determines the normalized character image as an unrecognized character (step S109). Also, in this process, the character recognition processing unit 223 stores the determined unrecognized character in the RAM 25.

Next, the character recognition processing unit 223 links the confirmed unrecognized character to its similar characters (step S110). In this process, the character recognition processing unit 223 links the registered characters similar to the unrecognized character as similar characters to the unrecognized character based on the degree of match with the registered characters obtained in the matching process for the unrecognized character performed in step S106. In the example shown in FIG. 8, three similar characters with high degrees of match are linked to one unrecognized character. In this process, the character recognition processing unit 223 also outputs a data set (see FIG. 8) in which the unrecognized character is linked to its similar characters to the storage unit 26 for storage.

After processing of step S108 or after processing of step S110, the character recognition processing unit 223 determines whether character recognition has been completed for all character images contained in the character area of the document image (step S111).

If, in the process of step S111, the character recognition processing unit 223 determines that character recognition has not been completed for all character images contained in the character area of the document image (if the determination in step S111 is NO), the character recognition processing unit 223 returns the process to the process of step S105 and repeats the processes of steps S105 to S111.

On the other hand, in the process of step S111, if the character recognition processing unit 223 determines that character recognition has been completed for all character images contained in the character area of the document image (if step S111 is a YES judgment), the character recognition processing unit 223 determines whether or not there are unrecognized characters in the group of characters that have been character recognized (step S112). If unrecognized characters are stored in the RAM 25 at this stage of the process, step S112 will be a YES judgment, and otherwise step S112 will be a NO judgment.

If, in the process of step S112, the character recognition processing unit 223 determines that there are no unrecognized characters in the group of characters that have been subjected to character recognition (if the determination in step S112 is NO), the character recognition processing unit 223 performs the process of step S117 described below.

On the other hand, in the process of step S112, if the character recognition processing unit 223 determines that an unrecognized character exists among the group of characters that have been subjected to character recognition (if step S112 is determined to be YES), the character recognition processing unit 223 performs an acquisition and comparison process of the registered special symbols stored in the memory unit 26 (step S113).

In this process, the character recognition processing unit 223 first acquires a registered special symbol that is an unrecognized character to which a character code has already been assigned and is stored in the storage unit 26. Next, the character recognition processing unit 223 compares the unrecognized character stored in the RAM 25 with the registered special symbol, and determines whether or not a special symbol having the same meaning as the unrecognized character exists in the registered special symbol group. Any method can be adopted for this determination process. For example, as in the process of step S106 described above, the determination may be made by calculating the degree of match between a similar character linked to the unrecognized character and a registered special symbol, or the similar character linked to the unrecognized character and the similar character linked to the registered special symbol may be compared. In the latter case, for example, the most similar character linked to the unrecognized character and the most similar character linked to the registered special symbol may be compared, and if the two match, it may be determined that the registered special symbol is a special symbol having the same meaning as the unrecognized character.

In addition, in the processing of step S113, if the above-mentioned comparison process determines that a special symbol having the same meaning as the unrecognized character exists in the group of registered special symbols, the character recognition processing unit 223 determines that the unrecognized character is a special symbol similar to the registered special symbol determined to have the same meaning.

Then, the character recognition processing unit 223 performs grouping and special symbol determination processing on unrecognized characters for which it has been determined by the above-mentioned comparison processing in step S113 that there is no special symbol with the same meaning in the group of registered special symbols (step S114). Note that if there is no unrecognized character for which it has been determined by the above-mentioned comparison processing in step S113 that there is no special symbol with the same meaning in the group of registered special symbols, the processing in step S114 is omitted.

If there is only one unrecognized character for which it is determined that there is no special symbol with the same meaning in the group of registered special symbols at the processing stage of step S114, the character recognition processing unit 223 determines the unrecognized character as a new special symbol. Also, if there are multiple unrecognized characters for which it is determined that there is no special symbol with the same meaning in the group of registered special symbols at the processing stage of step S114, the character recognition processing unit 223 first performs a grouping process on the multiple unrecognized characters. In this grouping process, unrecognized characters with the same meaning among multiple unrecognized characters are determined to be the same special symbol.

The grouping process for the multiple unrecognized characters is performed based on the similar characters associated with each unrecognized character and their degree of matching. Specifically, the character recognition processing unit 223 first compares the most similar characters associated with each unrecognized character with each other, and determines that the multiple unrecognized characters having the same most similar characters are the same special symbol and group them. Next, the character recognition processing unit 223 compares the similar characters associated with the unrecognized characters not grouped with the next most similar characters associated with the special symbols that are the grouped unrecognized characters. Next, in the comparison process, if there is a grouped special symbol associated with the same similar character as the next most similar character associated with the unrecognized characters not grouped, the character recognition processing unit 223 determines that the unrecognized character is a special symbol similar to the grouped special symbol and groups it. The character recognition processing unit 223 then performs the above comparison process until there are no more similar characters associated with the ungrouped unrecognized character to be compared, and if any unrecognized characters remain that are not grouped, the unrecognized character is determined to be a new special symbol.

For example, when the above grouping process is applied to the document image N shown in FIG. 1, the first similar character comparison process by the character recognition processing unit 223 groups the special symbols SC1 and SC2, which are the same character "重" that has the highest degree of similarity. Then, in the second similar character comparison process by the character recognition processing unit 223, the similar character "動" that has the second highest degree of similarity after the special symbol SC3 is included in the similar characters linked to the special symbols SC1 and SC2, so the special symbol SC3 is grouped with the special symbols SC1 and SC2. Finally, the similar characters that do not match the similar characters linked to each of the special symbols SC1 to SC3, and the linked special symbol SC4, remain ungrouped.

Returning to the explanation of the processing of step S115, the character recognition processing unit 223 assigns a character code to each special symbol determined in step S113 and/or step S114 (step S115). Since the special symbol determined in the processing of step S113 is a registered special symbol, the character recognition processing unit 223 assigns the character code associated with the registered special symbol to the special symbol determined in the processing of step S113. As a result, the special symbol determined in the processing of S113 and the corresponding registered special symbol are grouped. Furthermore, since the special symbol determined in the processing of step S114 is a special symbol that is not registered in the character recognition system 1, the character recognition processing unit 223 assigns a character code different from the character code associated with the registered special symbol to the special symbol determined in the processing of step S114.

In this embodiment, a combination of characters predetermined for special symbols, such as character strings "code a" and "code b" in FIG. 10, is used as the character code assigned to the special symbol, but the present invention is not limited to this. For example, characters other than the registered characters used in the matching process in step S106 may be used as the character code for the special symbol. Also, for example, characters not included in the character strings correctly recognized in the matching process in step S106 (such as text information TX1-TX4 in FIG. 1) may be used as the character code for the special symbol. Also, for example, a character string combining characters not included in the character strings correctly recognized in the matching process in step S106 (such as text information TX1-TX4 in FIG. 1) may be used as the character code for the special symbol.

Next, the character recognition processing unit 223 outputs and stores in the memory unit 26 a data set linking the confirmed special symbol with the character code assigned to it (step S116).

If the result of step S112 is NO (if there are no unrecognized characters), or after step S116, the character recognition processing unit 223 outputs and stores text data (see FIG. 9), which is the result of the character recognition processing on the document image, in the storage unit 26 (step S117). After step S117, the character recognition processing on one document image ends.

[Procedure for character recognition processing on multiple document images]
Next, a description will be given of character recognition processing for a plurality of document images in the character recognition system 1. Fig. 14 is a flowchart showing the procedure of character recognition processing for a plurality of document images in the character recognition system 1 of this embodiment. The processing described below is started when a plurality of documents to be character-recognized are fed to the image reading unit 13 of the image forming apparatus 10.

Note that in character recognition processing for multiple original images, the character recognition processing for one original image described in FIG. 13 is repeated a number of times corresponding to the number of original images. Therefore, the processing contents of steps S201 to S217 during the character recognition processing for multiple original images shown in FIG. 14 are the same as the processing contents of steps S101 to S117 during the character recognition processing for one original image described in FIG. 13. Therefore, here, the description of the processing of steps S201 to S217 during the character recognition processing for multiple original images shown in FIG. 14 will be omitted, and the description will start from the processing following step S217.

After processing in step S217, the character recognition processing unit 223 determines whether reading of all document images has been completed (step S218).

If, in the process of step S218, the character recognition processing unit 223 determines that reading of all document images has not been completed (if step S218 is a NO judgment), the character recognition processing unit 223 returns the process to the process of step S201 and repeats the processes from step S201 onwards. On the other hand, in the process of step S218, if the character recognition processing unit 223 determines that reading of all document images has been completed (if step S218 is a YES judgment), the character recognition processing unit 223 ends the character recognition process for the multiple document images.

In the character recognition process for multiple original images described above, unrecognized characters extracted from a specific original image among the multiple original images are grouped together with unrecognized characters extracted from an original image other than the specific original image among the multiple original images, and it is possible to assign the same character code to each of the grouped unrecognized characters.

[effect]
As described above, in the character recognition system 1 of the present embodiment, even if the document image contains a special symbol that cannot be properly recognized by the character recognition process, such as a handwritten special symbol, a character code is assigned to the special symbol, the special symbol is converted into data, and the digitized special symbol is stored in the storage unit 26. In addition, in the character recognition system 1 of the present embodiment, similar characters are linked to the extracted special character, and multiple special symbols having the same meaning are grouped based on the similar characters. Therefore, in the character recognition system 1 of the present embodiment, even if the document image contains unrecognized characters such as handwritten special symbols, the special symbols can be efficiently searched for, classified, and the like.

In addition, a character recognition system has been proposed that simultaneously displays the candidate character with the highest degree of match (degree of recognition) for each recognized character and other candidate characters, thereby giving the user a clue to guess the original character even if a recognition error occurs. However, in such a character recognition system, even if there are multiple identical characters in the text to be recognized, candidate characters are displayed individually for each character. Therefore, in this case, multiple identical characters cannot be managed together. In contrast, in this embodiment, unrecognized characters having the same meaning, such as handwritten special symbols, can be grouped and managed, thereby solving the above problem.

<Various modified examples>
The configuration of the character recognition system 1 and the character recognition method according to the present embodiment have been described above. However, the present invention is not limited to these, and various other modified embodiments can be adopted without departing from the gist of the present invention described in the claims.

In the character recognition system 1 of the above embodiment, an example has been described in which the results of character recognition of the document image are output to and stored in the memory unit 26 of the information processing device 20, but the present invention is not limited to this. For example, the results of character recognition of the document image may be transmitted to the image forming device 10 via the communication unit 27 and network 30 of the information processing device 20 and also stored in the memory unit 18 of the image forming device 10. In this case, it is also possible to display the results of character recognition of the document image on the display unit of the operation display unit 11 of the image forming device 10.

In the character recognition system 1 of the above embodiment, an example has been described in which character recognition processing is performed on an original image read by the image forming device 10, but the present invention is not limited to this. For example, it is also possible to import an image captured by a digital camera, a smartphone, or the like as an original image into the character recognition device unit 22 of the information processing device 20 and perform character recognition processing thereon.

1...character recognition system, 10...image forming device, 11, 21...operation display unit, 12...automatic document feeder, 13...image reading unit, 14...printer unit, 15, 23...CPU, 16, 24...ROM, 17, 25...RAM, 18, 26...storage unit, 19, 27...communication unit, 20...information processing device, 22...character recognition device unit, 30...network, 101, 201...bus, 221...layout analysis unit, 222...character extraction unit, 223...character recognition processing unit

Claims (13)

an image reading unit that reads an image of a document;
a character recognition unit that extracts characters included in the image of the document read by the image reading unit and performs character recognition on the extracted characters;
a storage unit that stores a result of character recognition performed on the image of the document;
the character recognition unit extracts unrecognized characters that cannot be recognized by character recognition, links similar characters to the unrecognized characters, compares the similar characters linked to each unrecognized character among the plurality of unrecognized characters, groups at least a portion of the plurality of unrecognized characters based on a comparison result, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters to the memory unit. 2. The character recognition system according to claim 1, wherein the character recognition unit selects a predetermined unrecognized character from the grouped plurality of unrecognized characters, and outputs and stores data linking the selected predetermined unrecognized character with the predetermined character code to the storage unit. the character recognition unit outputs data associating the unrecognized character with a plurality of similar characters associated with the unrecognized character, and a degree of match between the unrecognized character and each of the similar characters to the storage unit, and stores the data;
3. The character recognition system according to claim 1, wherein when a plurality of unrecognized characters are extracted by the character recognition unit, the character recognition unit compares the plurality of similar characters associated with each of the plurality of unrecognized characters among the plurality of unrecognized characters, and assigns the same predetermined character code to unrecognized characters having the same similar character among the plurality of unrecognized characters. When a plurality of unrecognized characters are extracted by the character recognition unit,
The character recognition unit is
comparing the plurality of similar characters associated with each of the plurality of unrecognized characters, and assigning the same predetermined character code to unrecognized characters having the same similar character that has the highest degree of matching among the plurality of unrecognized characters;
comparing the plurality of similar characters associated with each of the first unrecognized characters to which the predetermined character code is assigned among the plurality of unrecognized characters with a second most similar character that is associated with each of the remaining second unrecognized characters to which the predetermined character code is not assigned, and assigning the predetermined code to the second unrecognized character that is included in the plurality of similar characters associated with each of the first unrecognized characters and that has the second most similar character that is included in the plurality of similar characters that are associated with each of the first unrecognized characters;
The character recognition system according to claim 3, characterized in that a comparison process between the plurality of similar characters associated with each of the first unrecognized characters and the similar characters associated with each of the second unrecognized characters is repeatedly performed until there are no more similar characters associated with the second unrecognized characters to be compared. When there are a plurality of images of the document to be read by the image reading unit,
The character recognition unit is
Performing character recognition on each image of the document;
The character recognition system according to any one of claims 1 to 4, characterized in that it is possible to group unrecognized characters extracted from a specified document image among the plurality of document images and unrecognized characters extracted from a document image other than the specified document image among the plurality of document images, and to assign the same specified character code to each of the grouped unrecognized characters. an image reading unit that reads an image of a document;
a character recognition unit that extracts characters included in the image of the document read by the image reading unit and performs character recognition on the extracted characters;
a storage unit that stores a result of character recognition performed on the image of the document;
When there are a plurality of images of the document to be read by the image reading unit, the character recognition unit performs character recognition for each image of the document, extracts unrecognized characters that cannot be recognized by character recognition, groups the unrecognized characters extracted in a predetermined document image among the plurality of images of the document and the unrecognized characters extracted in a document image different from the predetermined document image among the plurality of images of the document, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters to the storage unit.
A character recognition system comprising: When outputting and storing a result of the character recognition for the image of the document to the storage unit, the character recognition unit outputs data linking the unrecognized character with the predetermined character code to the storage unit and stores the data.
7. The character recognition system according to claim 1, In the data in which the unrecognized character is associated with the predetermined character code, information specifying an area in the image of the document in which the unrecognized character is located is associated with the predetermined character code.
8. The character recognition system according to claim 1, The data linking the unrecognized characters with the predetermined character codes is table data.
9. The character recognition system according to claim 1, 10. The character recognition system according to claim 1, wherein the predetermined character code is a character other than a character that has been recognized by the character recognition unit. 10. The character recognition system according to claim 1, wherein the predetermined character code is a character string that combines characters other than those that can be recognized by the character recognition unit. A character recognition method for a character recognition system including an image reading unit that reads an image of a document, a character recognition unit that extracts characters included in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a storage unit that stores a result of the character recognition on the image of the document, comprising:
the character recognition unit extracts unrecognized characters that cannot be recognized by character recognition, links similar characters to the unrecognized characters, compares the similar characters linked to each unrecognized character among the plurality of unrecognized characters, groups at least a portion of the plurality of unrecognized characters based on a comparison result, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters to the memory unit. A character recognition method for a character recognition system including an image reading unit that reads an image of a document, a character recognition unit that extracts characters included in the image of the document read by the image reading unit and performs character recognition on the extracted characters, and a storage unit that stores a result of the character recognition on the image of the document, comprising:
When there are a plurality of images of the document to be read by the image reading unit, the character recognition unit performs character recognition for each image of the document, extracts unrecognized characters that cannot be recognized by character recognition, groups the unrecognized characters extracted in a predetermined document image among the plurality of images of the document and the unrecognized characters extracted in a document image different from the predetermined document image among the plurality of images of the document, assigns the same predetermined character code to each of the grouped unrecognized characters, and outputs and stores data in which the same predetermined character code is assigned to each of the grouped unrecognized characters to the storage unit.
A character recognition method comprising: