JP7642944B2 - Information processing device, program, and information processing method - Google Patents

Description

The present invention relates to an information processing apparatus , a program , and an information processing method .

Information processing devices such as character recognition devices generally employ optical character recognition (OCR) technology (see, for example, Patent Document 1). OCR technology is a technology that captures characters written on a medium such as a form as an image using optical means such as a camera or image scanner, and converts the characters in the captured image into character information (for example, character codes) that can be used by a computer or the like.

JP 2010-073201 A

However, the character recognition accuracy of this type of OCR technology is not 100%. For this reason, there is a demand for information processing devices that can improve the accuracy of character recognition in character recognition processing.

An information processing device according to a preferred embodiment of the present invention includes a recognition unit that recognizes a character included in a character image, which is an image of a document including one or more items, by performing a plurality of character recognition processes that recognize characters included in an image and calculate the accuracy of the recognized character, and a determination unit that determines one of a plurality of candidate characters obtained as a recognition result of the one character by each of the plurality of character recognition processes as a confirmed character corresponding to the recognition result of the one character, where a priority of the character recognition processes used to recognize characters is determined for each item, and the determination unit determines one of the plurality of candidate characters as the confirmed character based on the accuracy of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes.

An information processing device according to another preferred aspect of the present invention includes a recognition unit that recognizes a character included in a character image by a first character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character, and a second character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character, and a determination unit that, when a common character exists that is included in both one or more first candidate characters obtained as a recognition result of the one character by the first character recognition process and one or more second candidate characters obtained as a recognition result of the one character by the second character recognition process, determines whether or not to make the common character a confirmed character corresponding to the recognition result of the one character.

An information processing device according to another preferred aspect of the present invention includes a recognition unit that executes a first character recognition process that recognizes a plurality of characters constituting a character string included in an image, and a second character recognition process that recognizes one of the plurality of characters included in the image, and a determination unit that, when a first candidate character obtained as a recognition result of the one character by the first character recognition process and a second candidate character obtained as a recognition result of the one character by the second character recognition process are the same character, determines the same character as a confirmed character corresponding to the recognition result of the one character.

A program according to a preferred embodiment of the present invention causes a processor to function as a recognition unit that recognizes a character included in a character image, which is an image of a document including one or more items, by performing a plurality of character recognition processes that recognize characters included in an image and calculate the accuracy of the recognized character, and a determination unit that determines one of a plurality of candidate characters obtained as a recognition result of the one character by each of the plurality of character recognition processes as a determined character corresponding to the recognition result of the one character, where a priority of the character recognition processes used to recognize characters is determined for each item, and the determination unit determines one of the plurality of candidate characters as the determined character based on the accuracy of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes.

In addition, a program according to another preferred aspect of the present invention causes a processor to function as a recognition unit that recognizes a character included in a character image by each of a first character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character, and a second character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character, and a determination unit that, when a common character exists that is included in both one or more first candidate characters obtained as a recognition result of the one character by the first character recognition process and one or more second candidate characters obtained as a recognition result of the one character by the second character recognition process, determines whether or not to make the common character a confirmed character corresponding to the recognition result of the one character.

In addition, a program according to another preferred aspect of the present invention causes a processor to function as a recognition unit that executes a first character recognition process that recognizes a plurality of characters constituting a character string included in an image, and a second character recognition process that recognizes one of the plurality of characters included in the image, and a determination unit that, when a first candidate character obtained as a recognition result of the one character by the first character recognition process and a second candidate character obtained as a recognition result of the one character by the second character recognition process are the same character, determines the same character as a confirmed character corresponding to the recognition result of the one character.

The present invention makes it possible to improve the accuracy of character recognition in character recognition processing.

1 is an explanatory diagram for explaining an overview of a character recognition system according to a first embodiment; FIG. 13 is an explanatory diagram showing an example of an application form image. FIG. 2 is a diagram illustrating an example of a hardware configuration of the character recognition device illustrated in FIG. 1 . 2 is a functional block diagram showing an example of the configuration of the character recognition device shown in FIG. 1 . FIG. 4 is an explanatory diagram illustrating an example of the priority table illustrated in FIG. 3 . FIG. 11 is an explanatory diagram for explaining an example of a confirmed character determination process. FIG. 13 is an explanatory diagram showing an example of an outline of a content confirmation screen. 2 is a flowchart showing an example of the operation of the character recognition device shown in FIG. 1 . 9 is a flowchart showing an example of the confirmed character determination process shown in FIG. 8 . 10 is a flowchart showing another example of the confirmed character determination process shown in FIG. 8 . FIG. 11 is an explanatory diagram for explaining an overview of a character recognition device according to a second embodiment. 12 is an explanatory diagram for explaining an example of an operation of a recognition result determination unit shown in FIG. 11 . 12 is a flowchart showing an example of an operation of a recognition result determination unit shown in FIG. 11 . FIG. 13 is an explanatory diagram for explaining an overview of a character recognition device according to a third embodiment. 15 is an explanatory diagram for explaining an example of an operation of the recognition result determination unit shown in FIG. 14; FIG. 11 is an explanatory diagram for explaining an example of a weighting table according to a first modified example. FIG. 11 is an explanatory diagram for explaining an example of a confirmed character determination process according to a first modified example. FIG. 13 is a diagram illustrating an example of a character recognition system according to a second modified example.

Below, the embodiments for carrying out the present invention will be described with reference to the drawings. Note that in each drawing, the dimensions and scale of each part are appropriately different from the actual ones. In addition, the embodiments described below are preferred examples of the present invention, and therefore various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[1. First embodiment]
First, an example of an overview of a character recognition system 1 according to a first embodiment will be described with reference to FIG.

Figure 1 is an explanatory diagram for explaining an overview of the character recognition system 1 according to the first embodiment.

The character recognition system 1 shown in FIG. 1 includes a character recognition device 10, a terminal device 20, and an image input device 30. The character recognition device 10 is an example of an "information processing device." The character recognition device 10 and the terminal device 20 are communicatively connected to each other, for example, via a network NW. The network NW may include the Internet and a local area network. For example, the network NW includes one or both of a wired network and a wireless network. Furthermore, the connection between the network NW and the character recognition device 10 may be, for example, a connection that allows multiple elements to communicate with each other, and may be a connection using either a wired or wireless connection, or a connection using both a wired and wireless connection.

As the character recognition device 10, any information processing device that can be connected to the network NW can be adopted. The character recognition device 10 executes a character recognition process that recognizes characters included in an image, for example, using optical character recognition (OCR) technology. An example of an image that is the subject of the character recognition process is described in FIG. 2, which will be described later. The configuration of the character recognition device 10 is described in FIG. 3 and FIG. 4, which will be described later. In this embodiment, it is assumed that the character recognition device 10 recognizes characters written on a membership application form for joining a specified service. For example, in this embodiment, the character recognition device 10 converts information written on the membership application form into electronic data using optical character recognition technology.

As the terminal device 20, any information processing device that can be connected to the network NW can be adopted. Specifically, the terminal device 20 may be, for example, a stationary information device such as a personal computer, or may be a portable information terminal such as a notebook personal computer, a tablet terminal, or a smartphone. The terminal device 20 shown in FIG. 1 has an operation device 27 that accepts operations by an operator of the terminal device 20, and a display device 28 that displays images, etc. For example, the terminal device 20 displays the results of the character recognition process, etc. on the display device 28. The results of the character recognition process are transmitted to the terminal device 20 from the character recognition device 10 via the network NW, for example. The display device 28 is an example of a "display unit".

The terminal device 20 is also connected to the image input device 30 so that they can communicate with each other. The connection between the terminal device 20 and the image input device 30 may be a wired or wireless connection, or may be a wired and wireless connection. The terminal device 20, for example, acquires image information indicating an image to be subjected to character recognition processing from the image input device 30, and transmits the image information acquired from the image input device 30 to the character recognition device 10 via the network NW.

The terminal device 20 does not have to include the display device 28. When the terminal device 20 does not include the display device 28, it is communicatively connected to the display device 28. In this case, the connection between the terminal device 20 and the display device 28 may be a wired or wireless connection, or may be both a wired and wireless connection.

The image input device 30 is, for example, an optical device such as a camera or an image scanner. For example, the image input device 30 captures an image of a document such as a membership application form, thereby generating image information showing an image of the document. The image information generated by the image input device 30 is transmitted to the terminal device 20. Note that the image input device 30 may be included in the terminal device 20.

Figure 2 is an explanatory diagram showing an example of an application form image GM.

The application form image GM is, for example, an image of a membership application form that includes multiple items. A membership application form that includes multiple items is an example of a "document that includes one or more items." In other words, the application form image GM is an example of a "text image."

For example, the application form image GM is an image that is the subject of character recognition processing. Therefore, the application form image GM includes multiple fields corresponding to multiple items. In the example shown in FIG. 2, the multiple items written on the application form are the application number, kana name, name, postal code, address, email address, and phone number.

Each of the multiple entry fields includes an item name entry area AM in which an item name, which is the name of an item of the application content to be entered in that entry field, is entered, and an application content entry area AK in which the application content related to that entry field is entered. Hereinafter, among the application form image GM, an image showing a character string related to the application content entered in each of the multiple application content entry areas AK is also referred to as a character string image GCL. Also, in FIG. 2, in order to distinguish the multiple character string images GCL from one another, a number (1, 2, 3, 4, 5, 6, or 7) is added to the end of each of the symbols of the multiple character string images GCL.

For example, character string image GCL1 shows a character string related to the application content written in the application content entry area AK related to "application number". Character string image GCL2 shows a character string related to the application content written in the application content entry area AK related to "kana name", and character string image GCL3 shows a character string related to the application content written in the application content entry area AK related to "name". Character string image GCL4 shows a character string related to the application content written in the application content entry area AK related to "zip code", and character string image GCL5 shows a character string related to the application content written in the application content entry area AK related to "address". Character string image GCL6 shows a character string related to the application content written in the application content entry area AK related to "email", and character string image GCL7 shows a character string related to the application content written in the application content entry area AK related to "telephone number".

In the following description, an image showing each of the multiple characters included in the character string represented by each of the multiple character string images GCL is also referred to as an individual character image GC. That is, each of the multiple character string images GCL is an image that includes multiple individual character images GC. For example, character string image GCL5 includes an individual character image GC1 of "sea" and an individual character image GC2 of "big", etc.

In the following, it is assumed that the character recognition device 10 performs character recognition processing on an application form image GM having the format exemplified in FIG. 2, but the present invention is not limited to this aspect. In this embodiment, the character recognition device 10 can perform character recognition processing on documents having any format or text with no set format (e.g., blank notes, etc.). In the following, it is assumed that the application content written in the application content writing area AK is written in handwritten characters, but the present invention is not limited to this aspect. For example, in this embodiment, some or all of the multiple characters shown in one or more character string images GCL may be written in type.

Next, the hardware configuration of the character recognition device 10 will be described with reference to FIG. 3.

Figure 3 is a diagram showing an example of the hardware configuration of the character recognition device 10 shown in Figure 1.

The character recognition device 10 has a processing device 12 that controls each part of the character recognition device 10, a storage device 14 that stores various information, and a communication device 16.

The storage device 14 includes, for example, one or both of a volatile memory such as a RAM (Random Access Memory) that functions as a working area for the processing device 12, and a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) that stores various information such as the control program PG1. The storage device 14 may be detachable from the character recognition device 10. Specifically, the storage device 14 may be a storage medium such as a memory card that is detachable from the character recognition device 10. The storage device 14 may also be, for example, a storage device (e.g., online storage) that is communicatively connected to the character recognition device 10 via a network NW or the like.

The storage device 14 shown in FIG. 3 stores a control program PG1, a priority table PTBL, and multiple learning models LM. In FIG. 3, in order to distinguish the multiple learning models LM from one another, a number (e.g., 1) or a lowercase alphabet (e.g., m) is added to the end of each of the symbols for the multiple learning models LM. For example, m indicates a natural number equal to or greater than 2.

The control program PG1 is an example of a "program." In this embodiment, the control program PG1 includes, for example, an application program for the character recognition device 10 to execute the character recognition process. However, the control program PG1 may also include, for example, an operating system program for the processing device 12 to control each part of the character recognition device 10.

The priority table PTBL stores information indicating multiple priorities associated with multiple character recognition units CR, for example, as shown in FIG. 4, which will be described later. The priority is the priority of the character recognition unit CR used to recognize characters, and is determined for each item. Details of the priority table PTBL will be described in FIG. 5, which will be described later.

Each of the multiple learning models LM is, for example, a multi-layer neural network, and is a learning model that learns the relationship between images containing one or more types of characters and the characters contained in the images. For ease of explanation, in this embodiment, it is assumed that the types of characters shown in the images are six types: kanji, hiragana, katakana, Arabic numerals (hereinafter also simply referred to as "numbers"), alphabets, and symbols. Here, symbols are a concept that includes, for example, dots, commas, periods, commas, hyphens, and arithmetic symbols.

The processing device 12 is a processor that controls the entire character recognition device 10, and is configured to include, for example, one or more CPUs (Central Processing Units). The processing device 12 executes, for example, a control program PG1 stored in the storage device 14, and operates according to the control program PG1, thereby functioning as a control unit 120 shown in FIG. 4, which will be described later. The control program PG1 may be transmitted from another device via the network NW.

For example, when the processing device 12 is configured to include multiple CPUs, some or all of the functions of the processing device 12 may be realized by these multiple CPUs operating in cooperation with each other according to a program such as the control program PG1. Furthermore, the processing device 12 may be configured to include hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to one or more CPUs, or in place of some or all of the one or more CPUs. In this case, some or all of the functions of the processing device 12 may be realized by hardware such as a DSP.

The communication device 16 is hardware for communicating with an external device outside the character recognition device 10 via one or both of a wired network and a wireless network.

Next, the functions of the character recognition device 10 will be described with reference to FIG.

Figure 4 is a functional block diagram showing an example of the configuration of the character recognition device 10 shown in Figure 1.

The control unit 120 is realized by the processing device 12 as described in FIG. 3. If the processing device 12 is configured to include multiple CPUs, some or all of the functions of the control unit 120 may be realized by these multiple CPUs operating in cooperation with each other according to the control program PG1. If the processing device 12 is configured to include hardware such as a DSP, some or all of the functions of the control unit 120 may be realized by the hardware such as a DSP.

The control unit 120 has, for example, an image information acquisition unit 122, a description content recognition unit 124, a recognition result determination unit 126, and a display information generation unit 128. The description content recognition unit 124 is an example of a "recognition unit", and the recognition result determination unit 126 is an example of a "determination unit". Furthermore, the display information generation unit 128 is an example of a "display control unit".

The image information acquisition unit 122 acquires, for example, image information (image information generated by the image input device 30) indicating the application form image GM to be subjected to character recognition processing from the terminal device 20 via the communication device 16.

The written content recognition unit 124 has multiple character recognition units CR. In this embodiment, it is assumed that the multiple character recognition units CR correspond one-to-one to the multiple learning models LM. In FIG. 4, in order to distinguish the multiple character recognition units CR from one another, a number (e.g., 1) or a lowercase alphabet (e.g., m) is added to the end of each of the symbols of the multiple character recognition units CR. For example, m indicates a natural number equal to or greater than 2.

Each of the multiple character recognition units CR performs character recognition processing, for example, by recognizing characters contained in an image and calculating a degree of accuracy indicating the likelihood of the recognized characters. The description content recognition unit 124 causes two or more of the multiple character recognition units CR to perform character recognition processing on the application form image GM indicated by the image information acquired by the image information acquisition unit 122. For example, the description content recognition unit 124 selects two or more character recognition units CR from the multiple character recognition units CR based on information such as priority stored in the priority table PTBL. Then, the description content recognition unit 124 recognizes characters contained in the application form image GM by having each of the two or more character recognition units CR selected from the multiple character recognition units CR perform character recognition processing on the application form image GM. Hereinafter, the degree of accuracy indicating the likelihood of the characters recognized by the character recognition unit CR is also simply referred to as the degree of accuracy of the characters.

Each of the multiple character recognition units CR (CR1 to CRm) uses a corresponding learning model LM from among the multiple learning models LM (LM1 to LMm) to recognize the character indicated by the individual character image GC. That is, in this embodiment, each of the multiple character recognition units CR uses a corresponding learning model LM from among the multiple learning models LM stored in the storage device 14 to perform character recognition processing using AI (Artificial Intelligence)-OCR.

In this embodiment, it is assumed that the multiple character recognition units CR use different algorithms to perform character recognition processing. For example, the algorithm used by the character recognition unit CR1 in the character recognition processing is different from the algorithm used by the character recognition unit CRm in the character recognition processing. Here, "different algorithms" may mean, for example, that the types of learning models LM used in the character recognition processing are different, or that the data sets used when training the learning models LM used in the character recognition processing are different. Alternatively, "different algorithms" may include, for example, both the concepts of the types of learning models LM used in the character recognition processing being different and the data sets used when training the learning models LM used in the character recognition processing being different. The type of learning model LM is, for example, a type of neural network.

Furthermore, "different data sets" means, for example, that the characters contained in the images used in one of the two data sets include a different type of characters than the characters contained in the images used in the other of the two data sets. Note that when the two data sets are different from each other, the images used in one of the two data sets may include both characters of a different type and the same type as the characters contained in the images used in the other of the two data sets.

In addition, when the two data sets used in the two learning models LM are different from each other, the two neural networks constituting the two learning models LM may be of different types or may be of the same type. In other words, the fact that the two data sets used in the two learning models LM are different from each other corresponds to "different algorithms" regardless of whether the types of the two neural networks are different from each other. In addition, for example, the fact that the two neural networks constituting the two learning models LM are different from each other corresponds to "different algorithms" regardless of whether the two data sets used in the two learning models LM are different from each other.

In this embodiment, it is assumed that the multiple character recognition units CR correspond one-to-one to the multiple learning models LM, but the present invention is not limited to this aspect. For example, some or all of the multiple character recognition units CR may be components that perform character recognition processing without using a neural network. In addition, if all of the multiple character recognition units CR are components that perform character recognition processing without using a neural network, the storage device 14 does not need to store the learning models LM.

The multiple character recognition unit CR may also include a character recognition unit CR that recognizes multiple characters that make up a character string included in an image. This type of character recognition unit CR may recognize multiple characters that make up a character string included in an image, for example, by using a predetermined dictionary in which multiple character strings are registered. Alternatively, the character recognition unit CR may recognize multiple characters that make up a character string included in an image by using a learning model LM that has learned to recognize character strings. The learning model LM that has learned to recognize character strings may be, for example, a learning model LM that has learned one or both of the relationship between one or more characters and a character that follows the one or more characters, and the relationship between a character and one or more characters that follows the one character.

The recognition result determination unit 126 acquires recognition information indicating the result of character recognition processing performed by the written content recognition unit 124 on a single character. For example, the recognition information indicates the recognition result of a single character by character recognition processing performed by each of two or more character recognition units CR among the multiple character recognition units CR. The recognition result of a single character by character recognition processing performed by each of two or more character recognition units CR includes, for example, multiple candidate characters that are candidates for the recognition result of the single character, and the accuracy of the multiple candidate characters.

For example, the recognition result determination unit 126 determines one of a plurality of candidate characters obtained as the recognition results of each of two or more character recognition units CR that have performed character recognition processing on one character as the confirmed character corresponding to the recognition result of the one character. Specifically, the recognition result determination unit 126 determines one of the plurality of candidate characters as the confirmed character based on the accuracy of each of the plurality of candidate characters and the priority of each of the plurality of character recognition units CR.

For example, when a weight value based on priority is determined for each of the multiple character recognition units CR, the recognition result determination unit 126 calculates an evaluation value for each of the multiple candidate characters by weighting the accuracy of each of the multiple candidate characters with the weight value of each of the multiple character recognition units CR. The weight value is, for example, a real number equal to or greater than 0. The recognition result determination unit 126 determines the candidate character with the highest evaluation value among the multiple candidate characters as the confirmed character corresponding to the recognition result of one character. Note that, when there are multiple candidate characters with the same evaluation value, the recognition result determination unit 126 may determine the candidate character with the highest accuracy among the multiple candidate characters as the confirmed character, or may determine the candidate character with the highest priority (or weight value) as the confirmed character. Alternatively, the recognition result determination unit 126 may accumulate, for each character recognition unit CR, whether or not a candidate character previously determined as a confirmed character has been corrected, and may determine the candidate character with the highest accuracy rate among multiple candidate characters with the same evaluation value as the confirmed character.

The display information generating unit 128 causes the display device 28 to display the confirmed characters determined by the recognition result determining unit 126. For example, the display information generating unit 128 generates display information for displaying an image for confirming the confirmed characters determined by the recognition result determining unit 126 (for example, a content confirmation screen CCH shown in FIG. 7 described later) on the display device 28. Then, the display information generating unit 128 outputs the generated display information to the terminal device 20 via the communication device 16.

The configuration of the character recognition device 10 is not limited to the examples shown in Figures 3 and 4. For example, the character recognition device 10 may have an operation device that accepts operations by an administrator of the character recognition device 10, etc.

Next, we will explain the priority table PTBL with reference to Figure 5.

Figure 5 is an explanatory diagram showing an example of the priority table PTBL shown in Figure 3. The priority table PTBL is referenced by the written content recognition unit 124 when the written content recognition unit 124 selects two or more character recognition units CR from the multiple character recognition units CR, and is referenced by the recognition result determination unit 126 when the recognition result determination unit 126 calculates an evaluation value, for example.

The priority table PTBL has, for example, multiple records that correspond one-to-one to multiple fields in the application form image GM (the image that is the subject of character recognition processing). Each record in the priority table PTBL has an item ID, an item name, and a priority.

The item ID is information for identifying each entry field among the multiple entry fields in the application form image GM. The item name is the name written in the item name writing area AM of each entry field in the application form image GM.

The priority is, for example, the priority of the character recognition unit CR used to recognize characters written in the entry field indicated by the item ID. Note that a hyphen ("-") in FIG. 5 means that a priority is not associated with the character recognition unit CR. For example, in each entry field (each item), a character recognition unit CR that is not associated with a priority is not used to recognize characters written in that entry field. Also, in this embodiment, a smaller priority number has a higher priority than a larger priority number. In each item, a character recognition unit CR with a higher priority is more suitable for recognizing characters written in the entry field of that item than a character recognition unit CR with a lower priority.

For example, the priority is determined for each item according to the learning level of each of the multiple character recognition units CR. The "learning level" may be a concept that includes, for example, some or all of the number of samples (images) included in the data set used for learning, the number of times learning is performed, and the accuracy rate. The priority table PTBL shown in FIG. 5 assumes that the written content recognition unit 124 has six character recognition units CR1, CR2, CR3, CR4, CR5, and CR6.

For example, the character recognition units CR1 and CR2 are character recognition units CR for alphanumeric characters specialized in recognizing alphabets, numbers, and symbols. One or both of the character recognition units CR1 and CR2 may be character recognition units CR with higher recognition accuracy for alphabets, numbers, and symbols than other character types (e.g., kanji, etc.). For example, the character recognition units CR3 and CR4 are character recognition units CR for kanji characters specialized in recognizing kanji and hiragana. One or both of the character recognition units CR3 and CR4 may be character recognition units CR with higher recognition accuracy for kanji and hiragana than other character types (e.g., numbers, etc.). For example, the character recognition units CR5 and CR6 are character recognition units CR for kana characters specialized in recognizing katakana. One or both of the character recognition units CR5 and CR6 may be character recognition units CR with higher recognition accuracy for katakana than other character types (e.g., numbers, etc.).

For example, in one item among multiple items, the priority of the character recognition unit CR that has a higher learning level for character strings related to the one item out of the two character recognition units CR is set to be higher than the priority of the other character recognition unit CR. In the example shown in FIG. 5, it is assumed that, out of the character recognition units CR3 and CR4 for kanji, the character recognition unit CR4 has a higher learning level for character strings related to "address" than the character recognition unit CR3. Therefore, in the item related to "address", the priority of the character recognition unit CR4 is higher than the priority of the character recognition unit CR3. Therefore, in the item related to "address", for example, the priority of the character recognition unit CR4 is set to "1" and the priority of the character recognition unit CR3 is set to "2".

As shown in FIG. 5, in this embodiment, the priority of the character recognition unit CR used to recognize characters is determined for each item. For example, when the description content recognition unit 124 recognizes a character string written in the application content entry area AK of one of the multiple items, it refers to a record corresponding to the one item among the multiple records held by the priority table PTBL. Then, the description content recognition unit 124 selects, from among the multiple character recognition units CR, a character recognition unit CR that is associated with a priority in the record corresponding to the one item as the character recognition unit CR to recognize the character string written in the application content entry area AK of the one item.

Specifically, when recognizing a character string written in the application content entry area AK related to "zip code", for example, the content recognition unit 124 refers to a record with an item ID of "k004" and selects the character recognition units CR1 and CR2 for alphanumeric characters. Also, when recognizing a character string written in the application content entry area AK related to "address", for example, the content recognition unit 124 refers to a record with an item ID of "k005". Then, the content recognition unit 124 selects the character recognition units CR3 and CR4 for kanji, the character recognition unit CR2 for alphanumeric characters, and the character recognition unit CR5 for kana.

When the recognition result determination unit 126 determines a confirmed character corresponding to the recognition result for a character included in a character string written in the application content description area AK of a certain item, the recognition result determination unit 126 refers to a record corresponding to the certain item among the multiple records in the priority table PTBL. The recognition result determination unit 126 then determines one of multiple candidate characters obtained as the recognition result of two or more character recognition units CR that recognized the certain character as the confirmed character based on the accuracy of each of the multiple candidate characters and each of the multiple priorities stored in the record corresponding to the certain item. Details of the process of determining the confirmed character will be described later in FIG. 6. Hereinafter, the process of determining the confirmed character will also be referred to as the confirmed character determination process.

Note that the priority table PTBL is not limited to the example shown in FIG. 5. For example, some of the multiple items may have only one character recognition unit CR associated with a priority. Also, for example, the priority table PTBL may store a weight value in addition to or instead of the priority. A weight value based on the priority may be determined for each item. For example, the weight value of the character recognition unit CR with a priority of "1" may be different or the same value for the "address" item and the "zip code" item. Also, when a weight value is stored in the priority table PTBL instead of a priority, the weight value corresponds to a "priority". In this embodiment, a large weight value has a higher priority than a small weight value.

The priority may also be determined using an index other than the learning degree. The index other than the learning degree may be, for example, the track record of whether or not the operator has corrected a candidate character previously determined as a confirmed character (for example, the accuracy rate or the correction rate). For example, the priority may be determined for each item according to the accuracy rate or the correction rate of each of the multiple character recognition units CR. In this case, for example, in one item, the priority of a character recognition unit CR with a high accuracy rate (or a character recognition unit CR with a low correction rate) is set higher than the priority of a character recognition unit CR with a low accuracy rate (or a character recognition unit CR with a high correction rate). The correction rate of each character recognition unit CR in each item is expressed, for example, as the ratio of the total number of corrections by the operator to the total number of candidate characters previously determined as confirmed characters in that item. The accuracy rate of each character recognition unit CR in each item is expressed, for example, as the value obtained by subtracting the correction rate of each character recognition unit CR in that item from 1. The accuracy rate or correction rate of each of the multiple character recognition units CR may be calculated for each character type.

Next, we will explain the outline of the final character determination process with reference to Figure 6.

Figure 6 is an explanatory diagram for explaining an example of the confirmed character determination process. In Figure 6, an example of a method for determining a confirmed character corresponding to the recognition result of the individual character image GC2 of "large" is explained using an example in which character recognition process is performed on a character string image GCL5 showing a character string written in the application content entry area AK related to "address".

In FIG. 6, it is assumed that the written content recognition unit 124 and the recognition result determination unit 126 refer to the priority table PTBL shown in FIG. 5. Therefore, in the "Address" item, as shown in FIG. 5, the character recognition unit CR4 is the character recognition unit CR with the highest priority among the multiple character recognition units CR2 to CR5. In FIG. 6, it is assumed that a weight value based on the priority is determined for each of the multiple character recognition units CR. For example, the weight value of the character recognition unit CR4 with a priority of "1" is set to 1.20, the weight value of the character recognition unit CR3 with a priority of "2" is set to 1.05, and the weight value of each of the character recognition units CR2 and CR4 with a priority of "3" is set to 1.00. In addition, the evaluation value is expressed, for example, as the product of the accuracy of the candidate character and the weight value.

In the example shown in FIG. 6, the recognition result determination unit 126 obtains recognition information indicating multiple candidate characters and the accuracy of each of the multiple candidate characters as a result of the character recognition process performed by each of the character recognition units CR2 to CR5 on the individual character image GC2 of "大".

For example, the candidate character obtained from character recognition unit CR3 is the character "丈" with an accuracy of 90%, and the candidate character obtained from character recognition unit CR4 is the character "大" with an accuracy of 85%. Note that the accuracy (85%) of the candidate character ("大") obtained from character recognition unit CR4 is lower than the accuracy (90%) of the candidate character ("丈") obtained from character recognition unit CR3. Also, for example, the candidate character obtained from character recognition unit CR2 is the character "K" with an accuracy of 30%, and the candidate character obtained from character recognition unit CR5 is the character "ス" with an accuracy of 60%. The accuracy of the candidate characters obtained from each of character recognition units CR2 and CR5 is lower than the accuracy of the candidate character obtained from character recognition unit CR3.

The recognition result determination unit 126 calculates an evaluation value by weighting the accuracy of each of the multiple candidate characters with the weight value of each of the multiple character recognition units CR. Specifically, the recognition result determination unit 126 calculates an evaluation value of the candidate character obtained from the character recognition unit CR3 by weighting the accuracy (90%) of the candidate character obtained from the character recognition unit CR3 with the weight value (1.05) of the character recognition unit CR3. Similarly, the recognition result determination unit 126 calculates an evaluation value of the candidate character obtained from the character recognition unit CR4 by weighting the accuracy (85%) of the candidate character obtained from the character recognition unit CR4 with the weight value (1.20) of the character recognition unit CR4.

Then, the recognition result determination unit 126 determines the candidate character with the highest evaluation value among the multiple candidate characters as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大". For example, the evaluation value of the candidate character ("丈") obtained from the character recognition unit CR3 is 95 (= 90 x 1.05), and the evaluation value of the candidate character ("大") obtained from the character recognition unit CR4 is 102 (= 85 x 1.20). The evaluation value of the candidate character ("K") obtained from the character recognition unit CR2 is 30 (= 30 x 1.00), and the evaluation value of the candidate character ("ス") obtained from the character recognition unit CR5 is 60 (= 60 x 1.00). Therefore, in the example shown in FIG. 6, the candidate character with the highest evaluation value among the multiple candidate characters ("丈", "大", "K" and "ス") is the character "大" with an evaluation value of 102, as indicated by the star in FIG. 6. Therefore, the recognition result determination unit 126 determines the character "大" with the highest evaluation value among the multiple candidate characters as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大".

Thus, in the example shown in FIG. 6, the candidate character ("large") obtained from the character recognition unit CR4, which has the highest priority among the multiple character recognition units CR2 to CR5, is determined to be the confirmed character corresponding to the recognition result of the individual character image GC2 of "large". The character recognition unit CR4, which has the highest priority among the multiple character recognition units CR2 to CR5, is more suitable for recognizing character strings related to "address" than the other character recognition units CR2, CR3, and CR5. For example, the character recognition unit CR4 is the character recognition unit CR that has the highest learning level for character strings related to "address" among the multiple character recognition units CR2 to CR5. For this reason, for example, the probability that the candidate character ("large") obtained from the character recognition unit CR4 is incorrect is considered to be lower than the probability that the candidate character obtained from another character recognition unit CR (for example, character recognition unit CR3) is incorrect.

Here, in the example shown in FIG. 6, the candidate character with the highest accuracy among the multiple candidate characters is the character "丈" with an accuracy of 90%, as shown by the double circle in FIG. 6. For example, in a form in which a confirmed character is determined without using an evaluation value (hereinafter also referred to as the first comparative example), the character "丈" with the highest accuracy among the multiple candidate characters is determined as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大". That is, in the first comparative example, for example, a candidate character ("丈") obtained from the character recognition unit CR3, which has a lower learning level for character strings related to "address" than the character recognition unit CR4, is determined as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大". For this reason, in the first comparative example, the probability that an incorrect character is determined as a confirmed character is higher than in the present embodiment. In other words, in the present embodiment, it is possible to suppress the determination of an incorrect character as a confirmed character compared to the first comparative example. That is, in the present embodiment, it is possible to improve the accuracy of character recognition in the character recognition process.

The confirmed character determination process is not limited to the example shown in FIG. 6. For example, the evaluation value may be the sum of the probability of the candidate character and a weight value. In this case, the weight value of each character recognition unit CR may be set to a value different from the weight value assumed in FIG. 6.

For example, whether or not to set a candidate character obtained as a result of recognition of a character by the character recognition unit CR4 having the highest priority among the multiple character recognition units CR2 to CR5 as a confirmed character may be determined without using a weight value, as shown in FIG. 10, which will be described later. In the following, the character recognition unit CR having the highest priority among the multiple character recognition units CR that performed character recognition processing on a character is also referred to as the first priority character recognition unit CR. In the following, a candidate character obtained as a result of recognition of a character by the first priority character recognition unit CR is also referred to as the first priority candidate character. The first priority candidate character is an example of a "first candidate character."

Below, an overview of a method for determining whether or not a first-priority candidate character is to be determined as a confirmed character without using a weight value is described using the recognition result shown in FIG. 6 as an example. For example, the recognition result determination unit 126 may determine whether or not to prioritize the first-priority character recognition unit CR4 over other character recognition units CR depending on whether the accuracy of the first-priority candidate character is equal to or greater than a predetermined value.

For example, when the predetermined value is 80%, the accuracy (85%) of the first priority candidate character obtained as the recognition result of a character by the first priority character recognition unit CR4 is equal to or greater than the predetermined value. In this case, the recognition result determination unit 126 may determine the first priority candidate character ("large") as the confirmed character. In other words, when the accuracy of the first priority candidate character obtained as the recognition result of a character by the character recognition unit CR with the highest priority among the multiple character recognition units CR is equal to or greater than the predetermined value, the recognition result determination unit 126 may determine the first priority candidate character among the multiple candidate characters as the confirmed character.

The first priority character recognition unit CR4 is, for example, the character recognition unit CR that has the highest degree of learning about character strings related to "address" among the multiple character recognition units CR2 to CR5. For this reason, when the accuracy of the first priority candidate character is equal to or greater than a predetermined value, the probability that the first priority candidate character is incorrect is considered to be lower than the probability that the candidate character obtained as the recognition result of a character by another character recognition unit CR is incorrect. Therefore, the recognition result determination unit 126 can prevent an incorrect character from being determined as the determined character by determining the first priority candidate character as the determined character when the accuracy of the first priority candidate character is equal to or greater than a predetermined value.

In addition, when the accuracy of the first-priority candidate character obtained as the recognition result of a character by the first-priority character recognition unit CR is less than a predetermined value, the recognition result determination unit 126 may determine the candidate character with the highest accuracy among the multiple candidate characters as the confirmed character. In this case, candidate characters with accuracy less than the predetermined value are prevented from being determined as the confirmed character.

Here, for example, in a form in which a first-priority candidate character is unconditionally selected as a confirmed character (hereinafter also referred to as the second comparison), even if the accuracy of the first-priority candidate character is less than a predetermined value, the first-priority candidate character is determined as a confirmed character. That is, in the second comparison, when the accuracy of the first-priority candidate character is less than a predetermined value, the first-priority candidate character is determined as a confirmed character regardless of whether or not there is a candidate character whose accuracy is equal to or greater than a predetermined value. When the accuracy of the recognition result of a character is less than a predetermined value, the probability that the character is erroneously recognized is higher than when the accuracy of the recognition result of a character is equal to or greater than a predetermined value. Therefore, in the second comparison, the probability that an incorrect character is determined as a confirmed character is higher than in a form in which the candidate character with the highest accuracy among multiple candidate characters is determined as a confirmed character when the accuracy of the first-priority candidate character is less than a predetermined value.

In other words, even in a configuration in which whether or not to prioritize the first-priority character recognition unit CR over other character recognition units CR is determined depending on whether the accuracy of the first-priority candidate character is equal to or greater than a predetermined value, it is possible to prevent an erroneous character from being determined as the confirmed character. Therefore, in this embodiment, even when whether or not to set the first-priority candidate character as the confirmed character is determined without using a weight value, it is possible to improve the accuracy of character recognition in the character recognition process.

The recognition result determination unit 126 may suspend the determination of the confirmed character when the accuracy of the first priority candidate character obtained as the recognition result of a character by the first priority character recognition unit CR is less than a predetermined value. In this case, for example, the confirmed character may not be displayed on the content confirmation screen CCH shown in FIG. 7 described later, or information indicating that the determination of the confirmed character has been suspended (for example, a "?" mark, etc.) may be displayed. When the determination of the confirmed character by the recognition result determination unit 126 is suspended, the confirmed character may be determined, for example, by having the user input or select the confirmed character.

Next, with reference to FIG. 7, we will explain the overview of the content confirmation screen CCH for confirming the confirmed characters determined by the recognition result determination unit 126.

Figure 7 is an explanatory diagram showing an example of the overview of the content confirmation screen CCH.

The content confirmation screen CCH is displayed, for example, on the display device 28. For example, the content confirmation screen CCH displays a plurality of item names, a plurality of character string images GCL that correspond one-to-one to the plurality of item names, a plurality of recognition character strings LMK that correspond one-to-one to the plurality of character string images GCL, a correction button SSB, and a confirmation button KTB. The plurality of item names correspond one-to-one to a plurality of entry fields in the application form image GM. In FIG. 7, in order to distinguish the plurality of recognition character strings LMK from one another, the same number (1, 2, 3, 4, 5, 6, or 7) is added to the end of the code of each of the plurality of recognition character strings LMK as the number added to the end of the code of the corresponding character string image GCL among the plurality of character string images GCL.

The recognized character string LMK is a character string obtained based on the result of the character recognition process performed on the character string image GCL included in the application form image GM in the character recognition device 10. Therefore, each of the multiple characters included in the recognized character string LMK is a confirmed character determined by the recognition result determination unit 126 of the character recognition device 10.

In addition, a caution mark MK may be displayed on the content confirmation screen CCH. Here, the caution mark MK is an image that is displayed on the content confirmation screen CCH when the character recognition device 10 determines that the character shown by the individual character image GC and the confirmed character may be different characters. In FIG. 7, a case is assumed in which the confirmed character ("sea") corresponding to the recognition result of the individual character image GC1 included in the character string image GCL5 is not recognized by the first priority character recognition unit CR4 but is recognized by the character recognition unit CR3. Therefore, in FIG. 7, a case is assumed in which the character recognition device 10 determines that the character shown by the individual character image GC1 and the confirmed character ("sea") corresponding to the recognition result of the individual character image GC1 may be different characters. In this case, for example, the caution mark MK is displayed at a position corresponding to the display position of the confirmed character corresponding to the recognition result of the individual character image GC1.

In FIG. 7, it is assumed that the candidate character ("umi") correctly recognized by the character recognition unit CR3 is determined as the confirmed character by the recognition result determination unit 126. That is, the character recognition device 10 has determined that the character ("umi") indicated by the individual character image GC1 and the confirmed character ("umi") corresponding to the recognition result of the individual character image GC1 may be different characters, but ultimately the correct candidate character ("umi") is determined as the confirmed character.

In the example shown in FIG. 6, the confirmed character ("Large") corresponding to the recognition result of the individual character image GC2 included in the character string image GCL5 is recognized by the first priority character recognition unit CR4. In this case, a warning mark MK indicating that the character ("Large") indicated by the individual character image GC2 included in the character string image GCL5 and the confirmed character ("Large") corresponding to the recognition result of the individual character image GC2 may be different characters is not displayed on the content confirmation screen CCH. In other words, the display mode of the confirmed character when the confirmed character is not recognized by the first priority character recognition unit CR is different from the display mode of the confirmed character when the confirmed character is recognized by the first priority character recognition unit CR.

By visually checking the attention mark MK, the operator operating the terminal device 20 can recognize that the character indicated by the individual character image GC and the confirmed character may be different characters. As a result, in this embodiment, the possibility of overlooking that the confirmed character is incorrect can be reduced compared to a configuration in which the attention mark MK is not displayed.

The correction button SSB is a GUI (Graphical User Interface) for making the confirmed character available for correction. For example, an operator operating the terminal device 20 compares an individual character image GC with the confirmed character corresponding to the individual character image GC on the content confirmation screen CCH. Then, when the operator operating the terminal device 20 determines that the confirmed character is different from the character shown by the individual character image GC, he or she selects (e.g., presses) the correction button SSB.

The Confirm button KTB is a GUI for confirming the multiple recognized character strings LMK displayed on the content confirmation screen CCH. For example, when the Confirm button KTB is selected, each confirmed character included in each recognized character string LMK is confirmed, and the state of the content confirmation screen CCH becomes a state in which the operator operating the terminal device 20 cannot change each confirmed character. As a result, the character recognition system 1 confirms the recognition results of the characters included in the application form image GM as the multiple confirmed characters displayed on the content confirmation screen CCH.

Note that examples of the content confirmation screen CCH are not limited to the example shown in FIG. 7. For example, when the pointer is placed over the attention mark MK displayed on the content confirmation screen CCH, a pop-up image including one or more correction candidate characters may be displayed. The one or more correction candidate characters may be, for example, a predetermined number of candidate characters selected from a plurality of candidate characters in descending order of evaluation value. Alternatively, when there are two or more candidate characters as the recognition result of a character by the first priority character recognition unit CR, the correction candidate characters may be two or more candidate characters obtained from the first priority character recognition unit CR.

Next, an overview of the operation of the character recognition device 10 will be described with reference to FIG.

Figure 8 is a flowchart showing an example of the operation of the character recognition device 10 shown in Figure 1. The operation shown in Figure 8 is executed by the processing device 12 functioning as the control unit 120 including the image information acquisition unit 122, the written content recognition unit 124, the recognition result determination unit 126, and the display information generation unit 128. For example, in step S100, the processing device 12 functions as the image information acquisition unit 122, and in steps S200, S220, S240, S260, and S400, the processing device 12 functions as the written content recognition unit 124. In addition, in step S300, the processing device 12 functions as the recognition result determination unit 126, and in step S500, the processing device 12 functions as the display information generation unit 128.

First, in step S100, the image information acquisition unit 122 acquires image information indicating the application form image GM from the terminal device 20.

Next, in step S200, the content recognition unit 124 identifies each of the multiple application content description areas AK included in the application form image GM indicated by the image information acquired in step S100.

Next, in step S220, the description content recognition unit 124 selects one application content description area AK from the multiple application content description areas AK identified in step S200. For example, the description content recognition unit 124 selects one application content description area AK for which character recognition processing has not been performed from the multiple application content description areas AK.

Next, in step S240, the description content recognition unit 124 selects two or more character recognition units CR that perform character recognition processing on the character string image GCL displayed in the application content description area AK selected in step S220. For example, the description content recognition unit 124 refers to a record of an item corresponding to the application content description area AK selected in step S220 among multiple records in the priority table PTBL, and selects a character recognition unit CR based on the priority stored in the referenced record. Note that in the explanation from step S240 onwards, unless otherwise specified, "character string image GCL" means the character string image GCL displayed in the application content description area AK selected in step S220.

Next, in step S260, the description recognition unit 124 causes the two or more character recognition units CR selected in step S240 to perform character recognition processing on the character string image GCL. That is, each of the two or more character recognition units CR selected in step S240 performs character recognition processing on the character string image GCL.

Next, in step S300, the recognition result determination unit 126 executes a confirmed character determination process to determine a confirmed character corresponding to each of the recognition results of the multiple individual character images GC included in the character string image GCL based on the result of the character recognition process in step S260. For example, as described in FIG. 6, the recognition result determination unit 126 determines a confirmed character corresponding to each of the recognition results of the multiple individual character images GC based on the accuracy of each of the multiple candidate characters and the priority of each of the multiple character recognition units CR.

Next, in step S400, the description content recognition unit 124 determines whether or not the character recognition process in step S260 has been performed on all of the multiple application content description areas AK identified in step S200. If the result of the determination in step S400 is positive, the processing device 12 advances the process to step S500. On the other hand, if the result of the determination in step S400 is negative, the processing device 12 returns the process to step S220.

In step S500, the display information generating unit 128 generates display information for displaying the confirmed characters determined by the recognition result determining unit 126 on the display device 28. For example, the display information generating unit 128 generates display information for displaying a content confirmation screen CCH on the display device 28, and outputs the generated display information to the terminal device 20. The operation shown in FIG. 8 is terminated by executing the process of step S500.

The operation of the character recognition device 10 is not limited to the example shown in FIG. 8. For example, the process of step S300 may be executed after the process of step S400, or may be executed in parallel with the series of processes of steps S200, S240, S260, and S400. In addition, for example, the process of step S400 may be executed by a functional block (e.g., the recognition result determination unit 126, etc.) other than the written content recognition unit 124.

Also, for example, in a part of step S240 that is repeated multiple times, the written content recognition unit 124 may select only one character recognition unit CR that performs character recognition processing on the character string image GCL. Also, for example, an upper limit on the number of character recognition units CR selected in step S240 may be determined in advance. In this case, the upper limit on the number of character recognition units CR selected in step S240 may be, for example, two. When the upper limit on the number of character recognition units CR selected in step S240 is two, for example, it is preferable that one or both of the character recognition units CR3 and CR4 can recognize multiple types of characters (six types in the embodiment).

Also, all of the multiple character recognition units CR may be used to recognize characters in all of the multiple items. In this case, a priority is determined for all of the multiple character recognition units CR in each item. Also, when all of the multiple character recognition units CR are used to recognize characters in all of the multiple items, for example, the processing of step S240 is omitted.

Next, the flow of the finalized character determination process will be explained with reference to Figure 9.

Figure 9 is a flow chart showing an example of the confirmed character determination process shown in Figure 8. The process shown in Figure 9 is an example of the confirmed character determination process of step S300 shown in Figure 8. In the confirmed character determination process shown in Figure 9, it is assumed that the confirmed characters are determined based on an evaluation value calculated using a weight value based on priority. The process of step S320 is executed, for example, after the process of step S260 in Figure 8 is executed. Note that in Figure 9, unless otherwise specified, "character string image GCL" refers to the character string image GCL displayed in the application content entry area AK selected in step S220 in Figure 8.

First, in step S320, the recognition result determination unit 126 identifies one individual character image GC for which a confirmed character has not been determined, from among the multiple individual character images GC contained in the character string image GCL.

Next, in step S340, the recognition result determination unit 126 calculates an evaluation value for each of the multiple candidate characters corresponding to the individual character image GC identified in step S320. Note that the multiple candidate characters corresponding to the individual character image GC are, for example, multiple candidate characters obtained as the recognition result of the individual character image GC by two or more character recognition units CR selected in step S240 of FIG. 8. Also, the evaluation value for each of the multiple candidate characters is expressed, for example, as the product of the accuracy of the candidate character and the weight value of the character recognition unit CR that recognized the candidate character.

Next, in step S360, the recognition result determination unit 126 determines the candidate character with the highest evaluation value calculated in step S340 from among the multiple candidate characters corresponding to one individual character image GC as the confirmed character corresponding to the recognition result of the one individual character image GC.

Next, in step S380, the recognition result determination unit 126 determines whether or not a confirmed character has been determined for all of the multiple individual character images GC included in the character string image GCL. If the result of the determination in step S380 is negative, the processing device 12 returns the process to step S320. On the other hand, if the result of the determination in step S380 is positive, the processing device 12 ends the confirmed character determination process and proceeds to step S400 in FIG. 8.

Next, we will explain another example of the finalized character determination process with reference to Figure 10.

Figure 10 is a flowchart showing another example of the confirmed character determination process shown in Figure 8. The process shown in Figure 10 is another example of the confirmed character determination process of step S300 shown in Figure 8. Elements similar to the operation described in Figure 9 are given the same reference numerals, and detailed description is omitted. In the confirmed character determination process shown in Figure 10, a case is assumed in which a weight value based on priority is not used. For example, in the confirmed character determination process shown in Figure 10, whether or not the first priority character recognition unit CR is prioritized over other character recognition units CR is determined depending on whether the accuracy of the first priority candidate character obtained as a recognition result of one character by the first priority character recognition unit CR is a predetermined value or more. The process of step S320 is executed, for example, after the process of step S260 in Figure 8 is executed. Note that, in Figure 10, unless otherwise specified, "character string image GCL" means the character string image GCL displayed in the application content description area AK selected in step S220 in Figure 8.

First, in step S320, the recognition result determination unit 126 identifies one individual character image GC for which a confirmed character has not been determined, from among the multiple individual character images GC contained in the character string image GCL.

Next, in step S342, the recognition result determination unit 126 determines whether the accuracy of the first priority candidate character obtained as the recognition result of one character by the first priority character recognition unit CR is equal to or greater than a predetermined value. If the result of the determination in step S342 is positive, the processing device 12 advances the process to step S362. On the other hand, if the result of the determination in step S342 is negative, the processing device 12 advances the process to step S364.

In step S362, the recognition result determination unit 126 determines the candidate character with the first priority as the confirmed character corresponding to the recognition result of one individual character image GC. After executing the process of step S362, the processing device 12 advances the process to step S380.

In step S364, the recognition result determination unit 126 determines the candidate character with the highest degree of certainty among the multiple candidate characters corresponding to one individual character image GC as the confirmed character corresponding to the recognition result of the one individual character image GC. After executing the process of step S364, the processing device 12 proceeds to the process of step S380.

In step S380, the recognition result determination unit 126 determines whether or not a confirmed character has been determined for all of the multiple individual character images GC included in the character string image GCL. If the result of the determination in step S380 is negative, the processing device 12 returns the process to step S320. On the other hand, if the result of the determination in step S380 is positive, the processing device 12 ends the confirmed character determination process and proceeds to step S400 in FIG. 8.

As described above, in this embodiment, the character recognition device 10 has a description content recognition unit 124 and a recognition result determination unit 126. The description content recognition unit 124 recognizes a character included in an image and recognizes a character included in an application form image GM, which is an image of a document including one or more items, by each of a plurality of character recognition processes (character recognition processes executed by each of a plurality of character recognition units CR) that recognize characters included in the image and calculate the accuracy of the recognized character. The priority of the character recognition process used for character recognition is determined for each item. The recognition result determination unit 126 determines one of a plurality of candidate characters obtained as a recognition result of a single character by each of the plurality of character recognition processes as a confirmed character corresponding to the recognition result of the single character. Specifically, the recognition result determination unit 126 determines one of a plurality of candidate characters as a confirmed character based on the accuracy of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes.

In this manner, in this embodiment, the confirmed character corresponding to one character is determined based on the results of multiple character recognition processes. Therefore, in this embodiment, the accuracy of character recognition can be improved compared to an aspect in which the confirmed character corresponding to one character is determined based on the result of one character recognition process. Furthermore, in this embodiment, the confirmed character is determined based on the accuracy of each of the multiple candidate characters and the priority of each of the multiple character recognition processes, so the accuracy of character recognition can be improved compared to an aspect in which the confirmed character is determined based only on the accuracy of each of the multiple candidate characters.

In addition, in this embodiment, a weight value based on priority may be set for each of the multiple character recognition processes. In this case, the recognition result determination unit 126 calculates an evaluation value for each of the multiple candidate characters by weighting the accuracy of each of the multiple candidate characters with the weight value of each of the multiple character recognition processes. Then, the recognition result determination unit 126 determines the candidate character with the highest evaluation value among the multiple candidate characters as the confirmed character. In this case, by appropriately setting the weight value, it is possible to prevent an incorrect character from being determined as the confirmed character. For example, in this embodiment, the accuracy of character recognition can be improved by determining a weight value for each item according to the learning degree of each of the multiple character recognition units CR.

In addition, in this embodiment, a weight value based on priority may not be determined for each of the multiple character recognition processes. In this case, when the accuracy of a first priority candidate character obtained as a recognition result of a character by a character recognition process with the highest priority among the multiple character recognition processes is equal to or greater than a predetermined value, the recognition result determination unit 126 may determine the first priority candidate character among the multiple candidate characters as the confirmed character. This makes it possible to prevent an incorrect character from being determined as the confirmed character in this embodiment, and improves the accuracy of character recognition.

In addition, when the accuracy of a first-priority candidate character obtained as a recognition result of a character by a character recognition process with the highest priority among the multiple character recognition processes is less than a predetermined value, the recognition result determination unit 126 may determine the candidate character with the highest accuracy among the multiple candidate characters as the confirmed character. Even in this case, it is possible to prevent an incorrect character from being determined as the confirmed character, and the accuracy of character recognition can be improved.

In addition, this embodiment further includes a display information generating unit 128 that displays the confirmed character determined by the recognition result determining unit 126 on the display device 28. The display information generating unit 128 changes the display mode of the confirmed character when the confirmed character is not recognized in the character recognition process having the highest priority among the multiple character recognition processes from the display mode of the confirmed character when the confirmed character is recognized in the character recognition process having the highest priority among the multiple character recognition processes. Therefore, for example, when the display mode of one confirmed character differs from the display mode of other confirmed characters, the operator can recognize that one confirmed character may be incorrect. As a result, in this embodiment, the possibility of overlooking that the confirmed character is incorrect can be reduced compared to when the display mode of the confirmed character does not change even when the confirmed character may be incorrect.

[2. Second embodiment]
Next, an example of an overview of the character recognition device 10 according to the second embodiment will be described with reference to FIG.

Figure 11 is an explanatory diagram for explaining an overview of a character recognition device 10 according to the second embodiment. Elements similar to those explained in Figures 1 to 10 are given the same reference numerals, and detailed explanations are omitted.

The character recognition device 10 shown in FIG. 11 has the same configuration as the character recognition device 10 shown in FIG. 3. For example, the character recognition device 10 has a processing device 12, a storage device 14, and a communication device 16. However, the control program PG2 stored in the storage device 14 differs from the control program PG1 shown in FIG. 3. Note that the control program PG2 is an example of a "program." In this embodiment, the control program PG2 includes, for example, an application program that causes the character recognition device 10 to execute character recognition processing. However, the control program PG2 may include, for example, an operating system program that causes the processing device 12 to control each part of the character recognition device 10.

In addition, in this embodiment, it is assumed that the priority is stored in the priority table PTBL as information indicating the character recognition unit CR used to recognize the characters written in the entry field of each item (more specifically, the application content entry area AK). Furthermore, in this embodiment, it is assumed that the number of character recognition units CR used to recognize the characters written in the entry field of each item is one or more and two or less. In at least one item of the multiple items, two character recognition units CR are used to recognize the characters. The priorities of the two character recognition units CR may be the same or different. In this embodiment, it is assumed that the priorities of the two character recognition units CR used to recognize the characters are the same. In each item, the character recognition process performed by one of the two character recognition units CR used to recognize the characters is an example of a "first character recognition process", and the character recognition process performed by the other of the two character recognition units CR is an example of a "second character recognition process".

The processing device 12 shown in FIG. 11 is similar to the processing device 12 shown in FIG. 3, except that the processing device 12 executes a control program PG2 instead of the control program PG1 shown in FIG. 3. For example, the processing device 12 functions as a control unit 120A by executing the control program PG2 read from the storage device 14.

The control unit 120A is similar to the control unit 120 shown in FIG. 4, except that it has a recognition result determination unit 126A instead of the recognition result determination unit 126. For example, the control unit 120A has an image information acquisition unit 122, a description content recognition unit 124, a recognition result determination unit 126A, and a display information generation unit 128. The recognition result determination unit 126A is an example of a "determination unit."

The image information acquisition unit 122 and the written content recognition unit 124 are the same as the image information acquisition unit 122 and the written content recognition unit 124 shown in FIG. 4. However, each of the multiple character recognition units CR of the written content recognition unit 124 outputs one or more candidate characters and the accuracy of each of the one or more candidate characters as the recognition result of one character.

The recognition result determination unit 126A acquires recognition information indicating the result of character recognition processing for one character by the written content recognition unit 124. For example, the recognition information indicates the recognition result of one character by character recognition processing performed by each of two character recognition units CR out of the multiple character recognition units CR, or the recognition result of one character by character recognition processing performed by one character recognition unit CR out of the multiple character recognition units CR. The recognition result of one character by character recognition processing performed by one character recognition unit CR out of the two character recognition units CR or one character recognition unit CR includes, for example, one or more candidate characters that are candidates for the recognition result of the one character, and the accuracy of the one or more candidate characters.

Note that one or more candidate characters obtained as a recognition result of a character by the character recognition process of one of the two character recognition units CR are an example of "first candidate characters," and one or more candidate characters obtained as a recognition result of a character by the character recognition process of the other of the two character recognition units CR are an example of "second candidate characters." In the following, a candidate character obtained as a recognition result of one of the two character recognition units CR that performed character recognition processing on a character is also referred to as a first candidate character, and a candidate character obtained as a recognition result of the other of the two character recognition units CR is also referred to as a second candidate character.

For example, when a common character is present that is included in both one or more first candidate characters and one or more second candidate characters, the recognition result determination unit 126A determines whether or not to make the common character a confirmed character corresponding to the recognition result of a character. The confirmed character determination process, which includes the process of determining whether or not to make the common character a confirmed character corresponding to the recognition result of a character, is described later in FIG. 12.

The display information generating unit 128 is similar to the display information generating unit 128 shown in FIG. 4, except for the conditions for changing the display mode of the confirmed character. For example, the display information generating unit 128 changes the display mode of the confirmed character when there is no common character from the display mode of the confirmed character when there is a common character. Specifically, for example, when there is no common character among the multiple candidate characters obtained as the recognition result of one character, the display information generating unit 128 generates display information for displaying a caution mark MK in the content confirmation screen CCH, and outputs the generated display information to the terminal device 20. As a result, for example, the caution mark MK is displayed in a position on the content confirmation screen CCH corresponding to the display position of the confirmed character corresponding to the recognition result of one character.

For example, when the accuracy of character recognition for one character by the two character recognition units CR is low, it is considered that there is a higher probability that no common characters exist compared to when the accuracy of character recognition for one character by the two character recognition units CR is high. Therefore, when there is no common character among the multiple candidate characters obtained as the recognition result of one character by the two character recognition units CR, it is considered that there is a higher probability that an incorrect character will be determined as the confirmed character compared to when a common character exists. For this reason, by visually checking the attention mark MK, the operator operating the terminal device 20 can recognize that the character indicated by the individual character image GC and the confirmed character may be different characters. As a result, in this embodiment, it is possible to reduce the possibility of overlooking that the confirmed character is incorrect compared to an aspect in which the attention mark MK is not displayed.

The operation of the control unit 120A is the same as the operation of the control unit 120 shown in FIG. 4, except for the confirmed character determination process of step S300 shown in FIG. 8. For example, the operation of the control unit 120A is the same as the operation shown in FIG. 8, except for the confirmed character determination process of step S300 shown in FIG.

Next, an example of the confirmed character determination process performed by the recognition result determination unit 126A will be described with reference to FIG. 12.

Figure 12 is an explanatory diagram for explaining an example of the operation of the recognition result determination unit 126A shown in Figure 11. As with Figure 6 described above, Figure 12 explains an example of a method for determining a confirmed character corresponding to the recognition result of individual character image GC2 of "large", using as an example a case where character recognition processing is performed on character string image GCL5 showing a character string written in application content entry area AK related to "address". However, in Figure 12, in order to make the explanation easier to understand, there are some parts where the accuracy, etc. are different from the example shown in Figure 6.

In FIG. 12, it is assumed that character recognition units CR3 and CR4 are selected as the character recognition units CR used to recognize characters in character string image GCL5. In this case, the character recognition process performed by one of character recognition units CR3 and CR4 corresponds to the "first character recognition process," and the character recognition process performed by the other of character recognition units CR3 and CR4 corresponds to the "second character recognition process." Hereinafter, the candidate characters obtained as the recognition results of character recognition unit CR3 are also referred to as first candidate characters, and the candidate characters obtained as the recognition results of character recognition unit CR4 are also referred to as second candidate characters.

In the example shown in FIG. 12, the recognition result determination unit 126A obtains a plurality of first candidate characters including the characters "丈", "太", "大", "火", and "木" as a result of the character recognition process performed by the character recognition unit CR3 on the individual character image GC2 of "大" and the accuracy of each of the plurality of first candidate characters. Furthermore, the recognition result determination unit 126A obtains a plurality of second candidate characters including the characters "犬", "大", "太", "木", and "丈" as a result of the character recognition process performed by the character recognition unit CR4 on the individual character image GC2 of "大".

The recognition result determination unit 126A searches for common characters that exist in both the first candidate characters and the second candidate characters from among multiple candidate characters including multiple first candidate characters and multiple second candidate characters. In FIG. 12, it is assumed that the characters "丈", "太", "大" and "木" exist in both the first candidate characters and the second candidate characters. Therefore, the recognition result determination unit 126A recognizes the characters "丈", "太", "大" and "木" as common characters.

Then, the recognition result determination unit 126A determines whether or not to set a common character as a confirmed character corresponding to a recognition result of one character, based on whether or not there are one or more common characters that satisfy both the first and second conditions. The first condition is a condition regarding the higher of the first accuracy, which is the accuracy of a common character included in one or more first candidate characters, and the second accuracy, which is the accuracy of a common character included in one or more second candidate characters. The second condition is a condition regarding the degree of difference between the first accuracy and the second accuracy. Specifically, the first condition is, for example, a condition that the higher of the first accuracy and the second accuracy is equal to or higher than a predetermined accuracy, and the second condition is a condition that the absolute value of the difference between the first accuracy and the second accuracy is equal to or lower than a predetermined value.

In FIG. 12, it is assumed that the first condition is that the higher of the first and second probabilities is 80% or more, and the second condition is that the absolute value of the difference between the first and second probabilities is 20% or less. In this case, among the multiple common characters ("丈", "太", "大" and "木"), the character "丈" does not satisfy the second condition because the difference between the first and second probabilities is 40%. The character "太" satisfies both the first and second conditions because the higher of the first and second probabilities is 85% and the difference between the first and second probabilities is 5%. The character "大" satisfies both the first and second conditions because the higher of the first and second probabilities is 87% and the difference between the first and second probabilities is 5%. In addition, the character "木" does not satisfy the first condition because the higher of the first and second probabilities is 60%.

Therefore, the recognition result determination unit 126A identifies the characters "太" and "大" from among the multiple common characters ("丈", "太", "大" and "木") as common characters that satisfy both the first and second conditions. For example, when a common character satisfies both the first and second conditions, the recognition result determination unit 126A determines the common character that satisfies both the first and second conditions as a confirmed candidate character that is a candidate for the confirmed character corresponding to the recognition result of one character. In other words, the recognition result determination unit 126A determines the characters "太" and "大" as confirmed candidate characters. When there are multiple confirmed candidate characters, the recognition result determination unit 126A determines one of the multiple confirmed candidate characters as the confirmed character.

When there is one confirmed candidate character, the recognition result determination unit 126A determines the confirmed candidate character as the confirmed character. Below, the first determination method, the second determination method, and the third determination method are described as examples of methods for determining a confirmed character when there are multiple confirmed candidate characters. However, the methods for determining a confirmed character when there are multiple confirmed candidate characters are not limited to the first determination method, the second determination method, and the third determination method described below.

In the first determination method, the recognition result determination unit 126A determines the confirmed candidate character with the highest certainty among multiple confirmed candidate characters as the confirmed character. In the recognition result shown in FIG. 12, of the two confirmed candidate characters ("bold" and "large"), the character "large" has the highest certainty. Therefore, in the first determination method, the recognition result determination unit 126A determines the confirmed candidate character with the highest certainty ("large") among the two confirmed candidate characters ("bold" and "large") as the confirmed character. In the first determination method, the confirmed character can be determined without performing calculations such as calculating the average value of the first certainty and the second certainty, which makes it possible to prevent the confirmed character determination process from becoming complicated.

In the second determination method, the recognition result determination unit 126A determines, as the determined character, the determined candidate character with the highest average value of the first and second probabilities among the multiple determined candidate characters. The average value of the probabilities of the character "bold" is 82.5% (= (85% + 80%)/2), and the average value of the probabilities of "large" is 84.5% (= (82% + 87%)/2). Therefore, in the second determination method, the recognition result determination unit 126A determines, as the determined character, the determined candidate character ("large") with the highest average probabilities among the two determined candidate characters ("bold" and "large"). In the second determination method, the determined character can be determined by a simple calculation (calculation of the average value of the first and second probabilities) without using a complex calculation, and therefore the determined character determination process can be prevented from becoming complicated.

In the third determination method, the recognition result determination unit 126A determines the confirmed candidate character with the highest evaluation value among multiple confirmed candidate characters as the confirmed character. For example, if the evaluation value is Ev, the higher of the first and second accuracies is ACmax, and the absolute value of the difference between the first and second accuracies is ACdiff, the evaluation value is expressed by the following formula (1) using coefficients α and β. Note that "*" in formula (1) indicates multiplication. The coefficients α and β are real numbers greater than 0.
Ev=ACmax*α−ACdiff*β (1)

In this way, in the third determination method, the confirmed character is determined using an evaluation value (Ev) based on the higher of the first and second accuracies (ACmax) and the absolute value of the difference between the first and second accuracies (ACdiff). Therefore, in the third determination method, the confirmed character can be determined with higher accuracy than when the confirmed character is determined without using the evaluation value (Ev) expressed by the above-mentioned formula (1).

The relationship between the evaluation value (Ev) and the first and second accuracies is not limited to the above formula (1). For example, the evaluation value (Ev) may be calculated using the average value of the first and second accuracies instead of the higher of the first and second accuracies (Acmax) in the above formula (1). The evaluation value (Ev) may also be calculated using the ratio of the first and second accuracies instead of the absolute value of the difference between the first and second accuracies (ACdiff). The ratio of the first and second accuracies is, for example, a value obtained by dividing the higher of the first and second accuracies by the lower of the first and second accuracies. When the value obtained by dividing the lower of the first and second accuracies by the higher of the first and second accuracies is used as the ratio of the first and second accuracies, the coefficient β in the above formula (1) becomes a negative real number.

Here, in the example shown in FIG. 12, of the multiple common characters, the common character with the highest accuracy is the character "丈", which has an accuracy of 90%. The accuracy of the character "丈" is 90%, but in character recognition unit CR4, the accuracy of the character "丈" is 50%. Therefore, the character "丈" satisfies the first condition related to the higher of the first and second accuracy, but does not satisfy the second condition related to the degree of difference between the first and second accuracy. When a common character that satisfies only one of the first and second conditions is determined as the confirmed character, it is considered that there is a higher probability that an incorrect character will be determined as the confirmed character than when a common character that satisfies both the first and second conditions is determined as the confirmed character.

In this embodiment, by determining a common character that satisfies both the first and second conditions as the confirmed character, it is possible to prevent an incorrect character from being determined as the confirmed character, compared to a case in which a common character that does not satisfy either or both of the first and second conditions is determined as the confirmed character.

The method of determining the confirmed character is not limited to the above example. For example, the recognition result determination unit 126A may determine the common character with the highest accuracy among the multiple common characters as the confirmed character without considering either the first condition or the second condition. Alternatively, the recognition result determination unit 126A may determine the common character with the highest average value of the first accuracy and the second accuracy among the multiple common characters as the confirmed character without considering either the first condition or the second condition. Alternatively, the recognition result determination unit 126A may determine the common character with the highest evaluation value represented by the above formula (1) among the multiple common characters as the confirmed character without considering either the first condition or the second condition.

Even when a common character is determined as a confirmed character without considering either the first or second condition, the probability of an incorrect character being determined as a confirmed character is considered to be lower than in a case where a candidate character with the highest probability among multiple candidate characters is unconditionally determined as a confirmed character. Hereinafter, the case where a candidate character with the highest probability among multiple candidate characters is unconditionally determined as a confirmed character is also referred to as the third comparison.

For example, in the recognition result shown in FIG. 12, the candidate character with the highest accuracy among the multiple candidate characters is the character "犬" with an accuracy of 92%. The character "犬" is included in the multiple second candidate characters obtained as the recognition result of one character by the character recognition unit CR4, but is not included in the multiple first candidate characters obtained as the recognition result of one character by the character recognition unit CR3. Therefore, in the third comparison example, an incorrect character ("犬") is determined as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大". Note that the correct candidate character ("大") as the confirmed character corresponding to the recognition result of the individual character image GC2 of "大" is included in the multiple common characters. Therefore, even if one of the multiple common characters is determined as the confirmed character without considering either the first or second condition, it is considered that the probability of an incorrect character being determined as the confirmed character is lower than in the third comparison example. That is, in this embodiment, even if neither the first nor the second condition is taken into account in determining the confirmed character, by determining one of one or more common characters as the confirmed character, it is possible to prevent an incorrect character from being determined as the confirmed character, compared to the third comparison example.

Next, the flow of the confirmed character determination process performed by the recognition result determination unit 126A will be described with reference to FIG. 13.

Fig. 13 is a flowchart showing an example of the operation of the recognition result determination unit 126A shown in Fig. 11. In the confirmed character determination process shown in Fig. 13, it is assumed that a first condition related to the higher of the first and second accuracies, and a second condition related to the degree of difference between the first and second accuracies, are used to determine the confirmed character.

The confirmed character determination process shown in FIG. 13 is executed, for example, as the confirmed character determination process of step S300 in the operation shown in FIG. 8. For example, the process of step S320 is executed after the process of step S260 in FIG. 8 is executed. Elements similar to those in the operation described in FIG. 9 and FIG. 10 are given the same reference numerals and detailed description is omitted.

In FIG. 13, unless otherwise specified, "character string image GCL" refers to the character string image GCL displayed in the application content entry area AK selected in step S220 of FIG. 8. Also, FIG. 13 assumes that two character recognition units CR were selected in step S240 of FIG. 8. Note that if one character recognition unit CR is selected in step S240 of FIG. 8, the series of processes from step S350 to step S372 is omitted, and the series of processes from step S320, S374, and S380 is executed.

First, in step S320, the recognition result determination unit 126A identifies one individual character image GC for which a confirmed character has not been determined, from among the multiple individual character images GC contained in the character string image GCL.

Next, in step S350, the recognition result determination unit 126A searches for a common character from among multiple candidate characters corresponding to one individual character image GC.

Next, in step S352, the recognition result determination unit 126A determines whether or not a common character exists. That is, the recognition result determination unit 126A determines whether or not a common character has been searched for among multiple candidate characters. If the result of the determination in step S352 is positive, the processing device 12 advances the process to step S354. On the other hand, if the result of the determination in step S352 is negative, the processing device 12 advances the process to step S374.

In step S354, the recognition result determination unit 126A determines whether or not a common character that satisfies the first condition exists. If the result of the determination in step S354 is positive, the processing device 12 proceeds to step S356. On the other hand, if the result of the determination in step S354 is negative, the processing device 12 proceeds to step S374.

In step S356, the recognition result determination unit 126A determines whether or not a common character that satisfies the second condition is present. If the result of the determination in step S356 is positive, the processing device 12 advances the process to step S370. On the other hand, if the result of the determination in step S356 is negative, the processing device 12 advances the process to step S372.

In step S370, the recognition result determination unit 126A determines one of the common characters (i.e., the confirmed candidate characters) that satisfies both the first and second conditions as the confirmed character. Note that, for example, if there are multiple confirmed candidate characters, the recognition result determination unit 126A may determine one of the multiple confirmed candidate characters as the confirmed character by any of the first determination method, the second determination method, and the third determination method described in FIG. 12 above. After executing the process of step S370, the processing device 12 advances the process to step S380.

In step S372, the recognition result determination unit 126A determines one of the common characters that satisfy the first condition as the confirmed character. For example, if there are multiple common characters that satisfy the first condition, the recognition result determination unit 126A determines the common character with the highest probability among the multiple common characters that satisfy the first condition as the confirmed character. Alternatively, the recognition result determination unit 126A may determine the common character with the highest evaluation value represented by the above-mentioned formula (1) among the multiple common characters that satisfy the first condition as the confirmed character. After executing the process of step S372, the processing device 12 advances the process to step S380.

In step S374, the recognition result determination unit 126A determines one of the multiple candidate characters as the confirmed character. For example, the recognition result determination unit 126A determines the candidate character with the highest degree of certainty among the multiple candidate characters as the confirmed character. After executing the process of step S374, the processing device 12 advances the process to step S380.

In step S380, the recognition result determination unit 126A determines whether or not a confirmed character has been determined for all of the multiple individual character images GC included in the character string image GCL. If the result of the determination in step S380 is negative, the processing device 12 returns the process to step S320. On the other hand, if the result of the determination in step S380 is positive, the processing device 12 ends the confirmed character determination process and proceeds to step S400 in FIG. 8.

The operation of the recognition result determination unit 126A is not limited to the example shown in FIG. 13. For example, instead of the determination in step S356, the recognition result determination unit 126A may determine whether or not a common character (i.e., a confirmed candidate character) that satisfies both the first and second conditions is present before the determination in step S354. In this case, the determination in step S354 is performed when a confirmed candidate character is not present, and the process in step S372 is performed when the result of the determination in step S354 is positive.

Also, for example, if the result of the determination in step S356 is negative, the recognition result determination unit 126A may execute the process of step S374. That is, the process of step S372 may be omitted. Also, for example, if the result of the determination in step S354 is negative, the recognition result determination unit 126A may determine one of the common characters as the confirmed character in step S374. For example, if the result of the determination in step S354 is negative, the recognition result determination unit 126A may determine the common character having the highest evaluation value represented by the above-mentioned formula (1) among the multiple common characters as the confirmed character.

As described above, in this embodiment, the character recognition device 10 has a description content recognition unit 124 and a recognition result determination unit 126A. The description content recognition unit 124 recognizes a character included in the application form image GM by each of a first character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character, and a second character recognition process that recognizes a character included in an image and calculates the accuracy of the recognized character (character recognition processes executed by each of the two character recognition units CR). When there is a common character included in both one or more first candidate characters obtained as a recognition result of a character by the first character recognition process and one or more second candidate characters obtained as a recognition result of a character by the second character recognition process, the recognition result determination unit 126A determines whether or not to set the common character as a confirmed character corresponding to the recognition result of the character.

Of the multiple candidate characters including the first candidate character and the second candidate character, the common character included in both the first candidate character and the second candidate character is a candidate character recognized in both the first character recognition process and the second character recognition process as a recognition result of one character. In other words, the candidate characters other than the common character are candidate characters recognized in only one of the first character recognition process and the second character recognition process as a recognition result of one character. Therefore, it is considered that the probability that a correct candidate character is included in the common characters as a recognition result of one character is higher than the probability that a correct candidate character is included in the candidate characters other than the common characters as a recognition result of one character. In this embodiment, when a common character exists, it is determined whether or not the common character is to be the confirmed character, so that it is possible to prevent an incorrect character from being determined as the confirmed character. In other words, in this embodiment, it is possible to improve the accuracy of character recognition in the character recognition process.

In addition, in this embodiment, the recognition result determination unit 126A may determine a common character as a confirmed candidate character that is a candidate for a confirmed character when the common character satisfies both the first condition and the second condition. The first condition is a condition related to the higher of the first accuracy, which is the accuracy of a common character included in one or more first candidate characters, and the second accuracy, which is the accuracy of a common character included in one or more second candidate characters. The second condition is a condition related to the degree of difference between the first accuracy and the second accuracy. It is considered that the probability that a correct candidate character is included in a common character that satisfies both the first condition and the second condition is higher than the probability that a correct candidate character is included in a common character that does not satisfy one or both of the first condition and the second condition. Therefore, in this embodiment, by determining a common character that satisfies both the first condition and the second condition as a confirmed candidate character, it is possible to suppress an incorrect character from being determined as a confirmed character compared to a form in which a common character is unconditionally determined as a confirmed character.

For example, in this embodiment, the first condition may be a condition that the higher of the first and second accuracies is equal to or higher than a predetermined accuracy, and the second condition may be a condition that the difference between the first and second accuracies is equal to or lower than a predetermined value. A common character that satisfies both the first and second conditions has a higher of the first and second accuracies equal to or higher than a predetermined accuracy, and the difference between the first and second accuracies is equal to or lower than a predetermined value. That is, a common character that satisfies both the first and second conditions has a higher of the higher of the first and second accuracies than a common character that does not satisfy the first condition, and has a smaller difference between the first and second accuracies than a common character that does not satisfy the second condition. Therefore, a common character that satisfies both the first and second conditions is considered to have a higher character recognition accuracy than a common character that does not satisfy one or both of the first and second conditions. Therefore, in this embodiment, by determining a common character that satisfies both the first and second conditions as a confirmed candidate character, it is possible to improve the accuracy of character recognition in the character recognition process.

In addition, in this embodiment, when there are multiple confirmed candidate characters, the recognition result determination unit 126A may determine the confirmed candidate character with the highest degree of certainty among the multiple confirmed candidate characters as the confirmed character. Even in this case, it is possible to prevent an incorrect character from being determined as the confirmed character, and the accuracy of character recognition in the character recognition process can be improved.

In addition, in this embodiment, when there are multiple confirmed candidate characters, the recognition result determination unit 126A may determine, as the confirmed character, the confirmed candidate character having the highest average value of the first probability and the second probability among the multiple confirmed candidate characters. Even in this case, it is possible to prevent an incorrect character from being determined as the confirmed character, and it is possible to improve the accuracy of character recognition in the character recognition process.

In addition, in this embodiment, the above-mentioned first and second conditions do not have to be set. In a form in which the first and second conditions are not set, when there are multiple common characters, the recognition result determination unit 126A may determine the common character with the highest probability among the multiple common characters as the confirmed character. Even in this case, it is possible to prevent an incorrect character from being determined as the confirmed character, compared to a case in which the candidate character with the highest probability among multiple candidate characters is unconditionally determined as the confirmed character, and the accuracy of character recognition in the character recognition process can be improved.

Alternatively, in a form in which the first condition and the second condition are not set, when there are multiple common characters, the recognition result determination unit 126A may determine, among the multiple common characters, the common character with the highest average value of the first probability and the second probability as the confirmed character. Even in this case, it is possible to prevent an incorrect character from being determined as the confirmed character, compared to a case in which the candidate character with the highest probability among multiple candidate characters is unconditionally determined as the confirmed character, and the accuracy of character recognition in the character recognition process can be improved.

In addition, in this embodiment, the display information generating unit 128 is further provided, which displays the confirmed character determined by the recognition result determining unit 126A on the display device 28. The display information generating unit 128 makes the display mode of the confirmed character when there is no common character different from the display mode of the confirmed character when there is a common character. When there is no common character, the accuracy of character recognition for one character by the two character recognition units CR is considered to be lower than when there is a common character. In other words, when there is no common character, the probability that an incorrect character is determined as the confirmed character is considered to be higher than when there is a common character. For this reason, for example, when the display mode of one confirmed character is different from the display mode of other confirmed characters, the operator can recognize that one confirmed character may be incorrect. As a result, in this embodiment, the possibility of overlooking that the confirmed character is incorrect can be reduced compared to when the display mode of the confirmed character does not change even when there is a possibility that the confirmed character is incorrect.

Furthermore, in this embodiment, a weight value may be associated with the character recognition unit CR for each item, as in the first embodiment described above.

[3. Third embodiment]
Next, an example of an overview of the character recognition device 10 according to the third embodiment will be described with reference to FIG.

Figure 14 is an explanatory diagram for explaining an overview of a character recognition device 10 according to the third embodiment. Elements similar to those explained in Figures 1 to 13 are given the same reference numerals and detailed explanations are omitted.

The character recognition device 10 shown in FIG. 14 has the same configuration as the character recognition device 10 shown in FIG. 11. For example, the character recognition device 10 has a processing device 12, a storage device 14, and a communication device 16. However, the control program PG3 stored in the storage device 14 differs from the control program PG2 shown in FIG. 11. Note that the control program PG3 is an example of a "program." In this embodiment, the control program PG3 includes, for example, an application program that causes the character recognition device 10 to execute character recognition processing. However, the control program PG3 may also include, for example, an operating system program that causes the processing device 12 to control each part of the character recognition device 10.

In addition, in this embodiment, the priority table PTBL and the upper limit of the number of character recognition units CR used to recognize characters written in the description column of each item are the same as those in the second embodiment described above. For example, in this embodiment, one to two character recognition units CR are used to recognize characters in each item, and two character recognition units CR are used to recognize characters in at least one item of the multiple items. Hereinafter, one of the two character recognition units CR used to recognize characters is also referred to as the first character recognition unit CR, and the other of the two character recognition units CR is also referred to as the second character recognition unit CR.

In this embodiment, it is assumed that the first character recognition unit CR recognizes multiple characters that make up a character string included in the application form image GM, and the second character recognition unit CR recognizes one character out of the multiple characters that make up the character string included in the application form image GM. The character recognition process performed by the first character recognition unit CR is an example of a "first character recognition process," and the character recognition process performed by the second character recognition unit CR is an example of a "second character recognition process." Additionally, the application form image GM is an example of an "image." The character string indicated by the character string image GCL included in the application form image GM corresponds to the character string included in the application form image GM.

The processing device 12 shown in FIG. 14 is similar to the processing device 12 shown in FIG. 11, except that the processing device 12 executes the control program PG3 instead of the control program PG2 shown in FIG. 11. For example, the processing device 12 functions as the control unit 120B by executing the control program PG3 read from the storage device 14.

The control unit 120B is similar to the control unit 120A shown in FIG. 11, except that it has a recognition result determination unit 126B instead of the recognition result determination unit 126A. For example, the control unit 120B has an image information acquisition unit 122, a description content recognition unit 124, a recognition result determination unit 126B, and a display information generation unit 128. The recognition result determination unit 126B is an example of a "determination unit."

The image information acquisition unit 122 and the description content recognition unit 124 are similar to the image information acquisition unit 122 and the description content recognition unit 124 shown in FIG. 11. However, the multiple character recognition units CR of the description content recognition unit 124 include a first character recognition unit CR and a second character recognition unit CR. Furthermore, the first character recognition unit CR outputs candidate characters and the accuracy of each candidate character for each of the multiple characters that make up the character string as a recognition result of the character string included in the application form image GM. The second character recognition unit CR outputs candidate characters and the accuracy of the candidate characters as a recognition result of one of the multiple characters.

A candidate character obtained as a result of the recognition of a character by the first character recognition unit CR is an example of a "first candidate character," and a candidate character obtained as a result of the recognition of a character by the second character recognition unit CR is an example of a "second candidate character." Hereinafter, a candidate character obtained as a result of the recognition of a character by the first character recognition unit CR is also referred to as a first candidate character, and a candidate character obtained as a result of the recognition of a character by the second character recognition unit CR is also referred to as a second candidate character.

The recognition result determination unit 126B acquires recognition information indicating the result of character recognition processing for a single character by the written content recognition unit 124. For example, the recognition information indicates the recognition result of a character string (a character string including the single character) by character recognition processing performed by the first character recognition unit CR, and the recognition result of the single character by character recognition processing performed by the second character recognition unit CR.

For example, when the first candidate character and the second candidate character are the same character, the recognition result determination unit 126B determines the same character as the confirmed character corresponding to the recognition result of one character.

The display information generating unit 128 is similar to the display information generating unit 128 shown in FIG. 11 except for the conditions for differentiating the display form of the confirmed character. For example, the display information generating unit 128 makes the display form of the confirmed character when the first candidate character and the second candidate character are different from the display form of the confirmed character when the first candidate character and the second candidate character are the same. Specifically, for example, when the first candidate character and the second candidate character are different from each other, the display information generating unit 128 generates display information for displaying a caution mark MK in the content confirmation screen CCH and outputs the generated display information to the terminal device 20. As a result, for example, the caution mark MK is displayed in a position on the content confirmation screen CCH corresponding to the display position of the confirmed character corresponding to the recognition result of one character.

For example, when the first candidate character and the second candidate character are different from each other, it is considered that there is a higher probability that an incorrect character will be determined as the confirmed character compared to when the first candidate character and the second candidate character are the same from each other. Therefore, by visually checking the attention mark MK, the operator operating the terminal device 20 can recognize that the character indicated by the individual character image GC and the confirmed character may be different characters. As a result, in this embodiment, it is possible to reduce the possibility of overlooking that the confirmed character is incorrect compared to an aspect in which the attention mark MK is not displayed.

In addition, the display information generating unit 128 may cause the display mode of the confirmed character when the first candidate character and the second candidate character are different from each other and the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more to be different from the display mode of the confirmed character when the first candidate character and the second candidate character are the same. For example, the display information generating unit 128 may generate display information that displays a caution mark MK in the content confirmation screen CCH when the first candidate character and the second candidate character are different from each other and the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more.

For example, even if the first candidate character and the second candidate character are different from each other, if the difference between the accuracy of the first candidate character and the accuracy of the second candidate character is less than a predetermined value, the confirmed character is considered to have a higher probability of being correct than when the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more. Therefore, for example, when the first candidate character and the second candidate character are different from each other and the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more, in a form in which a warning mark MK is displayed in the content confirmation screen CCH, the warning mark MK can be displayed effectively. Displaying the warning mark MK effectively means, for example, suppressing the warning mark MK from being displayed frequently when the confirmed character is correct.

In addition, the display information generating unit 128 may change the display mode of the confirmed character when the first candidate character and the second candidate character are different between a case where the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more and a case where the difference between the accuracy of the first candidate character and the accuracy of the second candidate character is less than a predetermined value. For example, the display information generating unit 128 may change the color of the attention mark MK between a case where the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more and a case where the difference between the accuracy of the first candidate character and the accuracy of the second candidate character is less than a predetermined value. In this case, for example, an operator operating the terminal device 20 can recognize the degree of possibility that the character indicated by the individual character image GC and the confirmed character are different characters by visually checking the color of the attention mark MK.

The operation of control unit 120B is the same as that of control unit 120A shown in FIG. 11, except for the confirmed character determination process. For example, the operation of control unit 120B is the same as that shown in FIG. 8, except that the confirmed character determination process described in FIG. 15, which will be described later, is executed in step S300 shown in FIG. 8.

Next, an example of the confirmed character determination process performed by the recognition result determination unit 126B will be described with reference to FIG. 15.

Fig. 15 is an explanatory diagram for explaining an example of the operation of the recognition result determination unit 126B shown in Fig. 14. Fig. 15 illustrates an example of a method for determining confirmed characters corresponding to the recognition results of the individual character images GC of "east", "blue", and "sea", using as an example a case where character recognition processing is performed on a character string image GCL5 showing a character string written in the application content entry area AK related to "address".

In FIG. 15, it is assumed that character recognition units CR3 and CR4 are selected as the character recognition units CR used to recognize characters in character string image GCL5. Furthermore, in FIG. 15, it is assumed that character recognition unit CR4 is the first character recognition unit CR, and character recognition unit CR3 is the second character recognition unit CR. Also, in FIG. 15, it is assumed that character recognition unit CR4 uses a dictionary related to addresses (hereinafter also referred to as an address dictionary) to identify candidate character strings that are the recognition results of character strings that include a certain character.

In FIG. 15, it is assumed that the two first candidate characters corresponding to the recognition result of the individual character images GC of "東" and "青" by the character recognition unit CR4 are the characters "東" and "青", respectively. Also, in FIG. 15, it is assumed that the first candidate characters corresponding to the recognition result of the individual character image GC1 of "海" by the character recognition unit CR4 are the characters "海", "梅" and "突っ伏".

The character recognition unit CR4 judges whether a character string formed by a combination of two first candidate characters corresponding to the recognition results of the individual character images GC of "Higashi" and "Ao" and a first candidate character corresponding to the recognition result of the individual character image GC1 of "Umi" is registered in the address dictionary. An x mark in FIG. 15 indicates that the character string is not registered in the address dictionary, and a circle mark in FIG. 15 indicates that the character string is registered in the address dictionary. For example, the character recognition unit CR4 judges that the character strings "Higashi Aoumi" and "Higashi Aoumi" are not candidate character strings because the character strings "Higashi Aoumi" and "Higashi Aoumi" are not registered in the address dictionary. Furthermore, the character recognition unit CR4 determines that the character string "Higashi Aoumi" is a candidate character string because the character string "Higashi Aoume" is registered in the address dictionary. In other words, the character recognition unit CR4 determines that the character string "Ume" is the first candidate character corresponding to the recognition result of the individual character image GC1 of "Umi".

On the other hand, the character recognition unit CR3 determines the character "海" with the highest accuracy among the second candidate character candidates ("海", "梅" and "臭") corresponding to the recognition result of the individual character image GC1 of "海" as the second candidate character corresponding to the recognition result of the individual character image GC1 of "海". Note that in FIG. 15, it is assumed that the two second candidate characters corresponding to the recognition result of the individual character images GC of "東" and "青" by the character recognition unit CR3 are the characters "東" and "青", respectively.

The recognition result determination unit 126B obtains the first candidate character ("Higashi") and the accuracy (90%) of the first candidate character as a result of the character recognition process performed by the character recognition unit CR4 on the individual character image GC of "Higashi". Furthermore, the recognition result determination unit 126B obtains the second candidate character ("Higashi") and the accuracy (88%) of the second candidate character as a result of the character recognition process performed by the character recognition unit CR3 on the individual character image GC of "Higashi". Because the first candidate character ("Higashi") and the second candidate character ("Higashi") are the same, the recognition result determination unit 126B determines the character "Higashi" as the confirmed character corresponding to the recognition result of the individual character image GC of "Higashi". Similarly, because the first candidate character ("blue") and the second candidate character ("blue") obtained as a result of the character recognition process on the individual character image GC of "blue" are the same, the recognition result determination unit 126B determines the character "blue" as the confirmed character corresponding to the recognition result of the individual character image GC of "blue."

The recognition result determination unit 126B also obtains the first candidate character ("Ume") and the accuracy (30%) of the first candidate character as a result of the character recognition process performed by the character recognition unit CR4 on the individual character image GC1 of "Umi". The recognition result determination unit 126B also obtains the second candidate character ("Umi") and the accuracy (90%) of the second candidate character as a result of the character recognition process performed by the character recognition unit CR3 on the individual character image GC1 of "Umi".

Although the first candidate character ("ume") and the second candidate character ("umi") are different from each other, the recognition result determination unit 126B determines the first candidate character ("ume") as the confirmed character corresponding to the recognition result of the individual character image GC1 of "umi". However, since the first candidate character ("ume") and the second candidate character ("umi") are different from each other, the recognition result determination unit 126B notifies the display information generation unit 128 that the first candidate character and the second candidate character are different from each other. Then, the display information generation unit 128 generates display information for displaying a caution mark MK in the content confirmation screen CCH, for example, and outputs the generated display information to the terminal device 20. As a result, the caution mark MK is displayed in a position on the content confirmation screen CCH corresponding to the display position of the confirmed character corresponding to the recognition result of the individual character image GC1 of "umi".

Note that the operation of the recognition result determination unit 126B is not limited to the above example. For example, when the first candidate character ("ume") and the second candidate character ("umi") are different from each other, the recognition result determination unit 126B may determine the second candidate character ("umi"), which has a higher probability among the first and second candidate characters, as the confirmed character corresponding to the recognition result of the individual character image GC1 of "umi". In other words, when the first candidate character and the second candidate character are different from each other, the recognition result determination unit 126B may determine the first candidate character and the second candidate character, which has a higher probability, as the confirmed character. Even in this case, the recognition result determination unit 126B notifies the display information generation unit 128 that the first candidate character and the second candidate character are different from each other.

Alternatively, when the first candidate character and the second candidate character are different from each other, the recognition result determination unit 126B may determine one of the first candidate character and the second candidate character as the confirmed character based on a determination value that is a value obtained by subtracting the accuracy of the first candidate character from the accuracy of the second candidate character. Specifically, for example, the recognition result determination unit 126B may determine the first candidate character as the confirmed character when the determination value is less than a predetermined value, and may determine the second candidate character as the confirmed character when the determination value is equal to or greater than the predetermined value.

In addition, when the first candidate character and the second candidate character are different from each other and the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more, the recognition result determination unit 126B may notify the display information generation unit 128 that the first candidate character and the second candidate character are different from each other.

As described above, in this embodiment, the character recognition device 10 has a written content recognition unit 124 and a recognition result determination unit 126B. The written content recognition unit 124 executes a first character recognition process that recognizes multiple characters that make up a character string included in the application form image GM, and a second character recognition process that recognizes one of the multiple characters included in the application form image GM. If a first candidate character obtained as a recognition result of a single character by the first character recognition process and a second candidate character obtained as a recognition result of a single character by the second character recognition process are the same character, the recognition result determination unit 126B determines the same character as a confirmed character corresponding to the recognition result of the single character.

When the first candidate character and the second candidate character are the same, it is considered that the probability of an incorrect character being determined as the confirmed character is lower than when the first candidate character and the second candidate character are different. In this embodiment, when the first candidate character and the second candidate character are the same, the same character is determined as the confirmed character, so that it is possible to prevent an incorrect character from being determined as the confirmed character, and it is possible to improve the accuracy of character recognition in the character recognition process.

In addition, this embodiment further includes a display information generating unit 128 that displays the confirmed characters determined by the recognition result determining unit 126B on the display device 28. The display information generating unit 128 makes the display mode of the confirmed characters when the first and second candidate characters are different from the display mode of the confirmed characters when the first and second candidate characters are the same. For this reason, for example, when the display mode of one confirmed character is different from the display mode of other confirmed characters, the operator can recognize that one confirmed character may be incorrect. As a result, in this embodiment, the possibility of overlooking that a confirmed character is incorrect can be reduced compared to when the display mode of the confirmed character does not change even when there is a possibility that the confirmed character is incorrect.

In addition, in this embodiment, the display information generating unit 128 may change the display mode of the confirmed character when the first candidate character and the second candidate character are different and the accuracy of the first candidate character is lower than the accuracy of the second candidate character by a predetermined value or more from the display mode of the confirmed character when the first candidate character and the second candidate character are the same. Even in this case, it is possible to reduce the possibility of overlooking that the confirmed character is incorrect, compared to a case in which the display mode of the confirmed character does not change even when there is a possibility that the confirmed character is incorrect.

Furthermore, in this embodiment, a weight value may be associated with the character recognition unit CR for each item, as in the first embodiment described above.

[4. Modifications]
The present invention is not limited to the above-described embodiment. Specific modified aspects are exemplified below. Two or more aspects selected from the following examples may be combined.

[First Modification]
In the first embodiment described above, a weight value is associated with each item in the character recognition unit CR, but the present invention is not limited to this. For example, a weight value may be associated with each character in each character recognition unit CR.

Figure 16 is an explanatory diagram for explaining an example of a weight table WTNL according to the first modified example. In this modified example, it is assumed that the weight table WTNL is provided separately from the priority table PTBL. For example, the storage device 14 stores the control program PG1, the priority table PTBL, the weight table WTNL, and multiple learning models LM. For example, the weight table WTNL is referenced by the recognition result determination unit 126 when the recognition result determination unit 126 calculates an evaluation value. Note that when the written content recognition unit 124 selects a character recognition unit CR to be used for character recognition from multiple character recognition units CR, the written content recognition unit 124 references the priority table PTBL.

The weight table WTNL is managed for each character recognition unit CR, for example. The weight table WTNL of each character recognition unit CR is updated, for example, at a predetermined timing. For example, the weight table WTNL of each character recognition unit CR has a plurality of records that correspond one-to-one to one or more candidate characters that have been previously determined as confirmed characters and have been corrected to the correct character. That is, in the example shown in FIG. 16, candidate characters that have not previously been determined as confirmed characters and candidate characters that have not been corrected are not registered in the weight table WTNL. In this modified example, the weight value of characters that are not registered in the weight table WTNL (for example, candidate characters that have not previously been determined as confirmed characters and candidate characters that have not been corrected, etc.) is treated as 1.

In the weight table WTNL of each character recognition unit CR, each record has a character ID and a weight value. The character ID is information for identifying each character from among the multiple characters recognized by the character recognition unit CR, and is information indicating each character. For example, the character ID may be a character code assigned to each character. Note that, in FIG. 16, the characters indicated by the character IDs are also illustrated for ease of understanding.

The weight value is, for example, a weight value associated with a character indicated by a character ID. The weight value is a real number between 0 and 1. However, as described above, in the example shown in FIG. 16, a weight value of 1 is associated with a character that is not registered in the weight table WTNL, so the weight values associated with characters registered in the weight table WTNL are real numbers between 0 and 1.

For example, the weight value of a character that has been corrected many times is set to a smaller value than the weight value of a character that has been corrected few times. For example, if the character recognition unit CR3 erroneously recognizes the individual character image GC of "木" as the character "大" and the incorrect candidate character ("大") obtained from the character recognition unit CR3 is determined to be the confirmed character, the character "大" is corrected to the character "木", and this is counted in the number of corrections of the character "大". Also, even if the character recognition unit CR3 has previously recognized the individual character image GC of "木" as an incorrect character, if it has never previously recognized an individual character image GC other than "木" as the character "木", the number of corrections of the character "木" is 0.

Note that the weight table WTNL is not limited to the example shown in FIG. 16. For example, the weight table WTNL may store, in association with each character registered in the weight table WTNL, the number of times that the character has been determined as a confirmed character since the character was first registered in the weight table WTNL. In this case, the weight value may be set taking into consideration the ratio of the number of corrections to the number of times that the character has been determined as a confirmed character (hereinafter also referred to as the correction rate). For example, the weight value of a character with a high correction rate may be set to a value smaller than the weight value of a character with a low correction rate.

Alternatively, the weight table WTNL of each character recognition unit CR may have multiple records that correspond one-to-one with all characters that the character recognition unit CR can recognize. In this case, the weight table WTNL of each character recognition unit CR may store the number of times each character has been determined as a confirmed character. Even in this case, the weight value may be set taking into account the correction rate.

Next, with reference to FIG. 17, we will explain an example of a process for determining a confirmed character using the weight table WTNL.

Figure 17 is an explanatory diagram for explaining an example of the confirmed character determination process according to the first modified example. As with Figure 6 described above, Figure 17 explains an example of a method for determining confirmed characters corresponding to the recognition result of individual character image GC2 of "large", using as an example a case where character recognition process is performed on character string image GCL5 showing a character string written in application content entry area AK related to "address". However, in Figure 17, in order to make the explanation easier to understand, there are some parts where the accuracy etc. are different from the example shown in Figure 6.

In FIG. 17, it is assumed that the recognition result determination unit 126 refers to the weighting table WTNL shown in FIG. 16. Furthermore, in order to make the explanation easier to understand, FIG. 17 also assumes that the character recognition units CR3 and CR4 are selected as the character recognition units CR used to recognize the characters in the character string image GCL5.

In the example shown in FIG. 17, the recognition result determination unit 126 obtains multiple candidate characters including the characters "丈", "太", "大", "火", and "木" as a result of character recognition processing performed by the character recognition unit CR3 on the individual character image GC of "大" and the accuracy of each of the multiple candidate characters. Furthermore, the recognition result determination unit 126 obtains multiple candidate characters including the characters "犬", "大", "太", "木" and "丈" as a result of character recognition processing performed by the character recognition unit CR4 on the individual character image GC of "大".

The recognition result determination unit 126 calculates an evaluation value by weighting the accuracy of each of the multiple candidate characters by the accuracy weight value of each of the multiple candidate characters. For example, the evaluation value of each candidate character is expressed as the product of the accuracy of each candidate character and the weight value of each candidate character. Specifically, the recognition result determination unit 126 calculates the product (90*0.8=72) of the accuracy of the candidate character "丈" obtained from the character recognition unit CR3 and the weight value of the candidate character "丈" (0.8) as the evaluation value of the candidate character "丈". Evaluation values are calculated for each of the candidate characters other than "丈" among the multiple candidate characters obtained from the character recognition unit CR3 and for each of the multiple candidate characters obtained from the character recognition unit CR4 in the same way as for the candidate character "丈".

Then, the recognition result determination unit 126 determines the candidate character with the highest evaluation value among the multiple candidate characters obtained as the recognition result of the individual character image GC2 of "大" by each of the character recognition units CR3 and CR4 as the confirmed character corresponding to the recognition result of the individual character image GC of "大". Note that, when there are multiple candidate characters with the highest evaluation values, the recognition result determination unit 126 may determine the candidate character with the highest priority among the multiple candidate characters as the confirmed character. That is, the recognition result determination unit 126 may determine one of the multiple candidate characters as the confirmed character based on the accuracy of each of the multiple candidate characters, the weight value of each of the multiple candidate characters, and the priority of each of the multiple character recognition units CR. Alternatively, when there are multiple candidate characters with the highest evaluation values, the recognition result determination unit 126 may determine the candidate character with the highest accuracy among the multiple candidate characters as the confirmed character.

Here, in the example shown in FIG. 17, the candidate character with the highest accuracy among the multiple candidate characters is the character "dog" with an accuracy of 92%, as shown by the double circle in FIG. 17. For example, in a form in which a weight value is not set for each character (hereinafter also referred to as the fourth comparative example), the character "dog" with the highest accuracy among the multiple candidate characters is determined as the confirmed character corresponding to the recognition result of the individual character image GC of "large". That is, in the fourth comparative example, the character "dog" which has been corrected more times than the character "large" among the multiple candidate characters obtained from the character recognition unit CR4 is determined as the confirmed character corresponding to the recognition result of the individual character image GC of "large". For this reason, in the fourth comparative example, the probability that an incorrect character is determined as a confirmed character is higher than in this modified example. In other words, in this modified example, it is possible to suppress an incorrect character from being determined as a confirmed character compared to the fourth comparative example.

As described above, in this modified example, the same effect as in the first embodiment described above can be obtained. Note that the priority of the character recognition unit CR used to recognize a character may be determined for each character depending on the number of times or the correction rate of the character. Also, in the second and third embodiments described above, a weight value may be associated with each character recognized by each character recognition unit CR, as in this modified example.

[Second Modification]
In the above-described embodiment and modified example, the character recognition device 10 and the terminal device 20 are separate from each other, but the present invention is not limited to such an embodiment. For example, the character recognition device 10 and the terminal device 20 may be configured as an integrated unit.

Figure 18 is a diagram showing an example of a character recognition system 1A according to the second modified example. Elements similar to those described in Figures 1 to 17 are given the same reference numerals, and detailed descriptions are omitted.

The character recognition system 1A has a character recognition device 10A and an image input device 30. For example, the character recognition device 10A is communicatively connected to the image input device 30. Note that the connection between the character recognition device 10A and the image input device 30 may be a wired or wireless connection, or may be both a wired and wireless connection.

Character recognition device 10A has a processing device 12 that controls each part of character recognition device 10A, a storage device 14 that stores various information, a communication device 16, an operation device 27 that accepts operations by an operator of character recognition device 10A, and a display device 28 that displays images, etc. Character recognition device 10A is similar to character recognition device 10 shown in FIG. 3 except that it has operation device 27 and display device 28. However, character recognition device 10A differs from character recognition device 10 in that processing device 12 executes control program PG4 instead of control program PG1. Therefore, storage device 14 stores control program PG4 instead of control program PG1. Character recognition device 10A is another example of an "information processing device", and control program PG4 is another example of a "program".

In this modified example, the control program PG4 includes, for example, an application program for the character recognition device 10A to execute the character recognition process. However, the control program PG4 may also include, for example, an operating system program for the processing device 12 to control each part of the character recognition device 10A. The control program PG4 may be transmitted from another device.

The processing device 12, which operates according to an application program included in the control program PG4, functions as a control unit 120 including, for example, an image information acquisition unit 122, a description recognition unit 124, a recognition result determination unit 126, and a display information generation unit 128. The control unit 120 realized by the processing device 12 shown in FIG. 18 is similar to the control unit 120 shown in FIG. 4, except that the processing executed via the terminal device 20 is executed without the terminal device 20. For example, the image information acquisition unit 122 acquires image information indicating an image to be subjected to character recognition processing from the image input device 30. Also, for example, the display information generation unit 128 outputs display information to the display device 28 for displaying an image such as a content confirmation screen CCH on the display device 28.

The configuration of the character recognition system 1A is not limited to the example shown in FIG. 18. For example, the image input device 30 may be included in the character recognition device 10A. Also, for example, the character recognition device 10A may not include the display device 28. When the character recognition device 10A does not include the display device 28, it is connected to the display device 28 so as to be able to communicate with it. The connection between the character recognition device 10A and the display device 28 may be a wired or wireless connection, or may be a wired and wireless connection. Also, the processing device 12 that operates according to the application program included in the control program PG4 may function as, for example, the control unit 120A or may function as the control unit 120B. When the processing device 12 functions as the control unit 120B, for example, a weight table WTBL is stored in the storage device 14.

As described above, this modified example can also achieve the same effects as the above-mentioned embodiment and modified examples.

[Third Modification]
In the above-described embodiment and modified example, the character recognition device 10 or 10A may learn the content of corrections to the results of the character recognition process. As described above, the modified example can also achieve the same effects as the above-described embodiment and modified example.

[Fourth Modification]
In the above-mentioned embodiment and modified example, the image to be subjected to the character recognition process is the application form image GM, but the present invention is not limited to such an embodiment. For example, the image to be subjected to the character recognition process may be an image of a document such as a prescription, an accounting document (e.g., an invoice, a financial statement, a transfer form, an estimate, etc.), and a recording sheet (e.g., a sheet on which measured values in an experiment, etc. are recorded, etc.).

1, 1A...character recognition system, 10, 10A...character recognition device, 12...processing device, 14...storage device, 16...communication device, 20...terminal device, 27...operation device, 28...display device, 30...image input device, 120, 120A, 120B...control unit, 122...image information acquisition unit, 124...content recognition unit, 126...recognition result determination unit, 128...display information generation unit, CCH...content confirmation screen, CR...character recognition unit, GM...application form image, LM...learning model, NW...network, MK...attention mark, PTBL...priority table, WTBL...weight table.

Claims (11)

a recognition unit that recognizes one character of a character string represented for each item of contents to be written on a medium in a character image that is an image of the medium by each of a plurality of character recognition processes that recognize characters included in the image and calculate the accuracy of the recognized characters;
a determination unit that determines one of a plurality of candidate characters obtained as a recognition result of the one character by each of the plurality of character recognition processes as a confirmed character corresponding to the recognition result of the one character;
Equipped with
A priority of a character recognition process to be used for character recognition is determined for each of the above items,
The determination unit is
determining one of the plurality of candidate characters as the determined character based on the certainty of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes;
23. An information processing apparatus comprising: a weight value based on the priority level is determined for each of the plurality of character recognition processes;
The determination unit is
calculating an evaluation value for each of the plurality of candidate characters by weighting the likelihood of each of the plurality of candidate characters with the weight value for each of the plurality of character recognition processes;
determining, from among the plurality of candidate characters, the candidate character having the highest evaluation value as the confirmed character;
The information processing device according to claim 1 . The determination unit is
determining, among the plurality of candidate characters, the first candidate character as the confirmed character when the accuracy of a first candidate character obtained as a recognition result of the one character by the character recognition process having the highest priority among the plurality of character recognition processes is equal to or greater than a predetermined value;
The information processing device according to claim 1 . A display control unit that displays the determined character determined by the determination unit on a display unit,
The display control unit is
a display mode of the confirmed character when the confirmed character is not recognized in the character recognition process having the highest priority among the plurality of character recognition processes is made different from a display mode of the confirmed character when the confirmed character is recognized in the character recognition process having the highest priority among the plurality of character recognition processes;
The information processing device according to claim 1 . a recognition unit that recognizes a character included in a character image by a first character recognition process that recognizes a character included in an image and calculates a probability of the recognized character, and a second character recognition process that recognizes a character included in an image and calculates a probability of the recognized character;
a determination unit that, when a common character exists that is included in both one or more first candidate characters obtained as a recognition result of the one character by the first character recognition process and one or more second candidate characters obtained as a recognition result of the one character by the second character recognition process, determines whether or not the common character is to be a confirmed character corresponding to the recognition result of the one character;
Equipped with
The determination unit is
determining the common character as a confirmed candidate character that is a candidate for the confirmed character when the common character satisfies both a first condition that is a condition on the higher of a first probability that is a probability of the common character included in the one or more first candidate characters and a second probability that is a probability of the common character included in the one or more second candidate characters, and a second condition that is a condition on the degree of difference between the first probability and the second probability.
23. An information processing apparatus comprising: The first condition is:
The condition is that the higher probability is equal to or greater than a predetermined probability,
The second condition is:
A condition that a difference between the first accuracy and the second accuracy is equal to or smaller than a predetermined value.
The information processing device according to claim 5 . The determination unit is
When there are a plurality of confirmed candidate characters,
determining a confirmed candidate character having the highest degree of certainty among the plurality of confirmed candidate characters as the confirmed character;
The information processing device according to claim 6. The determination unit is
When there are a plurality of confirmed candidate characters,
determining, as the confirmed character, a confirmed candidate character having the highest average value of the first probability and the second probability among the plurality of confirmed candidate characters;
The information processing device according to claim 6. a recognition unit that recognizes a character included in a character image by a first character recognition process that recognizes a character included in an image and calculates a probability of the recognized character, and a second character recognition process that recognizes a character included in an image and calculates a probability of the recognized character;
a determination unit that, when a common character exists that is included in both one or more first candidate characters obtained as a recognition result of the one character by the first character recognition process and one or more second candidate characters obtained as a recognition result of the one character by the second character recognition process, determines whether or not the common character is to be a confirmed character corresponding to the recognition result of the one character;
Equipped with
The determination unit is
If there are multiple common characters,
determining, among the plurality of common characters, a common character having the highest average value of a first probability, which is a probability of the common character included in the one or more first candidate characters, and a second probability, which is a probability of the common character included in the one or more second candidate characters, as a confirmed candidate character that is a candidate for the confirmed character;
23. An information processing apparatus comprising: The processor,
a recognition unit that recognizes one character of a character string represented for each item of contents to be written on a medium in a character image that is an image of the medium by each of a plurality of character recognition processes that recognize characters included in the image and calculate the accuracy of the recognized characters;
a determination unit that determines one of a plurality of candidate characters obtained as a recognition result of the one character by each of the plurality of character recognition processes as a confirmed character corresponding to the recognition result of the one character;
Function as a
A priority of a character recognition process to be used for character recognition is determined for each of the above items,
The determination unit is
determining one of the plurality of candidate characters as the determined character based on the certainty of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes;
program. An information processing method for recognizing one character of a character string represented for each item of contents to be written on a medium in a character image that is an image of a medium by performing a plurality of character recognition processes that recognize characters included in an image and calculate the accuracy of the recognized character, and determining one of a plurality of candidate characters obtained as a recognition result of the one character as a confirmed character corresponding to the recognition result of the one character, comprising:
A priority is assigned to each of the plurality of character recognition processes,
determining one of the plurality of candidate characters as the determined character based on the certainty of each of the plurality of candidate characters and the priority of each of the plurality of character recognition processes;
23. An information processing method comprising:

Images