JP6989790B2 - Information processing system, its control method, and program - Google Patents

Description

The present invention relates to an information processing system, a control method thereof, and a program.

Conventionally, there is a character recognition technique that analyzes an imaged character pattern and outputs it as character data. By acquiring the image of the characters written on the application form or the form as character data, it is possible to improve the efficiency of office work and the like.

Further, in recent years, in order to further improve the accuracy of this character recognition technique, a mechanism for performing character recognition and character data extraction using machine learning has been provided.

Patent Document 1 describes a mechanism for improving the accuracy of extracting character information from a form such as an invoice by using a plurality of machine learning devices that perform different learning.

Japanese Unexamined Patent Publication No. 2019-82814

By the way, when character recognition is performed by OCR (Optical Character Recognition) processing, there arises a problem that if the image to be read is a deteriorated image, it cannot be processed well. For example, if the image for character recognition is an image obtained by taking a form with a camera or a scanned image, shadows or overexposure may occur in a part of the image and character recognition may not be performed properly.

Therefore, by having a learning device learn such an image for which character recognition is difficult as a deteriorated image and then performing OCR processing on the image, it is possible to improve the recognition accuracy of the OCR processing and the accuracy of the output result.

However, in order to perform machine learning, a lot of image data for learning is required. Since there are various deterioration patterns of images, it is very troublesome for the user to collect and prepare many deteriorated images for each of a plurality of deterioration patterns.

Therefore, an object of the present invention is to provide a mechanism capable of easily generating an image necessary for analyzing a deterioration tendency.

The information processing system of the present invention includes an acquisition means for acquiring a first image including information on image deterioration, and a learning means for causing a learning model to learn the first image acquired by the acquisition means as training data. , A third image different from the first image, and by inputting the scanned second image into the training model, the second image and the result learned by the training model are used. It is characterized by comprising a generation means for generating an image of.

According to the present invention, it is possible to provide a mechanism capable of easily generating an image necessary for analyzing a deterioration tendency.

It is a figure which shows an example of the structure of the information processing system including the server apparatus 102 which concerns on embodiment of this invention. It is a figure which shows an example of the hardware composition of the server apparatus 102 which concerns on embodiment of this invention. It is a figure which shows an example of the functional structure of the server apparatus 102 which concerns on embodiment of this invention. It is a figure of the flowchart which shows an example of the deterioration image learning model generation processing and deterioration image generation processing which concerns on embodiment of this invention. It is a figure of the flowchart which shows an example of the OCR model learning process which concerns on embodiment of this invention. It is a figure of the flowchart which shows an example of the processing of the whole image from the preprocessing to the OCR result output in the OCR processing of the image which concerns on embodiment of this invention. It is a figure of the flowchart which shows an example of the detailed processing from the preprocessing to the OCR result output in the OCR processing of the image which concerns on embodiment of this invention. It is a figure which shows an example of the pretreatment pattern 800 by group which concerns on embodiment of this invention. It is a figure which shows an example of the pre-processing setting screen 900 which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

As shown in FIG. 1, the information processing system 100 of the present embodiment has a configuration in which an information processing device 101 and a server device 102 are communicably connected by a LAN 103. The information processing device 101 captures an image of a document by taking an image with a camera, scanning with an image forming device, or the like, and transmits the image to the server device 102. The server device 102 analyzes the image and transmits the analysis result to the information processing device 101. Further, the server device 102 learns information for generating a deteriorated image based on the image acquired from the information processing device 101, generates a deteriorated image based on the learned information, and transmits the deteriorated image to the information processing device 101. do. Further, the server device 102 sorts the image into a group for which preprocessing is predetermined based on the image acquired from the information processing device 101, performs preprocessing for the group, and converts the image into the preprocessed image. OCR is performed on the object, and the OCR result is transmitted to the information processing apparatus 101. The information processing device 101 may be a portable mobile terminal such as a mobile phone, a smartphone, a tablet terminal, a notebook PC, and a PDA terminal. The information processing device 101 has a browser or an image analysis application installed on it, and can communicate with the server device 102 via the LAN 103.

The system configuration of the information processing system in this embodiment is an example, and there may be other configuration examples other than the information processing device 101 and the server device 102.

Next, an example of the hardware configuration applicable to the server device 102 will be described with reference to FIG.

The CPU 201 comprehensively controls each device and controller connected to the system bus 204.

Further, the ROM 202 or the external memory 211 stores a BIOS (Basic Input / Output System), which is a control program of the CPU 201, an operating system program (hereinafter, OS), and the like. .. Further, the ROM 202 or the external memory 211 stores various programs and the like necessary for realizing the functions executed by each server or each PC. The RAM 203 functions as a main memory, a work area, and the like of the CPU 201.

The CPU 201 realizes various operations by loading a program or the like necessary for executing a process into the RAM 203 and executing the program.

Further, the input controller (input C) 205 controls an input from an input device 209 such as a keyboard or a pointing device such as a mouse (not shown).

The video controller (VC) 206 controls the display on a display such as the display 210. The display may be a CRT or a liquid crystal display.

The memory controller (MC) 207 controls access to an external memory 211 such as a hard disk (HD) that stores a boot program, browser software, various applications, font data, user files, edit files, various data, and the like. Further, the memory controller (MC) 207 controls access to an external memory 211 such as a card type memory connected to the flexible disk (FD) or the PCMCIA card slot via an adapter.

The communication I / F controller (communication I / FC) 208 connects and communicates with an external device via a network, and executes communication control processing on the network. For example, Internet communication using TCP / IP is possible.

The CPU 201 enables display on the display 210 by, for example, executing an outline font expansion (rasterization) process in the display information area in the RAM 203. Further, the CPU 201 enables a user instruction with a mouse cursor or the like (not shown) on the display 210.

Since the GPU 212 can perform efficient calculations by processing more data in parallel, it is effective to perform the processing with the GPU 212 when learning is performed multiple times using a learning model such as deep learning. Is. Therefore, in the embodiment of the present invention, the GPU 212 is used in addition to the CPU 201 for the processing by the learning model generation unit 303 and the machine learning unit 305. Specifically, when a learning program including a learning model is executed, learning is performed by the CPU 201 and the GPU 212 collaborating to perform an operation. The processing by the learning model generation unit 303 and the machine learning unit 305 may be performed only by the CPU 201 or the GPU 212. Further, the GPU 212 may be similarly used for the deterioration pattern analysis unit 302 and the image analysis unit 306.

Various programs and the like used by the various devices and servers of the present invention to execute various processes described later are recorded in the external memory 211, and are executed by the CPU 201 and the GPU 212 by being loaded into the RAM 203 as needed. It is a thing. Further, the definition file and various information tables used by the program according to the present invention are stored in the external memory 211.

FIG. 3 is a block diagram showing an example of the functional configuration of the server device 102.

The server device 102 includes an image acquisition unit 301, a deterioration pattern analysis unit 302, a learning model generation unit 303, a deterioration image generation unit 304, and a machine learning unit 305. Further, the server device 102 includes an image analysis unit 306, a preprocessing unit 307, a preprocessing image generation unit 308, and an OCR processing unit 309.

The image acquisition unit 301 is a functional unit that acquires image data of a document or a form from the information processing apparatus 101 using a camera, a scanner, or the like.

The deterioration pattern analysis unit 302 is a functional unit that analyzes the deterioration pattern of the image acquired from the information processing apparatus 101. Image deterioration patterns include deterioration due to shadows, light reflection, deterioration due to overexposure, deterioration due to character distortion due to document folding or bending, and the like. The deterioration pattern analysis unit 302 analyzes which of these deterioration patterns the acquired image applies to from information such as the brightness, contrast, and inclination of the image.

The learning model generation unit 303 is a functional unit that generates a learning model for generating a deterioration image using the information analyzed by the deterioration pattern analysis unit 302. Using the learning model generated by the learning model generation unit 303, the server device 102 can generate a deteriorated image from a specific image.

The deteriorated image generation unit 304 is a functional unit that generates a deteriorated image by using the specific image acquired by the image acquisition unit 301 and the learning model generation unit 303. The deteriorated image generation unit 304 compares the generated deteriorated image with the deteriorated image for comparison acquired by the image acquisition unit 301 or stored in advance, and confirms whether or not the desired deteriorated image can be generated.

The machine learning unit 305 is a functional unit that learns the deterioration characteristics of the deteriorated image and performs machine learning by inputting the deteriorated image into the learning model generated by the learning model generation unit 303.

The image analysis unit 306 is a functional unit that analyzes the tendency of the image by analyzing the image acquired by the image acquisition unit 301 and distributes it to the preprocessing unit for each group.

The pre-processing unit 307 is a functional unit that performs pre-processing on the image acquired by the image acquisition unit 301. The pre-processing referred to here refers to processing for adjusting contrast, removing noise, binarizing, etc. for an image to be OCRed, and is a processing process performed on an image in order to improve the recognition accuracy of OCR. Show that.

The pre-processed image generation unit 308 is a functional unit that generates a pre-processed image that is the result of pre-processing by the pre-processing unit 307. When performing a plurality of preprocessing on the image acquired by the image acquisition unit 301, the acquired image is duplicated and different preprocessing is performed. Therefore, the server device 102 performs different preprocessing from one image. Preprocessed images can be generated.

The OCR processing unit 309 is a functional unit that performs OCR processing on the preprocessed image that has been preprocessed by the preprocessing unit 307. The OCR processing unit 309 may include a plurality of OCR engines, and in the present embodiment, the OCR processing unit 309 includes three engines, an OCR engine A, an OCR engine B, and an OCR engine C. The number of OCR engines may be one or more.

Next, an example of the deteriorated image learning model generation process and the deteriorated image generation process according to the present embodiment will be described with reference to the flowchart of FIG. The degraded image learning model generation process is an example of a process of generating a learning model used when generating a degraded image, and the degraded image generation process uses a degraded image learning model generated by the degraded image learning model generation process. An example of the process of automatically generating a deteriorated image is shown.

In step S401, the CPU 201 of the server device 102 acquires a deteriorated image from the information processing device 101 by the function of the image acquisition unit 301. In the present embodiment, the deteriorated image is acquired from the information processing device 101, but the acquisition source is any medium such as a camera, a mobile terminal, a facsimile (hereinafter, FAX), or an image forming apparatus (not shown) that can acquire the image. You may. The deteriorated image in the present embodiment is the reading of characters due to deterioration due to shadows, light reflection, overexposure, distortion of characters due to folding or bending of documents, character sprinkling due to deterioration of image resolution, bleeding, and the like. Shows a difficult image. In step S401, an image including information on such deterioration is acquired.

In step S402, the CPU 201 of the server device 102 accepts the deterioration pattern of the image acquired in step S401 by the user's selection. It is preferable to accept the tendency of the deterioration pattern visually judged by the user on the deterioration pattern selection screen (not shown) by the user's input and select the deterioration pattern.

In step S403, the CPU 201 of the server device 102 analyzes the tendency of image deterioration by the function of the deterioration pattern analysis unit 302. The deterioration tendency is analyzed for each deterioration pattern by using the information that can be acquired from the image such as the brightness and contrast of the image and the deterioration pattern received in step S402. As for the deterioration pattern, the server device 102 may specify the deterioration pattern by the analysis in step S403 without accepting the deterioration pattern from the user in step S402. Further, in step S403, when analyzing the deterioration tendency of the image, information such as the model of the device that captured the image, the resolution, and the image size may be acquired from the property of the image and used for the analysis. The analyzed information and the information regarding the properties of the acquired image are stored in the external memory of the server device 102.

In step S404, the CPU 201 of the server device 102 generates a degraded image learning model based on the information of the degraded image analyzed and acquired in step S403. That is, a degraded image learning model is generated by training the learning model with the image acquired in step S401 as learning data. The degraded image learning model can machine-learn the information of the degraded pattern by repeating the processes of steps S401 to S403, and can generate a degraded image using this degraded image learning model.

In step S405, the CPU 201 of the server device 102 acquires (inputs) an image scanned and read by a scanner (not shown) or an image forming device (not shown) from the information processing device 101 and inputs it to the deteriorated image learning model. .. The image read at this time is an image different from the image acquired in step S401.

In step S406, the CPU 201 of the server device 102 generates a deteriorated image based on the image input in step S405. This process indicates, that is, a process of generating a new deteriorated image by inputting an image into the deteriorated image learning model in step S405.

In step S407, the CPU 201 of the server device 102 compares the image acquired in step S405 with the deteriorated image generated in step S406 based on the image acquired in step S405. The deteriorated image generated in step S406 may be compared with the same image previously stored in the server device 102 if there is one.

In step S408, the CPU 201 of the server device 102 compares the two images in step S407, and as a result, determines whether or not the image acquired in step S405 and the deteriorated image generated in step S406 are similar. The determination of the similarity of the deteriorated image may be made by using the image analysis of the prior art, for example, using the feature amount of the image, or any technique may be used as long as the similarity of the images can be determined. When the CPU 201 of the server device 102 determines that the deteriorated image acquired in step S405 and the deteriorated image generated in step S406 are similar, it determines that an appropriate deteriorated image has been generated and ends a series of processes. do. If the CPU 201 of the server device 102 determines that the image acquired in step S405 and the deteriorated image generated in step S406 are not similar, the process returns to step S401. At this time, the CPU 201 of the server device 102 may return the process to step S405 instead of step S401.

Next, the deteriorated image generation process will be described using the processes of steps S409 to S412 in the flowchart of FIG. The process from step S409 to step S412 is a process for generating a degraded image using the degraded image learning model generated in the process from step S401 to step S408. In the present embodiment, the degraded image generated by using the degraded image learning model is used as learning data for training the degraded image learning model, and further as learning data for adjusting the OCR model described later. Used. It should be noted that a degraded image may be generated using a degraded image learning model in order to generate a degraded image for purposes other than learning data.

In step S409, the CPU 201 of the server device 102 acquires an image scanned or captured by a camera (not shown) from the information processing device 101 by the function of the image acquisition unit 301.

In step S410, the CPU 201 of the server device 102 accepts the user to select the deterioration pattern of the image acquired in step S409. For example, it is preferable to accept the tendency of the deterioration pattern visually determined by the user on the deterioration pattern selection screen (not shown) by the input of the user.

In step S411, the CPU 201 of the server device 102 inputs (inputs) the image acquired in step S409 into the degraded image learning model created in step S404.

In step S412, the CPU 201 of the server device 102 generates a deteriorated image by using the deteriorated image learning model by the function of the deteriorated image generation unit 304. More specifically, the CPU 201 of the server device 102 generates a deteriorated image based on the image acquired in step S409 based on the deterioration pattern for which selection is accepted in step S410. For example, the CPU 201 of the server device 102 changes and adjusts the brightness and contrast of the image acquired in step S409 based on the deterioration pattern information and the property information of the image to reproduce the deterioration of the image. Then, the deteriorated image generated in step S412 is input to the deteriorated image learning model as learning data. By repeating this process, the degraded image learning model repeatedly learns about the degraded image.

In this way, the server device 102 can automatically generate a plurality of types of deteriorated images by repeating the processes of steps S409 to S412. Further, in the deteriorated image learning model, it is possible to learn the tendency of deterioration according to the acquired image based on the deterioration pattern for accepting selection, the type of the scanned image to be acquired, and the property information. Further, the degraded image learning model can generate a plurality of types of degraded images from one image by using the learned result. This makes it possible to generate a large amount of deteriorated images that reproduce different types of deterioration without the user's trouble.

Next, the OCR model learning process will be described using the flowchart of FIG. The OCR model of the present embodiment is a combination of an OCR engine and machine learning, and can learn OCR results for each deterioration pattern to improve OCR recognition accuracy.

In step S501, the CPU 201 of the server device 102 inputs the deteriorated image generated in step S412 of FIG. 4 into the OCR model.

In step S502, the CPU 201 of the server device 102 analyzes and identifies the deterioration pattern of the deteriorated image input to the OCR model by the function of the deterioration pattern analysis unit 302.

In step S503, the CPU 201 of the server device 102 reads the deteriorated image, recognizes the character string included in the deteriorated image, performs OCR, and causes the OCR model to learn the result by the function of the machine learning unit 305. The learning of steps S502 and S503 is repeated in order to improve the recognition accuracy of the OCR model.

In step S504, the CPU 201 of the server device 102 inputs the deteriorated image for testing stored in advance for testing into the OCR model.

In step S505, the CPU 201 of the server device 102 performs OCR of the deteriorated image for testing input in step S504 by the function of the OCR processing unit 309, and determines whether or not the OCR result is equal to or higher than a predetermined value. If the CPU 201 of the server device 102 determines that the OCR result is equal to or higher than a predetermined value, the process proceeds to step S506. If the CPU 201 of the server device 102 determines that the OCR result is not equal to or higher than a predetermined value, the OCR reading accuracy is still insufficient, so the process returns to step S501 and the learning of the OCR model is repeated.

In step S506, the CPU 201 of the server device 102 outputs the result of performing OCR in step S504. At this time, if OCR is performed by a plurality of OCR engines, the OCR results are merged and output. The output result may be trained by the OCR model.

This is the end of the description of the degraded image learning model generation process, the degraded image generation process, and the OCR model learning process using FIGS. 4 and 5.

By performing the above processing, it is possible to create multiple images with various deterioration patterns from a small number of deterioration images, so it is possible to reduce the time and effort of the user who prepares a large amount of deterioration images for machine learning. Become. Further, by training the generated deteriorated image as learning data in the deteriorated image learning model, it is possible to increase the re-limit of the deterioration of the deteriorated image. Further, by training the generated deteriorated image in the OCR model as learning data, it is possible to improve the recognition accuracy of the OCR process.

Next, with reference to FIGS. 6 to 7, when performing OCR with a plurality of OCR engines, a series of flows for easily performing appropriate preprocessing for each OCR engine will be described. In order to perform OCR processing of an image using a plurality of OCR engines, it is necessary to distribute the image to an appropriate OCR engine according to the image, and further perform preprocessing according to the OCR engine to distribute the image. Therefore, in the present embodiment, while distributing the image to an appropriate OCR engine, preprocessing is performed according to the distributed OCR engine, and processing for improving the recognition accuracy of OCR is performed.

FIG. 6 is a flowchart showing the entire flow from performing preprocessing to the image acquired by the server device 102 to performing OCR.

In step S601, the CPU 201 of the server device 102 analyzes the acquired image. The analysis of the acquired image will be described in detail using the flowchart of the analysis of the input image of FIG. 7.

In step S701 of FIG. 7, the CPU 201 of the server device 102 acquires an image from the information processing device 101 and acquires it by the function of the image acquisition unit 301.

In step S702, the CPU 201 of the server device 102 identifies the property information of the image acquired in step S701. For example, information such as the model of the device that captured, scanned, or faxed the image and the resolution of the image is specified.

In step S703, the CPU 201 of the server device 102 extracts the paper surface portion from the image by the function of the image analysis unit 306, and analyzes the resolution and inclination information of the paper surface portion. Since the image in the present embodiment is an image of a document or a form, the information on the paper surface thereof is extracted and analyzed.

In step S704, the CPU 201 of the server device 102 analyzes the brightness of the paper surface portion extracted in step S703. Brightness is analyzed from information such as image brightness and contrast values.

In step S705, the CPU 201 of the server device 102 determines whether or not the image acquired in step S701 corresponds to the preset image group based on the information acquired, specified, and analyzed in steps S702 from step S702. To judge. If the CPU 201 of the server device 102 determines that the image acquired in step S701 corresponds to the image group, the process proceeds to step S706, and if it determines that the image does not correspond to the image group, the process proceeds to step S707.

Here, with reference to FIG. 8, a group of images and a preprocessing pattern for each group will be described. The preprocessing pattern 800 for each group in FIG. 8 is a diagram showing an example of the correspondence between the group of images and the preprocessing pattern for each group. Further, the pre-processing pattern 800 for each group is also a diagram showing an example of the correspondence between the characteristics of the image and the pre-processing pattern. The image group differs depending on the characteristics of the deterioration pattern. For example, group A applies to an image with noise, and group B applies to an image that is generally dark (brightness of a predetermined value or less). Further, if the overall numerical values such as the brightness and the inclination of the image are equal to or more than a predetermined value, the group C is applicable as a good image, and the dark image is applicable to the group D after the image contains noise.

Further, in the preprocessing pattern 800 for each group, a preprocessing pattern to be applied to the image is defined for each group. As for which pre-processing is to be performed on the group, the pre-processing for which the input is accepted on the pre-processing setting screen 900 of FIG. 9 is set for each group. On the pre-processing setting screen 900, it is possible to set the necessary pre-processing by accepting the selection in the check box, and for the binarization process, the degree of binarization is set by setting the binarization value in detail. It is possible to specify for each binarization pattern.

The explanation is returned to FIG. For example, the preprocessing pattern a is applied to the group A in FIG. In the pre-processing pattern a, the pre-processing of the binarization pattern A, the black spot noise removal and the direction correction, and the pre-processing of the binarization pattern D and the vertical line noise removal are set, so that the pre-processing of this combination is performed. Applies to images.

The explanation is returned to FIG. In step S706, the CPU 201 of the server device 102 determines the image group to be applied when it is determined that the image acquired in step S705 corresponds to the existing image group. For example, the image acquired in step S705 is confirmed in the group A of FIG.

In step S707, when the CPU 201 of the server device 102 determines in step S705 that the image does not correspond to the existing image group, the image is determined as the other group. Other groups indicate groups for which no preprocessing pattern is set.

This is the end of the description of FIG. By this processing, it is possible to group the acquired images according to the tendency of the images according to the deterioration and the type of the acquired images, and it is possible to perform appropriate preprocessing on the images in the preprocessing step described later.

Here, the explanation is returned to FIG. In step S602, the CPU 201 of the server device 102 performs preprocessing for each image group. Details will be described with reference to steps S708 to S712 of FIG.

In step S708 of FIG. 7, the CPU 201 of the server device 102 determines whether or not the image group is fixed. If the CPU 201 of the server device 102 determines that the image group has been determined, the process proceeds to step S709, and if the image group has not been determined, that is, if the image group has been determined as another group in step S707, the process proceeds to step S710. Proceed with processing.

In step S709, the CPU 201 of the server device 102 performs the preprocessing (processing process) set in the image group determined for the image acquired in step S701. At this time, one preprocessing pattern (processing pattern) is executed for one image. For example, when an image is confirmed in group A, since two preprocessing patterns are set in group A, the image is duplicated and one of the images is set in the preprocessing pattern a in FIG. Perform the pretreatment that has been done. Then, the preprocessing set in the preprocessing pattern d is performed on the other duplicated image, and each preprocessed image is generated.

In step S710, the CPU 201 of the server device 102 generates a preprocessed image. More specifically, the CPU 201 of the server device 102 duplicates the image acquired in step S701 for the set preprocessing pattern, and executes all the preprocessing patterns set for each of the duplicated images (). Perform processing). At this time, one preprocessing pattern is executed for one image, and preprocessing images for all the set preprocessing patterns are generated. The CPU 201 of the server device 102 repeats the process of step S710 until all the preprocessed images of the set preprocessed patterns are generated.

In step S711, the CPU 201 of the server device 102 executes OCR with a plurality of OCR engines on the generated preprocessed image by the function of the OCR processing unit 309. Since the pre-processing pattern is set as an appropriate pre-processing for each OCR engine, the pre-processing image is distributed to the OCR engine corresponding to the executed pre-processing and OCR is performed.

In step S711, the CPU 201 of the server device 102 outputs the result of the OCR performed in step S711.

With the above processing, the description of the preprocessing for each image group in FIG. 7 ends. As a result, necessary preprocessing can be performed on the image for which the image group is confirmed, and OCR processing can be performed by an appropriate OCR engine. Normally, in order to perform optimum preprocessing, preprocessed images for all the set preprocessing patterns are generated, and OCR is performed by all OCR engines. However, in that method, it is necessary to generate a large amount of preprocessed images, and a large amount of preprocessing must be executed, which puts a load on the server device 102. Therefore, by performing the processing as in the present invention, it is possible to perform the minimum necessary and appropriate preprocessing and execute OCR with the optimum OCR engine.

The explanation is returned to FIG. In FIG. 6, the result of performing OCR on the preprocessed image of the preprocessing performed in step S602 is output. More specifically, the process of steps S713 to S717 of FIG. 7 will be used for description.

In step S713 of FIG. 7, the CPU 201 of the server device 102 compares the OCR results performed by a plurality of OCR engines for each item of the form in the image of the form subjected to OCR, and the value of the item matches the regular expression. Determine if it is.

In step S714, the CPU 201 of the server device 102 determines whether or not it is within the correct value range for each item of the form in the image of the form for which OCR has been performed.

In step S715, the CPU 201 of the server device 102 determines whether or not the certainty level is equal to or higher than the reference value for each item of the form in the image of the form subjected to OCR.

The CPU 201 of the server device 102 repeats the processes of steps S713 to S715 for each item of the form as much as the result output by OCR.

In step S716, the CPU 201 of the server device 102 extracts the item determined to be the best by using the result of performing OCR using a plurality of OCR engines and the determination result in the processing of steps S713 to S715. Then, the CPU 201 of the server device 102 determines (adopts) the final OCR result for each item using the extracted items.

In step S717, the CPU 201 of the server device 102 merges the items extracted in step S716 from the results of performing OCR using a plurality of OCR engines, and outputs the final OCR result.

This completes the description of FIG. 7. As a result, when multiple OCR engines are used, optimum preprocessing is performed for each OCR engine, and the best OCR results are finally merged from the OCR results using multiple OCR engines to achieve OCR recognition accuracy. It becomes possible to raise.

In the present embodiment, all the processes are performed by the server device 102, but the device that performs the processes may be divided for each process and each process may be executed by a plurality of devices.

As described above, according to the present invention, it is possible to provide a mechanism capable of easily generating an image necessary for analyzing a deterioration tendency.

It is needless to say that the structure and contents of the various data described above are not limited to this, and are composed of various structures and contents depending on the intended use and purpose.

Although the embodiments of the present invention have been described in detail above, the present invention can be, for example, an embodiment as a system, an apparatus, a method, a program, a storage medium, or the like, and specifically, a plurality of devices. It may be applied to a system composed of, or it may be applied to a device consisting of one device.

Further, an object of the present invention is to supply a storage medium in which a program code of software that realizes the functions of the above-described embodiment is recorded to a system or an apparatus, and a computer (or a CPU or MPU) of the system or the apparatus is a storage medium. Needless to say, it can also be achieved by reading and executing the program code stored in.

In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the program code itself and the storage medium storing the program code constitute the present invention.

As the storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, or the like can be used.

Further, by executing the program code read by the computer, not only the function of the above-described embodiment is realized, but also the OS (basic system or operating system) running on the computer is realized based on the instruction of the program code. ) And the like perform a part or all of the actual processing, and it goes without saying that the processing may realize the function of the above-described embodiment.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function is based on the instruction of the program code. Needless to say, there are cases where the CPU provided in the expansion board or the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiment.

It should be noted that the present invention also includes all the configurations in which each of the above-described embodiments and modifications thereof are combined.

130 LAN
101 Information processing device 102 Server device 201 CPU
202 ROM
203 RAM
211 External memory 212 GPU

Claims (13)

An acquisition method for acquiring a first image containing information on image deterioration,
A learning means that causes a learning model to learn the first image acquired by the acquisition means as learning data.
A third image that is different from the first image and that uses the second image and the result of learning by the learning model by inputting the scanned second image into the learning model. The generation method to generate the image and
An information processing system characterized by being equipped with. The information processing system according to claim 1, wherein the learning means trains the learning model of the third image generated by the generation means as learning data. The learning means causes the learning model to learn information about the deterioration of the first image for each deterioration pattern.
The information processing system according to claim 1 or 2, wherein the generation means generates the third image for each deterioration pattern. The generation means is characterized in that the third image is generated by applying the information on the deterioration trained by the learning means to the second image input to the learning model. Item 1. The information processing system according to Item 1. The information according to claim 4, wherein the generation means generates the third image by changing or adjusting at least one of the brightness and the contrast of the image based on the information regarding the deterioration of the image. Processing system. A reception means that accepts the selection of which deterioration pattern is among a plurality of deterioration patterns,
Further prepare
The generation means is characterized in that, by inputting the second image into the learning model, the third image based on the deterioration pattern whose selection is accepted by the reception means is generated. The information processing system according to any one of 4. The information processing according to claim 6, wherein the plurality of deterioration patterns include at least one of deterioration due to shadow, deterioration due to light reflection, deterioration due to overexposure, and deterioration due to character distortion. system. The information processing system according to any one of claims 1 to 7 , wherein the first image is a photographed or scanned image. The information processing system according to claim 8, wherein the first image is an image in which characters are difficult to read. The first image is a deteriorated image containing at least one of deterioration due to shadow, reflection of light, blown-out highlights, distortion of characters due to folding or bending of a document, sprinkling of characters due to a decrease in image resolution, and bleeding. The information processing system according to claim 8 or 9. OCR processing means that performs OCR processing on an image,
Further prepare
The information processing system according to any one of claims 1 to 10, wherein the OCR processing means performs OCR processing of the third image generated by the generation means. It is a control method for information processing systems.
The acquisition step to acquire the first image, which contains information about image degradation,
A learning step in which the learning model learns the first image acquired in the acquisition step as learning data, and
A third image that is different from the first image and that uses the second image and the result of learning by the learning model by inputting the scanned second image into the learning model. The generation step to generate the image and
A method of controlling an information processing system, which comprises. It is a program to function as an information processing system.
The information processing system
An acquisition method for acquiring a first image containing information on image deterioration,
A learning means that causes a learning model to learn the first image acquired by the acquisition means as learning data.
A third image that is different from the first image and that uses the second image and the result of learning by the learning model by inputting the scanned second image into the learning model. Generating means to generate images,
A program to function as.

Images