JP7574572B2 - Inspection device, inspection program, and inspection method - Google Patents

Description

The present invention relates to an inspection device for inspecting documents.

BPO (Business Process Outsourcing) is used to outsource operations that a company does not have the operational know-how to external companies. One BPO operation is a print service that produces personalized printed materials containing the client's customer data and seals them. In print services, multiple documents containing personal information may be sent to each customer. In this case, if documents intended for different customers are mixed up and sent, the client will lose the trust of the customer. For this reason, print services visually check the entire set of documents before sealing them to make sure no documents have been mixed up, which requires labor.

To address such a situation, an envelope inspection system has been proposed that sets a punching pattern to be the same for multiple documents to be inserted into the same envelope, and then inspects the punching pattern after the documents are inserted to detect envelopes containing documents for different customers (Patent Document 1).

JP 2013-193777 A

However, marking each document with a perforation pattern or other markings is a work-related matter and would spoil the appearance of the document, so it is difficult to obtain permission from the client.

The present invention was made in light of these circumstances. Its purpose is to provide an inspection device that can inspect whether documents belonging to different customers are mixed together without marking each document.

The inspection device according to one aspect of the present application is characterized by comprising an acquisition unit that acquires multiple document images, a recognition unit that refers to a storage unit that stores feature data and document types in association with each other and recognizes the document type of the document contained in each document image based on the acquired multiple document images and feature data, an extraction unit that extracts a reference key from the document image whose document type is a reference document and extracts a reference key from the document image whose document type is a comparison document, a determination unit that determines whether all of the extracted reference keys match the reference key, and an output unit that outputs the determination result.

In one aspect of the present application, it is possible to check whether documents belonging to different customers are mixed together without marking each document.

FIG. 1 is an explanatory diagram illustrating an example of the configuration of an inspection system. FIG. 2 is a block diagram showing an example of a hardware configuration of an inspection device. FIG. 2 is a block diagram showing an example of a hardware configuration of a terminal. FIG. 11 is an explanatory diagram illustrating an example of a task master DB. FIG. 13 is an explanatory diagram showing an example of an additional information DB. FIG. 2 is an explanatory diagram illustrating an example of a history DB. FIG. 2 is an explanatory diagram illustrating an example of a work status DB. FIG. 2 is an explanatory diagram illustrating an example of a training data DB. 11 is a flowchart showing an example of a business procedure. 13 is a flowchart showing another example of a business procedure. 13 is a flowchart illustrating an example of a procedure for a matching process. FIG. 1 is an explanatory diagram regarding a generation process of a learning model. 13 is a flowchart illustrating an example of a procedure for a learning model generation process. FIG. 13 is an explanatory diagram showing an example of a matching result screen. FIG. 13 is an explanatory diagram showing an example of a matching result screen. FIG. 2 is a block diagram showing an example of a functional unit included in the inspection device.

(Embodiment 1)
An embodiment will be described below with reference to the drawings. Fig. 1 is an explanatory diagram showing an example of the configuration of an inspection system. The inspection system 100 includes an inspection device 1, a terminal 2, and an imaging device 3. The inspection device 1 and the terminal 2 are connected to each other via a network N so as to be able to communicate with each other.

The inspection device 1 is configured with a server computer, a PC (Personal Computer), etc. The inspection device 1 may also be configured with a multicomputer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. Furthermore, the functions of the inspection device 1 may be realized by a cloud service.

The terminal 2 is configured as a notebook computer, a panel computer, a tablet computer, a smartphone, or the like. The imaging device 3 is configured as a digital camera, a scanner, or the like. The imaging device 3 may be configured as a plurality of digital cameras. The imaging device 3 may also be configured as a scanner with a feeder, a handheld scanner, or the like. In FIG. 1, the imaging device 3 is connected to the terminal 2, but may also be connected to the network N.

In FIG. 1, two terminals 2 are shown, but the number may be one or more than three. Similarly, the number of imaging devices 3 may be one or more than three.

Figure 2 is a block diagram showing an example of the hardware configuration of the inspection device. The inspection device 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, a communication unit 15, a clock unit 16, and a reading unit 17. The control unit 11, the main memory unit 12, the auxiliary memory unit 13, the communication unit 15, the clock unit 16, and the reading unit 17 are connected by a bus B.

The control unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), etc. The control unit 11 reads out and executes a control program 1P (inspection program) stored in the auxiliary storage unit 13, thereby performing various information processing, control processing, etc. related to the inspection device 1, and realizing functional units such as an acquisition unit, a recognition unit, an extraction unit, a determination unit, and an output unit.

The main memory unit 12 is a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory unit 12 mainly temporarily stores data required for the control unit 11 to execute arithmetic processing.

The auxiliary memory unit 13 is a hard disk or SSD (Solid State Drive) or the like, and stores the control program 1P and various DBs (Databases) necessary for the control unit 11 to execute processing. The auxiliary memory unit 13 stores a task master DB 131, ancillary information DB 132, history DB 133, task status DB 134, and training data DB 135. The auxiliary memory unit 13 also stores a learning model 141. The auxiliary memory unit 13 may be an external storage device connected to the inspection device 1. The various DBs and the like stored in the auxiliary memory unit 13 may be stored in a database server or cloud storage different from the inspection device 1.

The communication unit 15 communicates with the terminal 2 via the network N. The control unit 11 may also use the communication unit 15 to download the control program 1P from another computer via the network N, etc., and store it in the auxiliary storage unit 13.

The clock unit 16 measures time, such as the time elapsed since the inspection device 1 was started, or the like. The clock unit 16 is a circuit that provides the control unit 11 with the measured time in response to a request from the control unit 11. It is desirable for the clock unit 16 to repeatedly communicate with a reliable NTP server using NTP (Network Time Protocol) to correct the time.

The reading unit 17 reads portable storage medium 1a, including CD (Compact Disc)-ROM and DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 17 and store it in the auxiliary storage unit 13. The control unit 11 may also download the control program 1P from another computer via a network N or the like and store it in the auxiliary storage unit 13. Furthermore, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

Figure 3 is a block diagram showing an example of the hardware configuration of a terminal. The terminal 2 includes a control unit 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, an input unit 25, a display unit 26, an interface unit 27, and a clock unit 28. Each component is connected by a bus B.

The control unit 21 has one or more arithmetic processing devices such as a CPU, MPU, GPU, etc. The control unit 21 reads and executes the control program 2P stored in the auxiliary storage unit 23 to perform various information processing, control processing, etc. related to the terminal 2.

The main memory unit 22 is an SRAM, DRAM, flash memory, etc. The main memory unit 22 mainly temporarily stores data required for the control unit 21 to execute arithmetic processing.

The auxiliary storage unit 23 is a hard disk or SSD, etc., and stores the control program 2P and various data necessary for the control unit 21 to execute processing.

The communication unit 24 communicates with the inspection device 1 via the network N. In addition, the control unit 21 may use the communication unit 24 to download the control program 2P from another computer via the network N, etc., and store it in the auxiliary storage unit 23.

The input unit 25 is a keyboard and a mouse. The display unit 26 includes a liquid crystal display panel, etc. The display unit 26 displays the document inspection results, etc. If the inspection results in an error, the display unit 26 displays the location and content of the error. The display unit 26 and the input unit 25 may be integrated into one to form a touch panel display. The terminal 2 may display on an external display device.

The interface unit 27 performs data communication in accordance with standards such as USB (Universal Serial Bus) and IEEE (Institute of Electrical and Electronics Engineers) 1394. The interface unit 27 receives images from the imaging device 3.

The clock unit 28 measures time, such as the time or the elapsed time since the terminal 2 was started. The clock unit 28 is a circuit that provides the control unit 21 with the measured time in response to a request from the control unit 21. It is desirable that the time held by the clock unit 28 and the time held by the clock unit 16 of the inspection device 1 are synchronized.

Next, the database used by the inspection system 100 will be described. Figure 4 is an explanatory diagram showing an example of the task master DB 131. The task master DB 131 stores information related to a workflow. The task master DB 131 includes a task code column and a task name column. The task code column stores a task code that uniquely identifies a workflow. The task name column stores the name of the workflow.

Figure 5 is an explanatory diagram showing an example of the additional information DB 132. The additional information DB 132 stores additional information related to documents. The additional information DB 132 includes a working code string, a document recognition information string, a document type string, a document name string, a key area coordinate string, a placement location string, and a sequence string. The working code string stores a working code. The document recognition information string stores data referenced when recognizing the document type. For example, when the document type is recognized using the learning model 141, the document recognition information string stores output label information. When the document type is recognized by image pattern matching, the document recognition information string stores a reference image. The document type string stores the document type. The document type is, for example, a reference, a collation, or a general purpose. The reference refers to a document that serves as a reference (reference document) when referencing a key. A document in which a key different from the key (reference key) written in the reference document is written results in an error. The collation refers to a document in which a key to be compared with the reference key is written when referencing a key (reference document). "General purpose" refers to a general purpose document (general purpose document) without a key. In this specification, the general purpose document includes an envelope. The document name column stores the name of the document. The key area coordinate column stores coordinates indicating the key area in the image of each document (hereinafter referred to as "document image"). For example, the key area is rectangular, and the coordinate values of the two diagonal end points of the rectangle are stored. If the key area spans multiple locations, a label including a sequence number such as "key 1" and "key 2" is attached to the coordinate value. The placement location column stores the correct position when arranging the documents on the workbench. A space for arranging the documents on the workbench is secured, the space is divided into multiple areas, and labels including sequence numbers are attached. The order column stores the order in which the documents are scanned when the imaging device 3 that acquires the document image is a scanner. The order of the documents indicates the order in which the documents are stacked when the documents are enclosed in an envelope. Place the document with sequence number 1 on top, then document with sequence number 2 underneath, document with sequence number 3, and so on, and place the documents in an envelope.

Figure 6 is an explanatory diagram showing an example of history DB 133. History DB 133 stores the history of the matching work. History DB 133 includes a working code column, a matching date and time column, a matching result column, a key column, an error column, an error location column, a first worker column, and a second worker column. The working code column stores the working code. The matching date and time column stores the date and time when the matching work was performed. The matching result column stores the matching result. If the matching is successful, the matching result column stores, for example, OK. A value other than OK, such as ○ or 1, may be stored in the matching result column. If the matching is unsuccessful, the matching result column stores NG. A value other than NG, such as × or 0, may be stored in the matching result column. The key column stores the reference key used for matching. The error column stores the error content when the matching fails. If the key of the matching document does not match the reference key, the error column stores, for example, key mismatch. If the document is placed incorrectly, the error column stores, for example, placement error. A code may be assigned to each error content, and the error column may store the code. If the match is successful, the error column stores -, NULL, 0, etc. The error location column stores the location of the error. When documents are lined up on a workbench, the error location column stores the label or sequence number of the area where the error occurred. When documents are scanned with a scanner, the error location column stores the sequence number of the document where the error occurred. The first worker column and second worker column each store an employee ID, worker ID, etc. that can uniquely identify the worker who performed the work.

Figure 7 is an explanatory diagram showing an example of the work status DB 134. The work status DB 134 stores the work status. The work status DB 134 includes a work code column, a work name column, a work date column, a first worker column, a second worker column, a match count column, an OK column, and an NG column. The work code column stores the work code. The work name column stores the name of the work flow. The work date column stores the work date. The first worker column and the second worker column store the employee ID of the worker, etc. The match count column stores the number of times the match was performed. The OK column stores the number of times the match was successful. The NG column stores the number of times the match was unsuccessful. The work status DB stores the work status of the day. The work status is used for display on the display unit 26 of the terminal 2. Therefore, the work status DB 134 may store only one record for the day. In this case, the work date column is not required. Furthermore, the OK column and the NG column are not required. The number of successful matches and the number of failed matches can be obtained from the history DB 133.

Figure 8 is an explanatory diagram showing an example of the training data DB 135. The training data DB 135 stores training data for generating or relearning the learning model 141. The training data DB 135 includes a working code string, an image string, and a correct label string. The working code string stores a working code. The image string stores a document image. The entity of the document image may be stored as a file in the auxiliary storage unit 13, and the image string may store a file name. The correct label string stores the correct label of the document type corresponding to the document image.

Next, the procedure for the inspection system 100 will be described. FIG. 9 is a flowchart showing an example of the procedure. The procedure shown in FIG. 9 is a procedure flow in the case where the imaging device 3 is a tabletop camera and a set of documents placed on a work table is photographed by the tabletop camera. The worker starts the inspection program, inputs the worker ID, and selects the work flow (step S1). The worker places a set of documents on the work table (step S2). The worker photographs the documents (step S3). After photographing, the inspection system 100 performs a comparison (step S4). The worker checks the comparison result returned by the inspection system 100 (step S5). The worker checks whether the comparison result is successful or not (step S6). If the worker checks that the comparison result is successful (YES in step S6), the worker puts the documents in an envelope and seals it (step S7). The worker decides whether to end the work (step S8). If the worker decides to continue the work (NO in step S8), the process returns to step S2 and works on the unprocessed documents. If the worker decides to end the work (YES in step S8), the work ends. If the worker confirms that the comparison result is unsuccessful (NO in step S6), the worker refers to the error message and takes appropriate action (step S9). The worker returns to step S3 and performs the inspection again. It is desirable that the documents placed on the work table are folded, especially large documents, so that they can be immediately enclosed. The reference document or comparison document is placed so that the key area is located in a position that is visible when folded, and the key area is photographed. A robot arm or the like may be used to place the set of documents on the work table. If it is determined that the arrangement of the documents on the work table is incorrect, a robot arm or the like may be used to arrange them in the correct arrangement.

Figure 10 is a flowchart showing another example of the business procedure. The business procedure shown in Figure 10 is a business flow in the case where the imaging device 3 is a scanner and scans each document that constitutes a set of documents. The worker starts the inspection program, inputs the worker ID, and selects the work flow (step S11). The worker scans the documents that constitute the set of documents in order (step S12). When the worker has scanned all the documents, he/she issues a matching instruction (step S13). The worker checks the matching result returned by the inspection system 100 (step S14). The worker checks whether the matching result is successful or not (step S15). If the worker checks that the matching result is successful (YES in step S15), the worker puts the documents in an envelope and seals it (step S16). The worker decides whether to end the work (step S17). If the worker decides to continue the work (NO in step S17), the process returns to step S12 and works on the unprocessed documents. If the worker decides to end the work (YES in step S17), the process ends. If the worker finds that the comparison result is unsuccessful (NO in step S15), he or she refers to the error message and takes appropriate action (step S18). The worker returns to step S12 and performs the inspection again. Note that it is desirable for the document to be scanned to be folded, especially for large documents, so that it can be enclosed immediately. The reference document or comparison document is scanned in a folded state so that the key area is located in a visible position and the key area can be read.

Next, the information processing performed by the inspection system 100 will be described. FIG. 11 is a flowchart showing an example of the procedure of the matching process. The control unit 11 of the inspection device 1 starts the matching process when the control unit 11 receives an image captured by the imaging device 3 from the terminal 2. The control unit 11 judges whether the received image is a single image or not (step S31). When a set of documents is captured by a table camera, the received image is a single image. When each document constituting the set of documents is scanned by a scanner, the received image is an image group consisting of multiple images. When the control unit 11 judges that the received image is not a single image (NO in step S31), the process proceeds to step S33. When the control unit 11 judges that the received image is a single image (YES in step S31), the control unit 11 cuts out the image of each document from the received image (step S32). For example, the area of each document is detected as follows. A green cloth or sheet is laid on the workbench, and the document is placed on it and photographed. In this case, the document area can be detected by threshold processing in color information. With reference to the color information (RGB values) of the image, it can be determined that the part with a large value indicating the green component is the background, and the other part is the document area. Also, a known edge detection process may be used to detect the outline of the document, and the document area may be detected based on the outline. Note that since the document area is assumed to be rectangular, a keystone correction process may be performed so that the detected document area becomes rectangular. The control unit 11 selects the document image to be processed (step S33). The control unit 11 recognizes the document type (step S34). The document type is recognized by inputting the document image into the learning model 141 and judging from the value of each label output by the learning model 141. The learning model 141 will be described later. Also, the document image may be compared with a reference image stored in the document recognition information column of the supplementary information DB 132 to judge the document. The control unit 11 judges whether the document type is a general-purpose document (step S35). When the control unit 11 determines that the document type is a general-purpose document (YES in step S35), the process proceeds to step S38. When the control unit 11 determines that the document type is not a general-purpose document (NO in step S35), the control unit 11 extracts a key area based on the value of the key area coordinate column in the supplementary information DB 132 (step S36). The control unit 11 performs character recognition processing on the key area to recognize the key (step S37). It should be noted that the extraction of the key area (step S36) is not essential. Character recognition may be performed on the entire document image without extracting the key area. Character area detection may be performed on the key area, and character recognition may be performed on the detected character area. This is expected to improve the recognition rate. The control unit 11 associates the recognized document type, key, and document placement position or scan order, and stores them in a temporary storage area (step S38). The temporary storage area is provided in the main storage unit 12 or the auxiliary storage unit 13. The control unit 11 determines whether there is an unprocessed document image (step S39). When the control unit 11 determines that there is an unprocessed document image (YES in step S39), the control unit 11 returns the process to step S33 and processes the unprocessed document image. When the control unit 11 determines that there is no unprocessed document image (NO in step S39), the control unit 11 compares the document type and the document arrangement position or scan order stored in the temporary storage area with the document type, arrangement position column, or sequence column in the additional information DB 132 (step S40). The control unit 11 compares the reference key (step S41). The control unit 11 acquires the reference key from the document type and key stored in the temporary storage area. The control unit 11 acquires the key of the comparison document (matching key) from the document type and key stored in the temporary storage area. The control unit 11 compares all the matching keys with the reference key and determines whether they all match. The control unit 11 outputs the matching result (step S42). If the arrangement positions or order of all the documents are correct and all the matching keys match the reference key, the matching result is successful. If any document is not placed in the correct position or order, if there are documents that are insufficient or excessive, or if there are documents whose matching key does not match the reference key, the matching result will be a failure. If the matching result is a failure, the control unit 11 outputs the details of the failure. The control unit 11 ends the matching process. When outputting the matching result in step S42, the control unit 11 adds a record of the matching result to the history DB 133 as history. Note that character recognition does not have to be performed in recognizing the key (step S37). In this case, in matching with the reference key (step S41), an image of the reference key and an image of the matching key are compared to determine whether they match.

Next, the learning model 141 will be described. FIG. 12 is an explanatory diagram of the generation process of the learning model 141. FIG. 12 conceptually shows the process of generating the learning model 141 by performing machine learning. The control unit 11 of the inspection device 1 generates a neural network that inputs document images and outputs document types as the learning model 141 by performing deep learning to learn the features of each document image in each work flow. The neural network is, for example, a CNN (Convolution Neural Network), and has an input layer that accepts input of document images, an output layer that outputs the document type, and an intermediate layer that extracts image features of the document image.

The input layer has a plurality of neurons that accept input of pixel values of each pixel included in the document image, and passes the input pixel values to the intermediate layer. The intermediate layer has a plurality of neurons, extracts features from the document image, and passes them to the output layer. For example, the intermediate layer has a configuration in which a convolution layer that convolves the pixel values of each pixel input from the input layer and a pooling layer that maps the pixel values convolved in the convolution layer are alternately connected, and finally extracts image features while compressing the pixel information of the document image. The output layer is composed of one or more neurons that output the document type, and outputs the document type based on the image features output from the intermediate layer. Note that in this embodiment, the learning model 141 is described as a CNN, but the learning model 141 is not limited to a CNN, and may be a model constructed with other learning algorithms such as a neural network other than CNN or an SVM (Support Vector Machine), and the document type may be determined without using a learning model such as pattern matching or a decision tree.

The control unit 11 performs learning using training data in which multiple document images are associated with correct answer values for the document type for each document image. For example, as shown in FIG. 12, the training data is data in which correct answer labels corresponding to the document type are assigned to the working code and document image. Label 0 corresponds to the reference document, label 1 corresponds to the comparison document, and label 2 corresponds to a general-purpose document.

The control unit 11 inputs document images, which are training data, into the input layer, and obtains the probability value of each label from the output layer after arithmetic processing in the intermediate layer. The control unit 11 compares the probability value of each label output from the output layer with the correct value label, and optimizes the parameters used in the arithmetic processing in the intermediate layer so that the probability value from the correct label in the output layer approaches 1. The parameters in question are, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method of optimizing the parameters, but for example, the control unit 11 optimizes various parameters using the backpropagation method. The control unit 11 performs the above processing on each document image included in the training data to generate a learning model 141.

Figure 13 is a flowchart showing an example of the procedure of the learning model 141 generation process. The control unit 11 selects one of the training data stored in the training data DB 135 (step S51). The control unit 11 extracts input data excluding the correct label from the training data (step S52). The control unit 11 inputs the input data to the input layer of the neural network (step S53). The control unit 11 acquires the output data and the probability value for each label from the output layer (step S54). The control unit 11 compares the probability value for each label with the correct label, and optimizes the parameters (weights, etc.) used in the calculation process in the intermediate layer so that the probability value of the correct label output from the output layer approaches 1 (step S55). The control unit 11 determines whether there is unused training data (step S56). If the control unit 11 determines that there is unused training data (YES in step S56), the process returns to step S51. If the control unit 11 determines that there is no unused training data (NO in step S56), the control unit 11 stores the generated learning model 141 in the auxiliary storage unit 13 (step S57) and ends the process.

Next, the screens displayed on the display unit 26 of the terminal 2 will be described. FIG. 14 is an explanatory diagram showing an example of a matching result screen. FIG. 14 is an example of a screen when matching is successful. The matching result screen d01 includes a name area d011, a worker area d012, a key area d013, and a message area d014, which include a result display area d015 and a counter area d016. The name area d011 displays the name of the work flow. The worker area d012 displays the worker IDs of the two workers. The key area d013 displays the recognized keys in a table format. FIG. 14 shows a case where there are two key areas. The first row of the key area d013 displays the recognized key value. The other rows show the image of the key area for each document. KEY displays an image of the reference key, and SUB1, SUB2, and SUB3 display images of the matching keys of the matching documents. The message area d014 displays a message supplementing the matching result. If matching is successful, nothing is displayed. The result display area d015 displays the matching result. In FIG. 14, a circle is displayed to indicate a successful match. The counter area d016 displays the number of matches made. The number of matches made is reset to 0 immediately after the inspection program is started and immediately after a workflow is selected. The number of matches made includes the number of successful matches and the number of unsuccessful matches. The number of successful matches and the number of unsuccessful matches may also be displayed separately.

Figure 15 is an explanatory diagram showing an example of a matching result screen. Figure 15 is an example of a screen when matching has failed. The screen configuration is the same as in Figure 14, so an explanation of each area will be omitted. In the example shown in Figure 15, an error has occurred because the key of the form shown in SUB1 does not match the reference key. To indicate that the SUB1 form is in error, the background of the SUB1 row in the key area d013 is hatched. In an actual screen, the background may be red, or may be a blinking display. An error message is displayed in the message area d014. The result display area d015 displays a cross indicating that matching has failed.

On the matching result screen d01, the name area d011, the worker area d012, and the counter area d016 may be displayed based on data stored in the work status DB 134. The work status DB 134 may be stored in the auxiliary storage unit 23 of the terminal 2.

In this embodiment, it is possible to check whether documents belonging to different customers are mixed together among multiple documents. In addition, the parts where matching has failed are displayed, so that the worker can quickly check the documents corresponding to the failed parts and replace them with the correct documents.

The matching process may be performed by the terminal 2 instead of the inspection device 1. In this case, the work master DB 131, the additional information DB 132, the work status DB 134, and the learning model 141 are stored in the auxiliary memory unit 23 of the terminal 2. The matching history may be stored in the auxiliary memory unit 23 during work and transferred to the inspection device 1 after the work is completed.

Document images may also be stored in the history DB 133. Once a certain amount of history data has been accumulated in the history DB 133, training data may be generated from the history data and the learning model 141 may be retrained. Retraining is performed by the inspection device 1. When recognition is performed by the terminal 2, the parameters of the learning model 141 obtained by retraining are reflected in the learning model 141 stored in the terminal 2.

Figure 16 is a block diagram showing an example of functional units of an inspection device. The inspection device 1 includes, as functional units, an acquisition unit 11a, a recognition unit 11b, an extraction unit 11c, a determination unit 11d, and an output unit 11e. Each of these functional units is realized by the control unit 11 operating based on a control program 1P.

The acquisition unit 11a acquires a document image for each document from images of multiple documents. The recognition unit 11b refers to a storage unit that stores feature data and document types in association with each other, and recognizes the document type of the document contained in each document image based on the acquired document image and feature data. The extraction unit 11c extracts a reference key from the document image whose document type is a reference document, and extracts a reference key from the document image whose document type is a matching document. The determination unit 11d determines whether all of the extracted reference keys match the reference key. The output unit 11e outputs the determination result.

The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed.
The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

REFERENCE SIGNS LIST 100 Inspection system 1 Inspection device 11 Control unit 11a Acquisition unit 11b Recognition unit 11c Extraction unit 11d Determination unit 11e Output unit 12 Main memory unit 13 Auxiliary memory unit 131 Work master DB
132 Additional information DB
133 History DB
134 Work status DB
135 Training Data DB
Reference Signs List 141 Learning model 15 Communication unit 16 Timekeeping unit 17 Reading unit 1P Control program 1a Portable storage medium 1b Semiconductor memory 2 Terminal 21 Control unit 22 Main storage unit 23 Auxiliary storage unit 24 Communication unit 25 Input unit 26 Display unit 27 Interface unit 28 Timekeeping unit 2P Control program 3 Imaging device

Claims (6)

An acquisition unit for acquiring a plurality of document images;
a recognition unit that refers to a storage unit that stores feature data extracted from a document image and a document type of a document included in the document image in association with each other, and recognizes a document type of a document included in each of the document images based on the acquired multiple document images and the feature data extracted from the multiple document images ;
an extracting unit that extracts a reference key from the document image whose document type is a reference document and extracts a reference key from the document image whose document type is a comparison document;
a determination unit that determines whether or not all of the extracted reference keys match the reference key;
and an output unit that outputs a result of the determination. the storage unit stores the document type and the key area coordinate sequence in association with each other;
2. The inspection device according to claim 1, wherein the extraction unit extracts the base key and the reference key based on values of the key area coordinate sequence. The storage unit stores the document layout and the document type in association with each other,
The multiple document images are a single image, and the recognition unit segments each document image;
Recognizing the arrangement of each of the document images within the plurality of document images;
The inspection device according to claim 1 or claim 2, characterized in that the judgment unit refers to the memory unit and judges whether there are any excess or shortage of documents and whether there are any errors in the arrangement based on the recognized document type and arrangement. The storage unit stores the document acquisition order and the document type in association with each other,
The plurality of document images are made up of a plurality of images to which sequential numbers indicating the order in which the document images were obtained are assigned,
The inspection device according to claim 1 or claim 2, characterized in that the judgment unit refers to the memory unit and judges whether there are any excess or shortage of documents and whether there is any error in the acquisition order based on the recognized document type and sequence number. A computer that can refer to a storage unit that stores feature data extracted from a document image and a document type of a document included in the document image in association with each other,
Acquire multiple document images,
referring to the storage unit, recognizing a document type of the document included in each of the document images based on the acquired document images and feature data extracted from the document images ;
Extracting a reference key from the document image whose document type is a reference document and extracting a reference key from the document image whose document type is a comparison document;
determining whether or not all of the extracted reference keys match the reference key;
and outputting a result of the judgment. A computer that can refer to a storage unit that stores feature data extracted from a document image and a document type of a document included in the document image in association with each other,
Acquire multiple document images,
referring to the storage unit, recognizing a document type of the document included in each of the document images based on the acquired document images and feature data extracted from the document images ;
Extracting a reference key from the document image whose document type is a reference document and extracting a reference key from the document image whose document type is a comparison document;
determining whether or not all of the extracted reference keys match the reference key;
and outputting a result of the determination.

Images