JP7790482B2 - Image Processing Device - Google Patents

Description

The present invention relates to an image processing device.

Conventionally, image processing devices are known that scan an identification card as a document and mask areas containing personal information from the image data obtained by the scan. Such an image processing device is disclosed, for example, in Patent Document 1.

Japanese Patent Application Laid-Open No. 2021-197682

For example, the personal information written on an ID card is categorized into several items. These items include name and address. With such an ID card, it may be desirable to mask only the areas of the image data obtained by scanning that correspond to specified items. Note that the areas that need to be masked will vary depending on the user.

Here, if you are only targeting a specific type of ID, you can set the area to be masked (in other words, the position within the image data of the ID) in advance. This means that if the scanned ID is of the specified type, only the area of the image data of the ID that corresponds to the specified item will be masked.

However, when scanning and masking a type of ID card that is different from the specified type, it is necessary to set the area to be masked each time. This is cumbersome and inconvenient for users.

The present invention has been made to solve the above problem, and aims to provide an image processing device that can easily mask only the areas of image data obtained by scanning a document that correspond to items specified by the user.

To achieve the above object, an image processing device according to one aspect of the present invention includes an image reading unit that reads a document containing multiple sets of item names and item values for personal information items; a control unit that recognizes target items that are to be masked and performs a masking process on image data obtained by reading the document with the image reading unit, thereby generating masked data in which at least a portion of the area of the target item present in the image data is masked; and an operation display unit that displays information and accepts operations. The operation display unit accepts an item designation operation to designate one of the items. The control unit recognizes the item designated in the item designation operation as the target item, and performs the masking process by masking at least the item value area of the area of the target item present in the image data to generate masked data.

The configuration of the present invention makes it easy to mask only the areas of the image data obtained by scanning a document that correspond to user-specified items.

FIG. 1 is a schematic diagram of a multifunction peripheral according to an embodiment. FIG. 1 is a block diagram of a multifunction peripheral according to an embodiment. 1 is a diagram illustrating an example of an identification card that can be read by a multifunction peripheral according to an embodiment; 10 is a flowchart illustrating the flow of a masking job executed by the multifunction peripheral of an embodiment. 1 is a diagram showing a first area in image data obtained by reading an identification card using a multifunction peripheral according to an embodiment; 10 is a diagram showing a second area in image data obtained by reading an identification card using the multifunction peripheral of the embodiment; FIG. 10 is a diagram illustrating masked data generated by a masking job performed by the multifunction peripheral of the embodiment. FIG. 10 is a conceptual diagram of dictionary data used in a masking job by the multifunction peripheral of an embodiment. 10 is a flowchart illustrating a flow of an area determination process performed in the multifunction peripheral according to an embodiment. 10 is a flowchart illustrating a flow of a first abnormality countermeasure process performed in the multifunction peripheral according to an embodiment. 10 is a flowchart illustrating a flow of second and third abnormality countermeasure processes performed in the multifunction peripheral of an embodiment. 10 is a flowchart illustrating a flow of a second abnormality countermeasure process performed in the multifunction peripheral according to an embodiment. 10 is a flowchart illustrating a flow of a third abnormality countermeasure process performed in the multifunction peripheral according to an embodiment. 10 is a flowchart illustrating a process flow performed when a masking job setting is accepted by the multifunction peripheral of an embodiment. FIG. 10 illustrates a reception screen displayed by the multifunction peripheral according to an embodiment. FIG. 10 illustrates a registration screen displayed by the multifunction peripheral according to an embodiment.

<Configuration of the multifunction device>
An image processing apparatus according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 16, taking as an example a multifunction peripheral 100 having multiple functions such as a scanning function, a printing function, and a transmission function.

As shown in FIG. 1, the multifunction device 100 (corresponding to an "image processing device") includes a printing unit 1. The printing unit 1 constitutes the main body of the multifunction device 100. The printing unit 1 prints an image on a sheet S. The printing method of the printing unit 1 is an electrophotographic method. However, this is not limited to this. The printing method of the printing unit 1 may also be an inkjet method.

The printing unit 1 forms an image based on image data input to the multifunction device 100. The printing unit 1 also transports a sheet S along a sheet transport path. The printing unit 1 prints an image on the sheet S as it is being transported. In Figure 1, the sheet transport path is indicated by a dashed line.

The printing unit 1 is equipped with a paper feed roller 11. The paper feed roller 11 contacts the sheet S stored in the sheet cassette CA and rotates in this state. As a result, the paper feed roller 11 feeds the sheet S from the sheet cassette CA into the sheet transport path.

The printing unit 1 includes an image forming unit 12. The image forming unit 12 includes a photosensitive drum 12a and a transfer roller 12b. The photosensitive drum 12a carries a toner image on its peripheral surface. The transfer roller 12b is pressed against the photosensitive drum 12a, forming a transfer nip between itself and the photosensitive drum 12a. The transfer roller 12b rotates together with the photosensitive drum 12a. The image forming unit 12 transfers the toner image onto the sheet S while transporting the sheet S that has entered the transfer nip.

Although not shown, the image forming unit 12 further includes a charging device, an exposure device, and a developing device. The charging device charges the circumferential surface of the photosensitive drum 12a. The exposure device forms an electrostatic latent image on the circumferential surface of the photosensitive drum 12a. The developing device develops the electrostatic latent image on the circumferential surface of the photosensitive drum 12a into a toner image.

The printing unit 1 includes a fixing unit 13. The fixing unit 13 includes a heating roller 13a and a pressure roller 13b. The heating roller 13a has a built-in heater (not shown). The pressure roller 13b presses against the heating roller 13a, forming a fixing nip between the heating roller 13a and the pressure roller 13b. The pressure roller 13b rotates together with the heating roller 13a. The fixing unit 13 fixes the toner image transferred onto the sheet S to the sheet S while transporting the sheet S that has entered the fixing nip. The sheet S that has exited the fixing nip is discharged onto the discharge tray ET.

The multifunction device 100 also includes an image reading unit 2. The image reading unit 2 is located on top of the main body of the multifunction device 100. For a job that involves reading an original document D, the original document D is set in the image reading unit 2. The image reading unit 2 reads the original document D set in the image reading unit 2 and generates image data of the read original document D.

The image reading unit 2 includes contact glasses G1 and G2. The contact glasses G1 and G2 are installed in the housing RH of the image reading unit 2. The housing RH has an opening on its top surface. The contact glasses G1 and G2 are attached to the opening on the top surface of the housing RH.

The image reading unit 2 includes a document transport device DP. The document transport device DP is attached to the housing RH. When viewed from the front of the multifunction device 100, the document transport device DP pivots around its rear portion as a fulcrum, swinging its front portion up and down. The document transport device DP opens and closes relative to the top surface of the housing RH.

The document transport device DP has a set tray ST on which the document D is set. The document transport device DP transports the document D set on the set tray ST onto the contact glass G1.

In the transport reading mode, the user places the document D on the set tray ST. The document D is then automatically transported onto the contact glass G1 by the document transport device DP (in other words, the document D passing over the contact glass G1) and read. In the placement reading mode, the user places the document D on the contact glass G2, and the document D on the contact glass G2 is read.

The image reading unit 2 includes a light source 21, an image sensor 22, a mirror 23, and a lens 24. The light source 21, the image sensor 22, the mirror 23, and the lens 24 are installed inside the housing RH. The image reading unit 2 performs a scanning operation in which light is irradiated from the light source 21 toward the contact glass G1 or G2 and photoelectrically converted by the image sensor 22.

The light source 21 has multiple LED elements. The multiple LED elements are arranged in a line in the main scanning direction (the direction perpendicular to the paper surface of Figure 1). The image sensor 22 has multiple photoelectric conversion elements lined up in the main scanning direction. The mirror 23 reflects light toward the lens 24. The lens 24 collects the light reflected by the mirror 23 and guides it to the image sensor 22.

The light source 21 and mirror 23 are mounted on a carriage 25 that can move in a sub-scanning direction (left and right in Figure 1) that is perpendicular to the main scanning direction. As the carriage 25 moves in the sub-scanning direction, the reading line of the image reading unit 2 moves in the sub-scanning direction.

As shown in FIG. 2, the multifunction device 100 also includes an operation display unit 3. The operation display unit 3 is an operation panel with a touch screen. The operation display unit 3 displays software buttons, messages, and the like on the touch screen. The operation display unit 3 also has a plurality of hardware buttons. The operation display unit 3 accepts operations from the user. The user can use the operation display unit 3 to configure the multifunction device 100 for various jobs, such as a masking job, which will be described later.

The multifunction device 100 includes a control unit 10. The control unit 10 includes a CPU, an ASIC, a memory, and the like. The control unit 10 also includes an image processing circuit. The control unit 10 performs various image processing operations on image data. The control unit 10 also controls the printing of an image onto a sheet S by the printing unit 1, and the reading of an original D by the image reading unit 2.

The control unit 10 also controls the operation and display unit 3. Specifically, the control unit 10 controls the display operation of the touch screen. The control unit 10 detects operations on the software buttons and hardware buttons. The control unit 10 sets jobs based on operations received from the user by the operation and display unit 3.

The multifunction device 100 includes a storage unit 101. The storage unit 101 is a non-volatile storage device. An HDD, SSD, or the like may be used as the storage unit 101. The storage unit 101 is connected to the control unit 10. The control unit 10 writes information to the storage unit 101 and reads information from the storage unit 101.

The memory unit 101 stores predetermined information in advance. For example, the memory unit 101 stores a character recognition program in advance. The control unit 10 performs character recognition processing such as OCR (Optical Character Recognition) processing based on the character recognition program. The control unit 10 subjects the image data obtained by reading the document D with the image reading unit 2 to character recognition processing.

The multifunction device 100 includes a communication unit 102. The communication unit 102 is an interface for communicatively connecting an external device to the multifunction device 100. The communication unit 102 includes a communication circuit, communication memory, and communication connector. The communication unit 102 is connected to the control unit 10. The control unit 10 uses the communication unit 102 to send and receive data to and from the external device.

The communication unit 102 is communicatively connected to external devices via a network NT such as a LAN or the Internet. Although not shown, the communication unit 102 may also be directly connected to external devices via a communication cable. An external device connected to the communication unit 102 is, for example, a personal computer 1000 (hereinafter referred to as PC 1000) used by a user of the multifunction device 100. An external device other than PC 1000 may also be communicatively connected to the multifunction device 100. By connecting the PC 1000 to the multifunction device 100, image data of the original D obtained by scanning the original D with the image reading unit 2 can be sent to the PC 1000. This allows the image data of the original D to be saved in the PC 1000.

<Masking job overview>
When entering into a contract for goods or services, the person wishing to enter into the contract may be required to register personal information. This is just one example, and there are various other cases where registration of personal information is required. Examples of personal information include name, address, and date of birth.

The user of the multifunction device 100 registers personal information. When registering personal information, the user uses the scanning function of the multifunction device 100 to scan their identification card. There are various types of identification cards that can be used, and these vary depending on the country and field. Examples of identification cards include driver's licenses, insurance cards, student ID cards, and passports.

An example of an ID card is shown schematically in Figure 3. Personal information is classified into multiple items and written on the ID card. That is, the ID card lists multiple sets of item names and item values for items related to personal information. In the example shown in Figure 3, the character string "Name" corresponds to the item name, and the character string "aaaa" corresponds to the item value of the item corresponding to the item name "Name". The character string "Address" corresponds to the item name, and the character string "bbbb" corresponds to the item value of the item corresponding to the item name "Address". The character string "Date of Birth" corresponds to the item name, and the character string "cccc" corresponds to the item value of the item corresponding to the item name "Date of Birth".

When the ID card shown in Figure 3 is read, the image data obtained by reading the ID card will contain a character area containing the string "Name" indicating the item name and a character area containing the string "aaaa" indicating the item value. A character area containing the string "Address" indicating the item name and a character area containing the string "bbbb" indicating the item value will also appear. A character area containing the string "Date of Birth" indicating the item name and a character area containing the string "cccc" indicating the item value will also appear.

In the following explanation, when it is necessary to distinguish between multiple character areas, the character area for the item name "Name" is assigned the code C11, and the character area for the item value "aaaa" is assigned the code C12. The character area for the item name "Address" is assigned the code C21, and the character area for the item value "bbbb" is assigned the code C22. The character area for the item name "Date of Birth" is assigned the code C31, and the character area for the item value "cccc" is assigned the code C32.

After the ID card is read by the multifunction device 100, an image based on the image data of the ID card obtained by the scan (i.e., the personal information written on the ID card) is printed on a sheet S. The sheet S with the printed personal information is then stored. In this manner, the personal information is registered. Note that the image data (i.e., electronic data) of the ID card itself may also be stored.

In some cases, it may be necessary to mask some personal information. For example, suppose an ID card has a field for date of birth, but the date of birth is not required in contracts for goods and services. In this example, it may be necessary to mask personal information related to the date of birth. Also, for example, in some countries, ID cards have a field for religion, but it is stipulated that personal information related to religion should not be recorded. In this example, it may be necessary to mask personal information related to religion.

For example, it has traditionally been common to manually black out personal information to conceal it. However, manually masking some personal information can be a hassle for users.

For this reason, the multifunction device 100 is equipped with a masking function. In other words, the multifunction device 100 is capable of executing jobs related to the masking function (hereinafter simply referred to as "masking jobs").

By using the masking function, it is possible to obtain masked data, which is image data in which some of the personal information on an identification card has been masked. An image based on the masked data can be printed on sheet S, or the masked data can be sent to PC 1000 and saved on PC 1000.

When executing a masking job, masking job settings are made. When configuring a masking job, you can specify any personal information to be masked. Masking job settings will be explained in detail later.

After setting up the masking job, the user places the ID card as document D into the image reading unit 2. In this state, the user performs a start operation for the masking job on the operation display unit 3. When the control unit 10 detects that a start operation has been performed on the operation display unit 3, it starts the masking job.

The flow of a masking job will be explained below with reference to the flowchart shown in Figure 4. The flow shown in Figure 4 starts when the control unit 10 detects a start operation for a masking job on the operation display unit 3.

Before starting a masking job, the user configures the masking job. When configuring the masking job, target items are specified, which are items that correspond to the item values to be masked. When executing the masking job, the control unit 10 recognizes the target items specified by the user.

In step #1, the control unit 10 causes the image reading unit 2 to read an identification card as document D. The image reading unit 2 reads the identification card and generates image data of the read identification card (i.e., scanned data of the identification card). The control unit 10 acquires the image data of the identification card obtained by the image reading unit 2 reading the identification card.

In step #2, the control unit 10 performs a first region detection process using a first learning model obtained by machine learning. The control unit 10 performs the first region detection process by detecting a first region that is predicted to individually include either an item name region or an item value region from the image data of the identification card.

The first learning model for the first area detection process is a learning model trained to detect the first area from image data obtained by reading an identification card using the image reading unit 2. The first learning model is a trained model and is pre-stored in the memory unit 101.

Identification cards typically contain multiple pairs of item names and item values. Therefore, multiple first areas are detected in the first area detection process. For this reason, if no first areas are detected as a result of the first area detection process, or if only one first area is detected, the masking job may be stopped and a message may be displayed on the operation display unit 3 prompting the user to check the document D set in the image reading unit 2.

In the ID card shown in Figure 3, the character area C11 for the item name "Name," the character area C12 for the item value "aaaa," the character area C21 for the item name "Address," the character area C22 for the item value "bbbb," the character area C31 for the item name "Date of Birth," and the character area C32 for the item value "cccc" are each individually detected as first areas. The first areas are clearly shown in Figure 5.

In step #3, the control unit 10 performs a second region detection process using a second learning model obtained by machine learning. The control unit 10 performs the second region detection process by detecting a second region from the image data of the identification card that is predicted to include both the area of the item name and the area of the item value that correspond to the same item.

The second learning model for the second area detection process is a learning model trained to detect the second area from image data obtained by reading an identification card using the image reading unit 2. The second learning model is a trained model and is pre-stored in the memory unit 101.

If an identification card has been read, at least one second area will be detected as a result of the second area detection process. If no second area is detected in the second area detection process, it is possible that something other than an identification card has been read. Therefore, if no second area is detected in the second area detection process, the masking job may be stopped, and a message may be displayed on the operation display unit 3 prompting the user to check the document D set in the image reading unit 2.

In the example shown in Figure 5, areas C10, C20, and C30 shown in Figure 6 are detected as second areas. Specifically, the single area C10 that includes the character area C11 of the item name "Name" and the character area C12 of the item value "aaaa" is detected as the second area. The single area C20 that includes the character area C21 of the item name "Address" and the character area C22 of the item value "bbbb" is detected as the second area. The single area C30 that includes the character area C31 of the item name "Date of Birth" and the character area C32 of the item value "cccc" is detected as the second area.

In step #4, the control unit 10 performs inclusion area detection processing for each of the multiple first areas detected in the first area detection processing. Each of the multiple first areas is subjected to the inclusion area detection processing once. The control unit 10 performs the inclusion area detection processing on one of the first areas as the processing target, and when that inclusion area detection processing is complete, it performs the inclusion area detection processing on another first area that has not yet been processed as the new processing target.

The control unit 10 performs inclusion region detection processing to detect a second region that encompasses the first region being processed. The control unit 10 detects a second region that encompasses the first region being processed by detecting a second region that has an image portion whose image similarity with the first region being processed is equal to or greater than the threshold for inclusion region detection processing. The control unit 10 determines that the first region being processed is encompassed by a second region that has an image portion whose image similarity with the first region being processed is equal to or greater than the threshold for inclusion region detection processing. Note that character recognition processing has not been performed at this point. In other words, image similarity refers to the similarity between image data, not the similarity between character strings (i.e., between text data). The control unit 10 performs inclusion region detection processing using, for example, known pattern matching technology. The threshold for inclusion region detection processing is not particularly limited, but is typically 70% or greater.

In the examples shown in Figures 5 and 6, the second region C10 has an image portion where the image similarity with the first region C11 (the region with the item name "Name") is equal to or greater than a threshold, and also has an image portion where the image similarity with the first region C12 (the region with the item value "aaaa") is equal to or greater than a threshold. The second region C20 has an image portion where the image similarity with the first region C21 (the region with the item name "Address") is equal to or greater than a threshold, and also has an image portion where the image similarity with the first region C22 (the region with the item value "bbbb") is equal to or greater than a threshold. The second region C30 has an image portion where the image similarity with the first region C31 (the region with the item name "Date of Birth") is equal to or greater than a threshold, and also has a portion where the image similarity with the first region C32 (the region with the item value "cccc") is equal to or greater than a threshold. As a result, it is determined that the first regions C11 and C12 are included in the second region C10, the first regions C21 and C22 are included in the second region C20, and the first regions C31 and C32 are included in the second region C30.

In step #5, the control unit 10 performs a region linking process. Specifically, the control unit 10 links multiple first regions contained in the same second region to each other.

In the examples shown in Figures 5 and 6, first regions C11 and C12 contained in second region C10 are linked to each other. First regions C21 and C22 contained in second region C20 are linked to each other. First regions C31 and C32 contained in second region C30 are linked to each other.

In step #6, the control unit 10 performs area determination processing. By performing area determination processing, the control unit 10 recognizes which of the multiple first areas linked to each other is an item name area and which is an item value area. In other words, by performing area determination processing, the control unit 10 distinguishes between item name areas and item value areas.

In the examples shown in Figures 5 and 6, of the second area C10, the first area C11 is determined to be the area for the item name, and the first area C12 is determined to be the area for the item value. Of the second area C20, the first area C21 is determined to be the area for the item name, and the first area C22 is determined to be the area for the item value. Of the second area C30, the first area C31 is determined to be the area for the item name, and the first area C32 is determined to be the area for the item value.

The control unit 10 recognizes the area of the target item by performing area determination processing. The area of the target item is a second area that encompasses both the first area corresponding to the item name of the target item and the first area corresponding to the item value of the target item. The area determination processing will be explained in detail later. Note that other methods may also be used to recognize the area of the target item.

In step #7, the control unit 10 performs a masking process to mask at least a portion of the area of the target item present in the image data of the identification card. By performing the masking process, the control unit 10 generates masked data in which at least a portion of the area of the target item present in the image data of the identification card is masked.

The control unit 10 masks the first area associated with the area of the item name of the target item (i.e., the area of the item value of the target item). Alternatively, the control unit 10 masks both the area of the item name and the area of the item value of the target item. Alternatively, substantially the entire area of the second area corresponding to the target item may be masked.

An example of masked data is shown in Figure 7. The masked data shown in Figure 7 is generated when the target item is date of birth. That is, at least the item value "cccc" corresponding to the item name "Date of Birth" is masked. Although not shown, both the item name "Date of Birth" and the item value "cccc" may be masked.

In step #8, the control unit 10 causes the output unit to perform output processing of the masked data. For example, in the masking job settings, the output method of the masked data can be selected. Output methods include printing and transmission.

When printing is selected as the output method, the control unit 10 causes the printing unit 1 to print (in other words, output) an image based on the masked data onto a sheet S. In this case, the printing unit 1 corresponds to the "output unit," and the output destination is the sheet S.

If transmission is selected as the output method, the control unit 10 causes the communication unit 102 to transmit (in other words, output) the masked data to the PC 1000. The masked data may be converted into PDF data and transmitted to the PC 1000. By transmitting the masked data to the PC 1000, the masked data can be saved on the PC 1000. In this case, the communication unit 102 corresponds to the "output unit," and the output destination is the PC 1000.

In this embodiment, a first area detection process, a second area detection process, an inclusion area detection process, and an area linking process are performed. This allows each area of the item name and item value corresponding to the same item in the image data of the identification card to be linked to each other with high accuracy.

By linking the fields of the item name and item value corresponding to the same item in the image data of the ID card to each other, it is possible to easily mask at least a portion of the target item. Specifically, it is sufficient to mask the field linked to the field of the item name of the target item. This allows the field value of the target item to be masked. With this configuration, it is possible to easily mask only the fields corresponding to the user-specified items in the image data obtained by scanning the document D (ID card) without transferring the image data of the ID card to a processing device on the cloud.

Masking the field of item values in the image data of an ID card can prevent the leakage of personal information. Even if the field of item names for target items is masked, personal information will be leaked if the field of item values for target items is not masked. Therefore, it is important to accurately link the field of item names and field values corresponding to the same item in the image data of an ID card.

Furthermore, in this embodiment, the first area detection process is performed using the first learning model, and the second area detection process is performed using the second learning model. Here, machine learning requires a huge amount of calculation and a huge amount of memory capacity. For this reason, machine learning processes are often performed by a processing device on the cloud. However, this requires transferring image data of the ID card to the processing device on the cloud, which is not desirable as it could result in the leakage of personal information.

In this embodiment, therefore, a trained first learning model and a trained second learning model are used. The first learning model and the second learning model are each pre-stored in the memory unit 101. This allows the first area detection process and the second area detection process to be performed within the multifunction device 100 without increasing the memory capacity within the multifunction device 100. In other words, there is no need to transfer image data of the identification card to a processing device on the cloud. As a result, the leakage of personal information can be suppressed.

<Area Determination Processing>
The control unit 10 performs a region determination process to determine whether each of a plurality of first regions (i.e., character regions containing character strings) present in the image data of the identification card is an area for an item name or an area for an item value. The region determination process uses dictionary data DD. The dictionary data DD is pre-stored in the storage unit 101 (see FIG. 2).

A conceptual diagram of dictionary data DD is shown in Figure 8. Dictionary data DD is data in which multiple different similar item names are predefined for one type of item. Dictionary data DD is created in advance for each of multiple types of items. The storage unit 101 pre-stores multiple pieces of dictionary data DD corresponding to multiple types of items. The multiple pieces of dictionary data DD are created in advance by the manufacturer of the multifunction device 100 and pre-stored in the storage unit 101. In Figure 8, items are indicated by A, B, etc. Similar item names corresponding to item A are indicated by a1, a2, a3, etc. Similar item names corresponding to item B are indicated by b1, b2, b3, etc.

Taking an identification number as an example of an item, one ID card might use the string "ID" as the item name, another might use the string "Number" as the item name, and yet another might use the string "Num" as the item name. Although the strings listed here are different from one another, they all correspond to the same item. Therefore, in the dictionary data DD corresponding to the identification number as an item, the string "ID," the string "Number," and the string "Num" are predefined as similar item names.

The flow of the area determination process will be explained below with reference to the flowchart shown in Figure 9. The flow shown in Figure 9 starts when the area linking process by the control unit 10 is completed.

In step #11, the control unit 10 performs character recognition processing on the image data of the ID card obtained by the image reading unit 2 reading the ID card (i.e., the processing of step #1 shown in Figure 4). The control unit 10 performs OCR processing on the image data of the ID card. As a result, the control unit 10 extracts character strings from each of the multiple first areas present in the image data of the ID card. The control unit 10 recognizes the character strings in each of the multiple first areas. Note that the first areas contain character strings indicating item names or item values.

In step #12, the control unit 10 performs preprocessing on the multiple character strings extracted by the character recognition process. There are no particular restrictions on the type of preprocessing performed. For example, the control unit 10 performs preprocessing by standardizing the case of the character strings. This process results in the character strings being indistinguishable from uppercase and lowercase letters. Furthermore, for example, the control unit 10 performs preprocessing by removing spaces and specified symbols.

After processing step #12, the control unit 10 performs item name area determination processing. When performing the item name area determination processing, the control unit 10 sets one of the multiple first areas detected in the first area detection processing as the processing target.

The control unit 10 then performs a similarity calculation process (step #13) as part of the item name region determination process. In step #13, the control unit 10 calculates the similarity between each character string of all similar item names defined in multiple dictionary data DD and the character string in the first region being processed. Hereinafter, the similarity between character strings will be referred to as character string similarity, to distinguish it from image similarity.

When performing the similarity calculation process, the control unit 10 selects one of the multiple dictionary data DD (here, the selected dictionary data DD is referred to as the target dictionary data DD). The control unit 10 also selects one of the multiple similar item names defined in the target dictionary data DD (here, the selected similar item name is referred to as the target similar item name).

The control unit 10 then calculates the similarity between the character string in the first region to be processed and the character string of the target similar item name. For example, the control unit 10 can calculate the similarity between the character string in the first region to be processed and the character string of the target similar item name by using the following formula (1):

In formula (1), S1 is one character string and S2 is the other character string. For example, S1 is the character string of the similar item name defined in dictionary data DD, and S2 is the character string of the first region extracted from the image data of an identification card. len(S1) is the number of characters in character string S1, and len(S2) is the number of characters in character string S2. M is the number of characters that match between character string S1 and character string S2.

For example, suppose character string S1 is "Number" and character string S2 is "Numbern." Even if "Number" is written on the ID card, it may be recognized as "Numbern" due to a misrecognition in the character recognition process.

In this example, len(S1) = 6 and len(S2) = 7. Furthermore, between strings S1 and S2, counting from the first character, the first character "N", the third character "m", the fourth character "b", the fifth character "e", and the sixth character "r" match each other. This results in M = 5.

In this example, the similarity value is 0.76923. If string S1 and string S2 contain completely different characters (i.e., M = 0), the similarity value will be 0, and if string S1 and string S2 contain exactly the same characters (i.e., M = len(S1) = len(S2)), the similarity value will be 1. In other words, when calculating the similarity between the string in the first region to be processed and the string of the target similar item name, the higher the similarity, the larger the similarity value, and the lower the similarity, the smaller the similarity value.

After determining the string similarity between the string in the first region to be processed and the string of the target similar item name, the control unit 10 newly selects an unselected similar item name from among the multiple similar item names defined in the target dictionary data DD. The control unit 10 switches the target similar item name and determines the string similarity between the string in the first region to be processed and the string of the new target similar item name. The control unit 10 determines the string similarity with the string in the first region to be processed for all of the multiple similar item names defined in the target dictionary data DD. The control unit 10 then stores the similar item name from the multiple similar item names defined in the target dictionary data DD that has the highest string similarity with the string in the first region to be processed, in association with its string similarity value.

The control unit 10 also calculates the string similarity between all similar item names defined in the multiple dictionary data DD and the string in the first region being processed. That is, the control unit 10 newly selects an unselected dictionary data DD from the multiple dictionary data DD. The control unit 10 switches the target dictionary data DD and calculates the string similarity between all of the multiple similar item names defined in the new target dictionary data DD and the string in the first region being processed. The control unit 10 then stores the similar item name, of the multiple similar item names defined in the target dictionary data DD, that has the highest string similarity with the string in the first region being processed, in association with its string similarity value. As a result, multiple similar item names corresponding to each of the multiple types of items are stored in association with a string similarity value. In other words, multiple similarity values (similarity values are string similarity values) corresponding to each of the multiple types of items are stored.

The control unit 10 performs a maximum value detection process (step #14) as part of the item name area determination process. In step #14, the control unit 10 compares the multiple similarity values (the similarity values are string similarity values) calculated and stored in the similarity calculation process. In other words, the control unit 10 compares the multiple similarity values corresponding to the multiple types of items.

The control unit 10 then detects the highest value of string similarity (i.e., the highest similarity value). That is, the control unit 10 calculates the string similarity between the string in the first region being processed and each string in all similar item names, and detects the highest value among the multiple similarity values thus calculated.

The control unit 10 performs a threshold comparison process (step #15) as part of the item name area determination process. In step #15, the control unit 10 compares a predetermined threshold value for the item name area determination process with the highest string similarity value.

For example, to set a threshold value for the item name area determination process, an identification card is actually read by the multifunction device 100, and the results of a similarity calculation process for character strings present in the item name area of the image data obtained from the read (here, the resulting value is referred to as the similarity value of the item name area) are obtained. In addition, the results of a similarity calculation process for character strings present in areas of the image data other than the item name area (here, the resulting value is referred to as the similarity value of other areas) are obtained. Note that multiple similarity values are obtained for the item name area, and multiple similarity values for other areas are also obtained.

In addition, a first average value, which is the average value of the similarity values of multiple item name regions, is calculated, and a second average value, which is the average value of multiple other regions, is calculated. The average value of the first average value and the second average value is then set as the threshold value for the item name region determination process.

In step #16, the control unit 10 determines whether the highest value of the character string similarity is equal to or greater than the threshold value for the item name area determination process. In Figure 9, the threshold value for the item name area determination process is represented as Th. If the control unit 10 determines that the highest value of the character string similarity is equal to or greater than the threshold value for the item name area determination process, the process proceeds to step #17.

In step #17, the control unit 10 determines that the first area to be processed is an area for an item name. Furthermore, in step #18, the control unit 10 determines the item corresponding to the first area to be processed based on the character string that serves as the item name present in the first area to be processed (i.e., the first area determined to be an area for an item name). Here, the character string in the first area to be processed is defined as a similar item name in one of the dictionary data DDs. Therefore, the control unit 10 recognizes the item corresponding to the dictionary data DD in which the character string in the first area to be processed is defined, and determines that the recognized item is the item corresponding to the first area to be processed.

If the control unit 10 determines in step #16 that the highest string similarity is less than the threshold value for the item name area determination process, the process proceeds to step #19. If the process proceeds to step #19, the control unit 10 determines that the first area to be processed is another area (i.e., an area different from the item name area).

Steps #13 to #19 are each performed as part of the item name area determination process. After completing the item name area determination process for a certain first area of a processing target, the control unit 10 determines whether any first areas remain for which the item name area determination process has not been performed (referred to here as unprocessed first areas). If any unprocessed first areas remain, the control unit 10 sets the unprocessed first area as the new processing target. In other words, the control unit 10 switches the processing target. The control unit 10 then performs the item name area determination process (steps #13 to #19) on the new first area of the processing target.

In this embodiment, dictionary data DD, in which multiple different similar item names for one type of item are predefined, is stored in advance in the storage unit 101 for each of multiple types of items. The item name area determination process is then performed using the dictionary data DD. This prevents the problem of an item being properly masked on an ID card that lists one of the multiple item names, while not being masked on an ID card that lists another item name (in other words, not detecting the item to be masked), when an item with multiple corresponding item names is to be masked.

For example, suppose an identification number is an item to be masked. In this case, in this embodiment, the identification number is masked correctly on both an ID card with "ID" written as the item name and an ID card with "Num" written as the item name. On the other hand, in conventional configurations, the identification number is masked correctly on an ID card with "ID" written as the item name, but the identification number is not masked on an ID card with "Num" written as the item name, which can be an inconvenience.

To prevent these problems from occurring, a large dictionary is required. Furthermore, when using machine learning models, the model size becomes large. However, because memory capacity is limited, processing is carried out on the cloud.

In this embodiment, by using dictionary data DD, processing on the cloud is unnecessary. That is, there is no need to transfer image data of the identification card (i.e., personal information) to a processing device on the cloud. This makes it possible to prevent the leakage of personal information. In other words, in this embodiment, it is possible to easily mask only the areas of the image data obtained by scanning the document D (identification card) that correspond to the items specified by the user, without transferring the image data of the identification card to a processing device on the cloud.

In addition, in this embodiment, the string similarity between the string in the first region to be processed and each string in all similar item names is calculated, and if the highest string similarity calculated is equal to or greater than a threshold, the first region to be processed is determined to be an item name region. Therefore, even if misrecognition occurs during character recognition processing, item name regions can be detected with high accuracy.

<Anomaly Countermeasure Processing>
First, the first abnormality countermeasure process will be described.

Normally, the multiple items of personal information listed on an ID card are different from each other. For example, it is highly unlikely that a single ID card will have multiple items with the name "Name" in them. However, if the accuracy of the character recognition process on the image data of the ID card is low, multiple item names (character strings thereof) that each correspond to the same item may be extracted from the image data of the ID card. The accuracy of the character recognition process will decrease if there is dirt or discoloration on parts of the ID card, or if the resolution used to read the ID card is low.

To prevent such problems from occurring, the control unit 10 performs a first abnormality countermeasure process. Specifically, when the character strings in multiple first areas determined to be item name areas correspond to the same first item, the control unit 10 determines that the first area containing the character string with the highest maximum value is the area corresponding to the first item, and determines that the remaining first areas do not correspond to the first item. In other words, the control unit 10 corrects the results of the item name area determination process.

The flow of the first abnormality countermeasure processing will be explained below with reference to the flowchart shown in Figure 10. The flow shown in Figure 10 starts when the item name area determination processing by the control unit 10 (i.e., processing according to the flow shown in Figure 9) is completed. After completing the item name area determination processing, the control unit 10 continues to perform the first abnormality countermeasure processing on the first area being processed.

In step #21, the control unit 10 determines whether the first area to be processed is determined to be an item name area. If the control unit 10 determines that the first area to be processed is an item name area, the process proceeds to step #22. If the control unit 10 determines that the first area to be processed is not an item name area, the process ends.

In step #22, the control unit 10 determines whether the item corresponding to the first area being processed is the same as the item corresponding to another first area. In other words, the control unit 10 determines whether the character strings in each of the multiple first areas determined to be item name areas correspond to the same item (first item).

If the control unit 10 determines in step #22 that the item corresponding to the first area to be processed is the same as the item corresponding to another first area, the process proceeds to step #23. In other words, if the control unit 10 determines that the character strings in each of the multiple first areas determined to be item name areas correspond to the same item (first item), the process proceeds to step #23. If the control unit 10 determines that the item corresponding to the first area to be processed is different from the item corresponding to another first area, the process ends. Hereinafter, another first area that has the same corresponding item as the first area to be processed will be referred to as the first area to be compared.

In step #23, the control unit 10 recognizes the highest value found in the highest value detection process (i.e., the process of step #14 shown in FIG. 9) for each of the first area to be processed and the first area to be compared. The control unit 10 then compares the highest values of the first area to be processed and the first area to be compared. As a result, the highest values of the first area to be processed and the first area to be compared are compared. For example, the control unit 10 determines whether the highest value M1 of the first area to be processed is greater than or equal to the highest value M2 of the first area to be compared.

If the control unit 10 determines in step #23 that the highest value M1 of the first region being processed is equal to or greater than the highest value M2 of the first region being compared, the process proceeds to step #24. If the control unit 10 determines that the highest value M1 of the first region being processed is less than the highest value M2 of the first region being compared, the process proceeds to step #25.

When proceeding to step #24, the control unit 10 does not change the determination result that the first region to be processed is the region corresponding to the first item, but changes the determination result for the first region to be compared. Specifically, the control unit 10 recognizes the second item, which is the item corresponding to the similar item name with the second highest string similarity to the string of the first region to be compared, and determines that the first region to be compared is the region corresponding to the second item.

When proceeding to step #25, the control unit 10 does not change the determination result that the first region to be compared is the region corresponding to the first item, but changes the determination result for the first region to be processed. Specifically, the control unit 10 recognizes the second item, which is the item corresponding to the similar item name with the second highest string similarity to the string in the first region to be processed, and determines that the first region to be processed is the region corresponding to the second item.

In other words, steps #23 to #25 perform the following processing:

When the character strings in multiple first regions determined to be regions of an item name correspond to the same first item, the control unit 10 determines, among the multiple first regions, the first region containing the character string with the highest maximum value as the region corresponding to the first item. The control unit 10 also recognizes, among the multiple first regions, a second item that corresponds to a similar item name with the second highest character string similarity to the character string in a first region not determined to be a region corresponding to the first item, and determines the first region not determined to be a region corresponding to the first item as the region corresponding to the second item.

Next, we will explain the second abnormality countermeasure process and the third abnormality countermeasure process.

Identification cards have multiple item columns. Each item column contains one item name, and it is rare for multiple item names to be written in a single item column. In other words, normally, two or more of the multiple first areas contained in the same second area are not determined to be item name areas. Also, normally, one of the multiple first areas contained in the same second area is determined to be an item name area. However, if the accuracy of the character recognition process for the image data of the identification card is low, it is possible that two or more of the multiple first areas contained in the same second area are determined to be item name areas, or that none of them are determined to be item name areas.

For this reason, the control unit 10 performs second abnormality countermeasure processing and third abnormality countermeasure processing. Here, multiple first areas contained in the same second area are linked to each other. Therefore, the control unit 10 performs second abnormality countermeasure processing or third abnormality countermeasure processing on multiple first areas that are linked to each other.

The following describes the flow of the second abnormality countermeasure processing and the third abnormality countermeasure processing, with reference to the flowcharts shown in Figures 11 to 13.

First, the control unit 10 performs processing according to the flow shown in FIG. 11. The flow shown in FIG. 11 starts when the area determination processing by the control unit 10 (i.e., processing according to the flow shown in FIG. 9) is completed. After completing the item name area determination processing, the control unit 10 continues processing according to the flow shown in FIG. 11.

In step #31, the control unit 10 performs a string acquisition process to acquire the string of characters in the first region linked to the first region to be processed. Then, in step #32, the control unit 10 determines whether or not the region determination process has been performed on the first region containing the string acquired in the string acquisition process (i.e., whether or not the region has been determined). If it determines that the region has been determined, the control unit 10 proceeds to step #33; if it determines that the region has not been determined, the control unit 10 ends this flow.

In step #33, the control unit 10 determines whether the first area to be processed and the first area linked to the first area to be processed are both determined to be item name areas. In other words, the control unit 10 determines whether two or more of the multiple first areas linked to each other are determined to be item name areas.

If the control unit 10 determines in step #33 that two or more of the multiple first areas linked to each other have been determined to be item name areas, the process proceeds to step #34. If the process proceeds to step #34, the control unit 10 performs second abnormality countermeasure processing.

If the control unit 10 determines in step #33 that two or more of the multiple first areas linked to each other have not been determined to be item name areas, the process proceeds to step #35. In step #35, the control unit 10 determines whether the first area to be processed and the first area linked to the first area to be processed have both been determined to be areas different from item name areas. In other words, the control unit 10 determines whether all of the multiple first areas linked to each other have been determined to be areas different from item name areas.

If the control unit 10 determines in step #35 that all of the linked first areas are determined to be areas different from the item name area, the process proceeds to step #36. If the process proceeds to step #36, the control unit 10 performs a third abnormality countermeasure process.

In step #35, if the control unit 10 does not determine that all of the multiple first areas linked to each other are areas other than item name areas, this flow ends. In other words, if only one of the multiple first areas linked to each other is determined to be an item name area, neither the second abnormality countermeasure process nor the third abnormality countermeasure process is performed.

The second abnormality countermeasure process is performed according to the flow shown in FIG. 12. That is, if the control unit 10 determines that two or more of the multiple first areas linked to each other (i.e., multiple first areas contained in the same second area) are item name areas, the process is performed according to the flow shown in FIG. 12.

In step #41, the control unit 10 recognizes the highest value found in the highest value detection process (i.e., the process of step #14 in Figure 9) for each of the multiple first regions that are linked to each other. In step #42, the control unit 10 recognizes the highest of the highest values.

Then, in step #43, the control unit 10 determines that, of the multiple first regions linked to each other, the first region containing the character string with the highest maximum value (here referred to as the first region with the highest maximum value) is an item name region. Furthermore, in step #44, the control unit 10 determines that, of the multiple first regions linked to each other, the remaining first regions other than the first region with the highest maximum value are regions other than item name regions.

Here, if a first area present in the image data of the ID card is determined to be an area for an item name, the control unit 10 stores information indicating that the first area corresponds to the item name. In this configuration, when the second abnormality countermeasure process is performed, for first areas that are re-determined to be areas other than the area for an item name, the information indicating that they correspond to the item name should be deleted. Therefore, in step #45, the control unit 10 deletes the information indicating that the remaining first areas, other than the first area with the highest maximum value, correspond to the item name.

The third abnormality countermeasure process is performed according to the flow shown in FIG. 13. That is, if the control unit 10 determines that all of the multiple first areas linked to each other (i.e., multiple first areas contained in the same second area) are areas different from the area of the item name, the process is performed according to the flow shown in FIG. 13.

In step #51, the control unit 10 recognizes the highest value found in the highest value detection process (i.e., the process of step #14 in Figure 9) for each of the multiple first regions that are linked to each other. In addition, in step #52, the control unit 10 recognizes the highest of the highest values.

Then, in step #53, the control unit 10 determines that, of the multiple first regions linked to each other, the first region containing the character string with the highest maximum value (here referred to as the first region with the highest maximum value) is an item name region. Furthermore, in step #54, the control unit 10 determines that, of the multiple first regions linked to each other, the remaining first regions other than the first region with the highest value are regions other than item name regions.

Here, if a first area present in the image data of the ID card is determined to be an area for an item name, the control unit 10 stores information indicating that the first area is an area corresponding to the item name. In this configuration, when the third abnormality countermeasure process is performed, information indicating that the first area corresponds to the item name should be added to the first area that is re-determined as an area for an item name by that process. Therefore, in step #55, the control unit 10 adds information indicating that the first area with the highest maximum value is an area corresponding to the item name.

In this embodiment, by performing the first to third abnormality countermeasure processes, it is possible to prevent the occurrence of the inconvenience of multiple detections of areas with item names corresponding to the same item. It is also possible to prevent the occurrence of the inconvenience of no detection of an item name area.

<Masking job settings>
The masking job can be set by the user via the operation and display unit 3. For example, the user can specify the item corresponding to the item value to be masked (i.e., the target item).

Specifically, the operation display unit 3 accepts an item designation operation to designate one of the items. The control unit 10 recognizes the item designated by the item designation operation as the target item. Then, as a masking process, the control unit 10 performs a process of masking at least the item value area of the target item present in the image data of the identification card to generate masked data.

The following describes the flow of processing performed by the control unit 10 when accepting settings for a masking job, with reference to the flowchart shown in Figure 14. For example, in a masking job, after the image reading unit 2 reads an ID card, the operation display unit 3 accepts item specification operations before the masking process begins. In other words, the flow shown in Figure 14 starts after the processing of step #1 shown in Figure 4 and before the processing of step #7.

In step #61, the operation display unit 3 displays a reception screen 31 as shown in FIG. 15. Reception buttons RB are arranged on the reception screen 31. The reception buttons RB include a basic settings button B1 and a custom button B2. The operation display unit 3 accepts an operation on any of the reception buttons RB as an item designation operation. The control unit 10 determines the user-designated item (i.e., the target item) based on which reception button RB the item designation operation was performed on.

Specifically, the storage unit 101 stores a masking list in which at least one item is registered. The reception button RB is associated with the masking list. If there are multiple masking lists, multiple reception buttons RB (specifically, the same number as the number of masking lists) are displayed on the reception screen 31. The multiple reception buttons RB are associated with different masking lists.

The masking list corresponding to the basic settings button B1 contains items pre-selected by the manufacturer. The masking list corresponding to the custom button B2 contains items arbitrarily selected by the user. By creating a new masking list, the custom button B2 (i.e., the reception button RB) corresponding to the newly created masking list will be displayed on the reception screen 31.

In order to receive a request from the user to create a new masking list, a new creation button NB is displayed on the reception screen 31. The operation display unit 3 receives an operation on the new creation button NB as a request to create a new masking list.

When a request to create a new masking list is received, the operation display unit 3 displays a registration screen 32 as shown in FIG. 16 and receives a registration operation from the user to select an item to be registered. The registration screen 32 displays multiple items (their item names) as options.

The operation display unit 3 accepts an operation on the display field of any of the displayed items as a registration operation. For example, if you want to register the item "height," you can select the item "height" as the item to be registered by operating the display field for the item "height." A check mark is placed in the display field of the item selected in the registration operation. You can complete the selection of the item through the registration operation by operating the confirm button DB on the registration screen 32.

When an operation is performed on the decision button DB, the control unit 10 determines that acceptance of the registration operation has been completed. After accepting the registration operation, the control unit 10 generates a new masking list in which the items selected in the registration operation are registered, and stores the masking list in the memory unit 101.

When a new masking list is created and the operation display unit 3 subsequently accepts an item designation operation, the control unit 10 causes the operation display unit 3 to display a reception screen 31 on which an acceptance button RB corresponding to the new masking list is arranged in addition to the acceptance buttons RB corresponding to the existing masking list.

When any of the reception buttons RB is operated, the control unit 10 determines that the operation display unit 3 has accepted an item specification operation. At this time, the control unit 10 determines that the masking list corresponding to the reception button RB for which the item specification operation was performed has been specified. In other words, the item specification operation is an operation for specifying any of the masking lists.

Returning to FIG. 14, in step #62, the control unit 10 recognizes the reception button RB on which the item designation operation was performed, i.e., the masking list designated in the item designation operation. The control unit 10 then recognizes the item registered in the masking list designated in the item designation operation as the target item.

In step #63, the control unit 10 performs masking processing. The processing in step #63 is the same as the processing in step #7 shown in Figure 4. Note that the control unit 10 performs masking processing after performing each of the processing in steps #2 to #6 shown in Figure 4.

In step #64, the control unit 10 causes the operation display unit 3 to display a preview image (not shown) showing the contents of the masked data, and also causes the operation display unit 3 to accept editing operations to edit the masked data. Editing operations are operations that change the items to be masked. In other words, editing operations include operations that add a new item to be masked, and operations that remove a masked item from the masking targets.

In step #65, the control unit 10 determines whether an editing operation has been performed. If an editing operation has been performed, the process proceeds to step #61. After this, the process proceeds to step #63, where the control unit 10 generates new masked data that reflects the edited content of the editing operation.

If no editing operation has been performed in step #65, the process proceeds to step #66. In step #66, the control unit 10 causes the output unit to perform output processing of the masked data. The processing in step #66 is the same as the processing in step #8 shown in Figure 4.

In this embodiment, after scanning the ID card, an item designation operation is received to specify the items to be masked (i.e., target items). By performing the item designation operation, at least the item value area of the target items present in the image data of the ID card is masked. In other words, it is easy to mask only the areas of the image data obtained by scanning the document D (ID card) that correspond to the items designated by the user.

For example, with conventional configurations, it is necessary to scan an ID card in advance and register the area to be masked (its position within the image data). In other words, the ID card must be pre-registered. With conventional configurations, if the ID card is pre-registered, it is possible to mask the desired area within the image data of the ID card.

However, from the user's perspective, the task of pre-registering ID cards is cumbersome. If there are multiple types of ID cards that can be used, pre-registration of each type of ID card must be performed, which increases the amount of work. Furthermore, the masking target cannot be changed flexibly. If a user wants to change the masking target, they must re-register the ID card, which is cumbersome.

In contrast, in this embodiment, when a certain ID card is scanned, the area of the image data corresponding to the item specified in the item specification operation is masked by performing an item specification operation. Also, when another ID card of a different type is scanned, the area of the image data corresponding to the item specified in the item specification operation is masked. Furthermore, for example, if the masking target is item A and it is necessary to add item B as a masking target (i.e., it is necessary to change the masking target), it is sufficient to simply specify items A and B in the item specification operation. This eliminates the need for work equivalent to pre-registering an ID card, improving user convenience.

Furthermore, in this embodiment, the item specification operation is an operation for specifying a masking list. As a result, if you want to mask multiple items, you do not need to specify each item you want to mask one by one. This further improves user convenience.

In addition, in this embodiment, new masking lists can be created. This is convenient for users, as they can create masking lists as they wish.

In addition, in this embodiment, masked data can be edited, improving user convenience. When editing masked data, a preview image showing the contents of the masked data is displayed. This allows the masked data to be previewed while being edited, making editing the masked data more efficient.

The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not by the description of the above embodiments, and further includes all modifications within the meaning and scope of the claims.

1. Printing unit (output unit)
2 Image reading unit 3 Operation display unit 10 Control unit 100 Multifunction device (image processing device)
102 Communication unit (output unit)
1000 PCs (external equipment)
D Manuscript S Sheet

Claims (3)

an image reading unit that reads a document on which a plurality of sets of item names and item values of items related to personal information are written;
a control unit that recognizes a target item that is an item to be masked, and performs a masking process on image data obtained by reading the document by the image reading unit, thereby generating masked data in which at least a portion of an area of the target item present in the image data is masked;
an operation display unit that displays information and accepts operations;
the operation display unit accepts an item designation operation for designating any one of the items;
the control unit recognizes the item specified by the item specifying operation as the target item, and performs, as the masking process, a process of masking at least an area of the item value among areas of the target item present in the image data to generate the masked data;
the control unit performs a first area detection process to detect a first area that is predicted to individually include either an area of the item name or an area of the item value from the image data;
When the document is read, a plurality of first areas are detected in the first area detection process,
If only one first area is detected as a result of the first area detection process, the control unit stops the masking job and displays a message on the operation display unit prompting the user to check the document set in the image reading unit. The image processing device according to claim 1, wherein, if the first area is not detected as a result of the first area detection process, the control unit stops the job and displays a message on the operation display unit prompting the user to check the document set in the image reading unit. an output unit that performs output processing of the masked data,
3. The image processing device according to claim 1, wherein the output unit is at least one of a printing unit that performs the output process of printing an image based on the masked data onto a sheet, and a communication unit that performs the output process of transmitting the masked data to an external device.