JP6926536B2 - Image processing equipment and programs - Google Patents

Description

The present invention relates to an image processing apparatus and a program.

In recent years, an image processing device having a function of reading a business card image with a scanner and managing the business card has been proposed (see, for example, Patent Document 1).

The image processing apparatus described in Patent Document 1 includes a first image acquisition unit that acquires the front surface of a business card as first image data, a second image acquisition unit that acquires the back surface as second image data, and a first image data. Based on the business card image acquisition unit that extracts the front side business card image from, generates and acquires the position information, extracts the back side business card image from the second image data, and generates and acquires the position information, and the front and back position information. The business card image determination unit that determines the business card image by associating the back surface business card image with each front surface business card image, the first determination unit that determines whether the front and back business card images are at the same position, and the front and back business card images are the same image. A display control unit that controls an image to be displayed on a display screen based on a result of determination using at least one of a second determination unit and a first determination unit and a second determination unit. And a storage unit for storing a business card image.

Japanese Unexamined Patent Publication No. 2015-056794

One of the problems of the present invention is to provide an image processing apparatus and a program capable of outputting the front surface and the back surface in association with each other even when the front surface and the back surface of the reading target are read in different arrangements. be.

[1] The first surface image is acquired from the first scanned image including the first surface to be read, and the second surface image is obtained from the second scanned image including the second surface to be read. The acquisition method to acquire and
The first surface image and the second surface image associated with the first surface image and the second surface image are associated with the degree of showing the relationship between the feature information of the first surface image acquired by the acquisition means and the feature information of the second surface image. and an output means for outputting the set,
The output means has feature information including the position of the component of the first surface image in the first surface image and the second component of the second surface image as a degree of showing the relevance. An image processing device that outputs the degree of matching with feature information including a position in a surface image.
[2] The image processing apparatus according to the above [1], further comprising a display means for displaying the first surface image or the second surface image by changing the display mode according to the degree.
[3] The image processing apparatus according to [2], wherein the display means displays the first surface image or the second surface image in an order according to the degree.
[4] The image processing apparatus according to the above [2] or [3], wherein the display means emphasizes and displays the first surface image or the second surface image according to the degree.
[ 5 ] The image processing apparatus according to [1 ], wherein the output means comprehensively evaluates the degree of matching for each component of an image acquired by the acquisition means and outputs the degree.
[6] Symbol component information indicating the position of a character and a character, or a information indicating the position of the graphical information and graphic information, the image processing apparatus according to [1] or [5].
[ 7 ] Computer
Acquisition to acquire the first surface image from the first scanned image including the first surface to be read and to acquire the second surface image from the second scanned image including the second surface to be read. Means and
The first surface image and the second surface image associated with the first surface image and the second surface image are associated with the degree of showing the relationship between the feature information of the first surface image acquired by the acquisition means and the feature information of the second surface image. It is a program to function as an output means to output a set of
The output means has feature information including the position of the component of the first surface image in the first surface image and the second component of the second surface image as a degree of showing the relevance. A program that outputs the degree of matching with feature information including the position in a surface image.

According to the inventions of claims 1 and 7 , even when the front surface and the back surface of the reading target are read in different arrangements, the operator can perform the front surface and the back surface having a high degree of association based on an objective index. It becomes possible to recognize the combination with and, and it becomes possible to output the front surface and the back surface in association with each other.
According to the inventions of claims 2, 3 and 4, the operator can visually recognize the combination of the front surface and the back surface in descending order of the degree of association .
According to請Motomeko 5, according to the sixth invention, it is possible to improve the accuracy of the degree of the front and back surfaces and association.

FIG. 1 is a diagram showing a configuration example of an image processing system according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of a control system of the image processing device according to the present embodiment. FIG. 3 is a diagram showing an example of a scanned image. FIG. 4 is a diagram showing an example of a rectangular image. FIG. 5 is a diagram showing an example of a result table. FIG. 6 is a diagram showing an example of displaying the result. FIG. 7 is a diagram showing an example of displaying a set of rectangular images. FIG. 8 is a flowchart showing an example of the operation of the image processing apparatus according to the present embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each figure, elements having substantially the same function are designated by the same reference numerals, and duplicate description thereof will be omitted.

[Summary of Embodiment]
The image processing apparatus according to the present embodiment acquires the first surface image from the first scanned image including the first surface to be read, and the second scanned image including the second surface to be read. The first aspect associated with the acquisition means for acquiring the second surface image from, and the degree of showing the relationship between the feature information of the first surface image acquired by the acquisition means and the feature information of the second surface image. It is provided with an output means for outputting a set of a surface image and a second surface image.

The reading target includes, for example, a personal authentication medium such as a business card or a driver's license. The reading target mainly has a rectangle, but may have a square. Further, the reading target includes a reading target having four rounded corners. Hereinafter, the reading target will be described as a business card.

The scanned image is an image obtained by scanning the scanned object. The surface image is an image of an area surrounded by the outline of the reading target, which includes any surface of the reading target. The surface image mainly has a rectangular shape.

The first surface may be one surface to be read and the other surface that has been read first, and the second surface may be a surface that has been read after the first surface. When the front surface and the back surface of the reading target can be defined, for example, the first surface may be the front surface of the reading target and the second surface may be the back surface of the reading target. When the front surface and the back surface of the reading target are described in different languages, the surface described in either language (for example, Japanese) may be used as the front surface.

The "feature information of a surface image" refers to a feature amount (for example, color, size, etc.) that objectively quantifies the features of the constituent elements constituting the surface image, such as character information, graphic information, and information on the ground.

Character information refers to information indicating the position of a character in addition to information indicating the attribute (size, character type, etc.) of the character. The character information includes, for example, affiliation information such as company name and department name, personal information such as job title, title, and name, contact information such as telephone number, fax number, e-mail address, and company address.

The graphic information refers to information indicating the graphic itself, information indicating a design including the graphic and characters, and information indicating the positions thereof. The graphic information includes, for example, a logo mark indicating a trade name or an organization name, a graphic including a catch phrase or a slogan, a symbol mark indicating a registered qualification or a certified standard, a photograph or a portrait indicating the owner, etc. included.

In the following, information indicating the position of characters and information indicating the position of information such as figures are collectively referred to as "layout".

The "degree of relevance" refers to an index indicating how well the above-mentioned feature information between different images matches.

The "associated set of the first surface image and the second surface image" refers to a pair of surface images having the same relationship between the first surface and the second surface of the reading target.

[Embodiment]
FIG. 1 is a diagram showing a configuration example of an image processing system according to the present embodiment. This image processing system includes an image processing device 1, an image forming device 2, and a terminal device 3, and the image processing device 1, the image forming device 2, and the terminal device 3 have, for example, a network 4 which is a public network. It is configured to be connected so that it can communicate via.

The image forming apparatus 2 may be a multifunction device having a plurality of functions such as copying, scanning, printing, and faxing, or an image forming apparatus 2 having only a scanning function, as long as it has a scanning function. The image forming apparatus 2 includes a document stand (not shown) on which documents are arranged, and an operation display unit 21 for inputting and displaying information.

For the terminal device 3, for example, a personal computer, a tablet terminal, a multifunctional mobile phone (smartphone), or the like can be used. The terminal device 3 includes, for example, an input unit 31 realized by a keyboard, a mouse, or the like, and a display unit 32 realized by, for example, a liquid crystal display or the like.

The network 4 is a communication network such as a LAN (local area network) or the Internet that transmits / receives data by wired communication, wireless communication, or the like. The network 4 may be, for example, Wi-Fi using a Wi-Fi router.

(Configuration of image processing device)
FIG. 2 is a block diagram showing an example of a control system of an image processing device. The image processing device 1 communicates between the control unit 10 that controls each part of the image processing device 1, the storage unit 11 that stores various data, and the image forming device 2 and the terminal device 3 via the network 4. The communication unit 12 is provided.

The control unit 10 is composed of a CPU (Central Processing Unit), an interface, and the like. By operating according to the program stored in the program storage unit 111, the CPU functions as a reception means 100, an acquisition means 101, an extraction means 102, a classification means 103, a calculation means 104, an output means 105, a display means 106, and the like. The classification means 103 and the calculation means 104 are examples of output means. Details of each means 100 to 106 will be described later.

The storage unit 11 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, and the like, and includes a program storage unit 111 for storing a program, an image data storage unit 112 for storing image data, and the like.

The communication unit 12 transmits and receives signals to and from the image forming apparatus 2 and the terminal apparatus 3 via the network 4.

Next, each means 100 to 106 will be described with reference to FIGS. 3 to 7. The image forming apparatus 2 optically reads a scanning object arranged on a platen (not shown), forms a scanned image, and passes the scanned image to the receiving means 100.

The receiving means 100 receives the scanned image formed by the image forming apparatus 2. Specifically, the receiving means 100 receives the first scanned image 20a and the second scanned image 20b. The reception means 100 stores the received read image in the image data storage unit 112.

3A and 3B are views showing an example of a scanned image, FIG. 3A is a diagram showing an example of a first scanned image, and FIG. 3B is a diagram showing an example of a second scanned image. As shown in FIG. 3A, the first read image 20a includes the first surface 200A of the reading target 200. Further, as shown in FIG. 3B, the second scanned image 20b includes the second surface 200B of the reading target 200. The numbers such as "A1" and "B1" in FIGS. 3A and 3B indicate IDs that identify the reading target 200. As shown in FIGS. 3A and 3B, the first scanned image 20a and the second scanned image 20b each include a plurality of (for example, seven) reading targets 200.

It should be noted that the first surface 200A of the plurality of reading targets 200 is not necessarily limited to one surface (for example, a surface) of the reading target 200, and one surface and the other surface (for example). For example, the back surface) may be mixed. Similarly, one surface and the other surface may be mixed in the second surface 200B.

(Structure of reception means)
The reception means 100 receives an operation input by the operator to the input unit 31 of the terminal device 3. The operation by the operator may be input to the operation display unit 21 of the image forming apparatus 2.

(Structure of acquisition means)
The acquisition means 101 performs a process (hereinafter, also referred to as “edge enhancement process”) of detecting the edge portion of the read target and extracting the contour line of the read target on the read image received by the reception means 100. A rectangular image surrounded by a contour line to be read (hereinafter, also referred to as a "rectangular image") is acquired. Specifically, the acquisition means 101 performs edge processing on the first scanned image 20a, and obtains a first rectangular image 200a corresponding to the first surface 200A of the read target 200 from the first scanned image 20a. At the same time, edge processing is performed on the second scanned image 20b, and a second rectangular image 200b corresponding to the second surface 200B of the scanning target 200 is acquired from the second scanned image 20b. The first rectangular image 200a is an example of a first surface image, and the second rectangular image 200b is an example of a second surface image.

(Structure of extraction means)
The extraction means 102 performs a process of extracting the outlines of characters and figures from the acquired rectangular image and recognizing the character information and the graphic information (hereinafter, “image recognition process”). For the image recognition process, for example, OCR (Optical Character Recognition) processing, OMR (Optical Mark Recognition), or the like is used.

Specifically, the extraction means 102 performs image recognition processing on the first rectangular image 200a, extracts the feature information of the first rectangular image 200a, and recognizes the image on the second rectangular image 200b. Processing is performed to extract the feature information of the second rectangular image 200b.

The feature information extracted by the extraction means 102 includes the size of the first rectangular image 200a, the components included in the first rectangular image 200a, the layout of the first rectangular image 200a, and the second rectangular image 200b. Includes size, components included in the second rectangular image 200b, and layout of the second rectangular image 200b.

The components include character information and graphic information. FIG. 4 is a diagram showing an example of a rectangular image. As shown in FIGS. 4A to 4F, the extraction means 102 has a company name 201, a department name 202, a name 203, a telephone number 204 as a direct contact, and a representative contact as character information. The telephone number 205 and the e-mail address 207 of the above are extracted. Further, the extraction means 102 extracts the logo mark 206 as graphic information.

The extraction means 102 determines the direction of the reading target 200 from the recognized character information, and if the reading target 200 is tilted, for example, a rotation process for rotating the rectangular image around a predetermined point or a rectangular image. The tilt of the rectangular image may be corrected by a tilt process or the like.

(Structure of classification means)
The classification means 103 classifies a rectangular image according to its size in the scanned image. Specifically, the classification means 103 classifies the first rectangular image 200a by size in the first scanned image 20a, and classifies the second rectangular image 200b by size in the second scanned image 20b. Classify.

FIG. 3 shows an example of the classification results. As shown in FIG. 3A, in the first read image 20a, the plurality of reading targets 200 have a first size (see the broken line frame 210) and a second size (dotted line frame). (See 211), those having a third size (see one-dot chain line frame 212), and the like. Further, as shown in FIG. 3B, in the second read image 20b, the plurality of read targets 200 have the first size (see the numbers "B1", "B3", and "B5"), and the second one. It is classified into those having a size of 2 (see numbering "B2", "B4", "B6"), those having a third size (see numbering "B7"), and the like.

The rectangular image is not limited to being classified by size, and may be classified by the shape of the rectangular image, for example.

(Structure of calculation means)
The calculation means 104 calculates the degree of matching between the components of the rectangular images classified into the same size in different scanned images. Specifically, the calculation means 104 calculates the degree of matching between the first rectangular image 200a and the second rectangular image 200b classified into the same size. The degree of matching between the components includes the degree of matching of character information, the degree of matching of layout, and the degree of matching of graphic information. Hereinafter, a specific description will be given.

The degree of matching of character information means the degree to which the character information extracted by the extraction means 102 matches between rectangular images classified into the same size in different scanned images. Specifically, the characters that make up the component (for example, name 203) extracted in the first rectangular image 200a and the characters that make up the same component (name 203) extracted in the second rectangular image 200b. Refers to the ratio of matching with each other, and is represented by a value in the range of 0 to 1, with 1 being a case of perfect matching. When the language described in the first rectangular image 200a and the language described in the second rectangular image 200b are different from each other, the calculation means 104 converts in advance for a specific character string. Keep a table and refer to it. For example, if one of the first and second rectangular images is Japanese and the other is English, a conversion table is held so that "Sales Department" and "Sales Department" are converted to each other as a specific character string. Will be done.

The degree of matching may be calculated by converting to either language. In the process of converting to either language, for example, when the display order of the surname and the first name is different between different languages, or the display order of the prefecture, city, ward, town, village, street address, etc. of the address is different from each other. In different cases, it involves converting to the order in which they are displayed in one language.

The layout matching degree is determined by the arrangement of characters constituting the character information extracted by the extraction means 102 among the rectangular images classified into the same size in different scanned images, and the position of the graphic information extracted by the extraction means 102. Refers to the ratio of matching with each other, and is represented by a value in the range of 0 to 1, with 1 being the case of perfect matching.

The degree of matching of graphic information refers to the degree to which the graphic information extracted by the extraction means 102 matches between rectangular images classified into the same size in different scanned images. Specifically, it refers to the ratio at which the graphic element (for example, the logo mark 206) extracted by the first rectangular image 200a and the graphic element (logo mark 206) extracted by the second rectangular image 200b match each other. For example, it is represented by a value in the range of 0 to 1, assuming that the case is exactly the same. For example, a pattern matching process may be used to calculate the degree of graphic information.

In the example shown in FIGS. 3A and 3B, the calculation means 104 has a reading target 200 having IDs "A1", "A2", and "A3" and a reading target having IDs "B1", "B3", and "B5". The above-mentioned degree of matching with 200 is calculated. The calculation means 104 calculates the above-mentioned degree of matching between the reading target 200 having the IDs "A4", "A5", and "A6" and the reading target 200 having the IDs "B2", "B4", and "B6". The calculation means 104 calculates the above-mentioned degree of matching between the reading target 200 having the ID “A7” and the reading target 200 having the ID “B7”.

Hereinafter, as an example, the degree of matching between the reading target 200 having the IDs "A1", "A2", and "A3" shown in each figure of FIG. 3 and the reading target 200 having the IDs "B1", "B3", and "B5". Will be described in detail by taking as an example. FIG. 4A is a diagram showing an example of a reading target 200 having an ID of “A1” in FIG. 3A, and FIG. 4B is a reading target 200 having an ID of “A2” in FIG. 3A. FIG. 4 (c) shows an example of a reading target 200 having an ID of “A3” in FIG. 3 (a), and FIG. 4 (d) shows an example of a reading target 200 having an ID of “A3” in FIG. 3 (a). FIG. 4 (e) shows an example of the reading target 200 of "B1", FIG. 4 (e) shows an example of the reading target 200 of FIG. 3 (b) having the ID "B3", and FIG. 4 (f) shows FIG. 3 (f). It is a figure which shows an example of the reading target 200 which the ID of b) is "B5".

The calculation means 104 calculates the above-mentioned degree of matching between the reading target 200 having the ID "A1" and the reading target 200 having the IDs "B1", "B3", and "B5". Next, the calculation means 104 calculates the above-mentioned degree of matching between the reading target 200 having the ID "A2" and the reading target 200 having the IDs "B1", "B3", and "B5". Next, the calculation means 104 calculates the above-mentioned degree of matching between the reading target 200 having the ID "A3" and the reading target 200 having the IDs "B1", "B3", and "B5".

The calculation means 104 calculates the degree C indicating the relationship between the feature information of the first rectangular image 200a and the feature information of the second rectangular image 200b. The degree C indicating this relevance is an index calculated by comprehensively evaluating the above-mentioned three degree of matching. Specifically, when the matching degree of the above-mentioned character information is Rc, the matching degree of the layout is Rl, and the matching degree of the graphic information is Rg, the calculation means 104 shows the degree of relevance by using the following formula. Find C.
C = αxRc + βxRl + γxRg,
Coefficient α: A value indicating the weighting of the matching degree Rc of character information,
Coefficient β: A value indicating the weighting of the layout matching degree Rl,
Coefficient γ: A value indicating the weighting of the matching degree Rg of graphic information

FIG. 5 is an example of the result table 230 showing the result of the calculated degree of matching. In the "reading target ID" column, the reading target IDs corresponding to the pair of rectangular images for which the degree of matching has been calculated are shown. In the "character information" column, the degree of matching Rc of the character information is shown. In the "Layout" column, the degree of matching Rl of the layout is shown. In the column of "graphic information", the degree of matching Rg of the graphic information is shown. In the column of "degree of relevance", the degree C indicating relevance is shown. The degree of relevance is the set of the reading target 200 having the ID "A1" and the reading target 200 having the ID "B5" (hereinafter, also simply referred to as "A1-B5"), A2-B1, and A3-B3. And it is a relatively high value.

The calculation means 104 determines whether or not the rectangular image has graphic information in both of the pair of rectangular images for which the degree of matching has been calculated. Specifically, the calculation means 104 determines whether or not there is graphic information in both the first rectangular image 200a and the second rectangular image 200b. For example, when the degree of matching of the above-mentioned graphic information is equal to or less than a predetermined threshold value (for example, 0.01), it may be determined that there is no graphic information. In the example shown in FIG. 5, the reading target 200 having the ID "B5" has a degree of matching of graphic information of 0.01 or less with all of the reading targets 200 having the IDs "A1", "A2", and "A3". Therefore, it is determined that the read target 200 having the ID "B5" has no graphic information.

If neither the first rectangular image 200a nor the second rectangular image 200b has graphic information, the calculation means 104 sets the coefficient γ related to the degree of matching of the graphic information to a predetermined value (for example, 0 or a value close to 0). ). The calculation means 104 calculates the degree C indicating the relevance based on the above-mentioned calculation formula by using the changed new coefficient.

(Configuration of output means)
The output means 105 outputs the calculated result described above to the display means 106. The output means 105 outputs the set of the associated first rectangular image 200a and the second rectangular image 200b to the display means 106. The pair of the associated first rectangular image 200a and the second rectangular image 200b may be, for example, a pair of the first rectangular image 200a and the second rectangular image 200b having the largest value of the degree C indicating the relevance. good.

(Structure of display means)
The display means 106 displays the output result on the display unit 32 of the terminal device 3 together with the first rectangular image 200a and the second rectangular image 200b. Specifically, the display means 106 changes the display mode (hereinafter, also referred to as “display mode”) according to the degree C indicating the relevance, and changes the display mode to the first rectangular image 200a and the second rectangular image 200b. Is displayed on the display unit 32 of the terminal device 3. The display means 106 may output the above-mentioned calculation result to the operation display unit 21 of the image forming apparatus 2.

6A and 6B are diagrams showing an example of displaying the calculated result, FIG. 6A is a diagram showing an example of a list of first thumbnails of the first rectangular image of the first rectangular image of the reading target 200, and FIG. 6B is a diagram showing an example. A diagram showing an example of a list of second thumbnails of a second rectangular image of a second rectangular image of a reading target 200, (c) a diagram showing an example of a matching degree table, and (d) a diagram showing an example of a highlighting legend. Is.

As shown in FIG. 6A, the display means 106 schematically shows the first rectangular image 200a of the plurality of reading targets 200 on the display surface 320 of the display unit 32 of the terminal device 3 (hereinafter, the image 240a). , Also referred to as "first thumbnail 240a"). The first thumbnail 240a is an example of a first surface image. An ID that identifies the corresponding reading target 200 is attached to the first thumbnail 240a.

As shown in FIG. 6B, the display means 106 schematically shows the second rectangular image 200b on the display surface 320 of the display unit 32 of the terminal device 3 (hereinafter, “second thumbnail”). 240b ") is displayed. The second thumbnail 240b is an example of a second surface image. The second thumbnail 240b is assigned an ID that identifies the corresponding reading target 200.

The display means 106 is operated by an operation to display between the display of the list of the first thumbnail 240a shown in FIG. 6A and the display of the list of the second thumbnail 240b shown in FIG. 6B. Make a switch.

The display means 106 has a reading target 200 corresponding to one of the plurality of first thumbnails 240a shown in FIG. 6A and a reading target 200 corresponding to the second thumbnail 240b shown in FIG. 6B. Shows the degree of relevance to. As shown in FIG. 6C, the matching degree table 250 is provided with a “reading target ID” column and a “matching degree” column, and the “matching degree” column is shown in FIG. 6 (a). The degree (C) indicating the relationship between the first thumbnail 240a with the ID "A01" and the plurality of second thumbnails 240b shown in FIG. 6B is shown.

The display means 106 may display the matching degree table 250 together with the second thumbnail 240b shown in FIG. 6 (b). The display means 106 may display the degree of relevance for each second thumbnail 240b by, for example, a balloon or the like. The display means 106 may display the second thumbnail 240b shown in FIG. 6B by rearranging them in an order according to the degree of relevance.

As shown in FIG. 6B, the display means 106 may be provided with a highlighting display 241 that highlights and displays the second thumbnail 240b according to the degree of relevance. The highlighting 241 includes, for example, adding a colored thick line to the edge of the second thumbnail 240b. For example, as shown in FIG. 6D, the coloring is green when the degree of relevance is equal to or higher than a predetermined threshold (for example, 0.9), and the degree of relevance is a predetermined threshold. When it is less than (for example, 0.7), it may be red, and when it is less than (for example, 0.7), it may be yellow.

In the above example, the highlighting 241 is attached to the second thumbnail 240b, but the present invention is not limited to this, and the highlighting 241 may be attached to the first thumbnail 240a.

The display means 106 displays the set of the associated first rectangular image 200a and the second rectangular image 200b output by the output means 105 on the display unit 32 of the terminal device 3. FIG. 7 is a diagram showing an example of displaying a set of rectangular images, and FIG. 7A is a diagram showing an example of a list of associated first rectangular images and a set of second rectangular images, (b). Is a diagram showing an example of an association change operation.

As shown in FIG. 7A, the display means 106 schematically shows a list of pairs of the first rectangular image 200a and the second rectangular image 200b associated with each other, the first thumbnail 240a and the first thumbnail 240a. It is displayed on the display surface 320 of the display unit 32 by the thumbnail 240b of 2. In the example shown in FIG. 7A, the reading target 200 having the ID “A01” and the reading target 200 having the ID “B01” are associated with each other. Similarly, the reading target 200 having the ID "A02" and the reading target 200 having the ID "B02", the reading target 200 having the ID "A03" and the reading target 200 having the ID "B03", and the reading target 200 having the ID "A04" are read. The target 200 and the reading target 200 whose ID is "B04" are associated with each other.

As shown in FIG. 7B, the display means 106 displays a list of the corresponding second thumbnails 240b with respect to one of the plurality of first thumbnails 240a to a degree indicating the calculated relevance. Display in the order according to. The display means 106 displays, for example, side by side from the top in descending order of degree of relevance. In the example shown in FIG. 7 (b), based on the result shown in FIG. 6 (c), the display means 106 is "B01" (C = 0. 95), "B06" (C = 0.90), "B04" (C = 0.88), and so on.

The display means 106 may display a scroll box 260 for browsing the list of the second thumbnail 240b by moving it up and down.

The display means 106 switches the display between the display shown in FIG. 7 (a) and the display shown in FIG. 7 (b) by an operation. This operation includes, for example, an operation of clicking the first thumbnail 240a displayed in FIG. 7A.

When the reception means 100 receives an operation for changing the association by the operator (hereinafter, also simply referred to as “change operation”), the display means 106 associates the first rectangular image 200a and the second rectangular image 200b. change. The display means 106 confirms whether or not there is a contradiction in the changed association. When the association is inconsistent due to the operation of the operator, for example, when one of the second thumbnails 240b is associated with the plurality of first thumbnails 240a, or when the second thumbnail 240b is any of the first thumbnails 240b. When the thumbnail 240a is not associated with the thumbnail 240a, the display means 106 displays an error message (warning) indicating a predetermined error.

The display means 106 displays the newly associated pair of the first rectangular image 200a and the second rectangular image 200b on the display unit 32 of the terminal device 3.

Although not shown, contrary to the example shown in FIG. 7B, the display means 106 has a first thumbnail 240a corresponding to one of the plurality of second thumbnails 240b. The list may be displayed in an order according to the degree of relevance calculated.

(Operation of the embodiment)
Next, an example of the operation of the image processing device 1 will be described with reference to FIG. FIG. 8 is a flowchart showing an example of the operation of the image processing device 1 according to the present embodiment.

(1) Reading of a reading target The operator arranges a plurality of reading targets 200 side by side on the platen of the image forming apparatus 2 so that the first surface 200A faces the surface of the platen. Next, the operator performs an operation instructing the start of scanning, such as pressing the start button of the operation display unit 21, for example. The instruction to start scanning may be given from the terminal device 3.

When the operator performs an operation instructing the start of scanning, the image forming apparatus 2 reads the first surface 200A of the plurality of scanning objects 200 arranged on the platen, forms the first scanned image 20a, and forms an image. It is passed to the reception means 100 of the processing device 1.

Next, the operator arranges a plurality of reading targets 200 side by side on the platen of the image forming apparatus 2 so that the second surface 200B faces the surface of the platen, and performs an operation of instructing the start of scanning. .. At this time, the arrangement of the plurality of reading targets 200 may be different from the arrangement of the reading targets 200 in the first scanned image 20a. The image forming apparatus 2 reads the second surface 200B of the plurality of reading targets 200, forms the second scanned image 20b, and passes it to the receiving means 100 of the image processing apparatus 1.

Next, the receiving means 100 receives the first scanned image 20a and the second scanned image 20b (S1). The receiving means 100 stores the received first read image 20a and the second read image 20b in the image data storage unit 112.

(2) Acquisition of Rectangle Image Next, the acquisition means 101 performs edge enhancement processing on the first read image 20a received by the reception means 100 to acquire the first rectangular image 200a and the second reading. Edge enhancement processing is performed on the image 20b to acquire a second rectangular image 200b (S2).

(3) Extraction of feature information Next, the extraction means 102 performs image recognition processing on the first rectangular image 200a to extract the feature information included in the first rectangular image 200a and the second rectangle. Image recognition processing is performed on the image 200b, and feature information included in the second rectangular image 200b is extracted (S3).

Next, the classification means 103 classifies the first rectangular image 200a by size and classifies the second rectangular image 200b by size (S4).

(4) Calculation of Degree of Relevance Next, the calculation means 104 calculates the matching degree Rc of the character information between the first rectangular image 200a and the second rectangular image 200b classified into the same size. (S5).

Next, the calculation means 104 calculates the layout matching degree Rl between the first rectangular image 200a and the second rectangular image 200b classified into the same size (S6).

Next, the calculation means 104 calculates the matching degree Rg of the graphic information between the first rectangular image 200a and the second rectangular image 200b classified into the same size (S7).

Next, the calculation means 104 determines whether or not there is graphic information in both the first rectangular image 200a and the second rectangular image 200b (S8).

If there is no graphic information in both the first rectangular image 200a and the second rectangular image 200b (S8: No), the calculation means 104 changes several coefficients used to calculate the degree of relevance. (S9).

Next, the calculation means 104 calculates the degree C indicating the relevance (S10).

(5) Output of result Next, the output means 105 outputs the calculated result described above to the display means 106. The display means 106 displays the calculated result on the display unit 32 of the terminal device 3, and changes the display mode according to the degree C indicating the relevance to the first rectangular image 200a and the second rectangular image 200b. Is displayed on the display unit 32 (S11).

Specifically, the display means 106 displays a list of the first thumbnails 240a. If the reception means 100 accepts a click operation by the operator to press one of the plurality of first thumbnails 240a while this list is displayed, the display means 106 receives the first thumbnail 240a. The list is switched to the list of the second thumbnail 240b and displayed, and the degree of the relationship between the first thumbnail 240a clicked by the operator and the plurality of second thumbnails 240b is displayed. The display means 106 may attach a highlighting 241 to the second thumbnail 240b depending on the degree of relevance.

On the other hand, if the reception means 100 accepts a click operation by the operator to press one of the plurality of second thumbnails 240b while the list of the second thumbnails 240b is displayed, the display means 106 will display. , The list of the second thumbnail 240b is switched to the list of the first thumbnail 240a and displayed, and the relationship between the second thumbnail 240b clicked by the operator and the plurality of first thumbnails 240a is displayed. Display the degree of indication. The display means 106 may attach a highlighting 241 to the first thumbnail 240a depending on the degree of relevance.

For example, the operator presses the changeover button (not shown), and from the state in which the calculated result is displayed, the associated set of the first rectangular image 200a and the second rectangular image 200b is displayed. Perform an operation to instruct switching to the state.

When the operator performs an operation instructing the start of scanning, the display means 106 displays a set of rectangular images shown in FIG. 7 (a) instead of displaying the results shown in FIGS. 6 (a) to 6 (d). Displayed on surface 320.

The operator clicks and operates one of the plurality of first thumbnails 240a shown in FIG. 7A. When the receiving means 100 accepts this click operation, the displaying means 106 displays a list of the corresponding second thumbnails 240b as shown in FIG. 7B.

The operator performs a change operation for changing the contents of the association with respect to the input unit 31 of the terminal device 3 as necessary. Specifically, the operator corresponds to the first rectangular image 200a corresponding to one first thumbnail 240a displayed on the display surface 320 of the display unit 32 and the second rectangular image 200b to be newly associated. The second thumbnail 240b to be selected is selected from the plurality of second thumbnails 240b and clicked.

Next, the display means 106 determines whether or not the reception means 100 has accepted the above-mentioned change operation by the operator (S12).

If the receiving means 100 accepts the change operation by the operator (S12: Yes), the display means 106 associates the first rectangular image 200a and the second rectangular image 200b based on the contents of the received operation. Change the content (S13).

The display means 106 displays the newly associated pair of the first rectangular image 200a and the second rectangular image 200b on the display unit 32 of the terminal device 3 (S14). If the operator performs a change operation and the contents of the association are inconsistent, the display means 106 displays an error message on the display unit 32.

As shown in FIGS. 3A and 3B, a first rectangular image 200a acquired from a first scanned image 20a including a first surface 200A of the reading target 200, and a second rectangular image 200a of the reading target 20. For the second rectangular image 200b acquired from the second scanned image 20b including the surface 200B, as shown in FIG. 7A, a set of the associated first rectangular image 200a and the second rectangular image 200b. Can be output.

<Modification example>
The highlighting 241 described above is not limited to the method using color coding or thick lines, and different hatching may be applied depending on the range of the degree of relevance. Further, the size of the first thumbnail 240a and the second thumbnail 240b (hereinafter, also referred to as "thumbnail 240" when both are collectively referred to) may be reduced or enlarged depending on the degree of relevance. The resolution of the thumbnail 240 may be adjusted according to the degree of relevance.

Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above embodiments, and various modifications and implementations are possible without changing the gist of the present invention. be.

Each means of the control unit 10 may be composed of hardware circuits such as a reconfigurable circuit (FPGA: Field Programmable Gate Array) and an application specific integrated circuit (ASIC), which are partially or wholly reconfigurable. good.

Further, it is possible to omit or change some of the components of the above embodiment without changing the gist of the present invention. Further, steps can be added, deleted, changed, replaced, etc. in the flow of the above-described embodiment without changing the gist of the present invention. Further, the program used in the above embodiment can be recorded on a computer-readable recording medium such as a CD-ROM and provided, stored in an external server such as a cloud server, and used via a network. You can also do it.

1 ... image processing device, 2 ... image forming device, 3 ... terminal device, 4 ... network,
10 ... Control unit, 11 ... Storage unit, 12 ... Communication unit, 20a ... First read image,
20b ... Second scanned image, 21 ... Operation display unit, 31 ... Input unit, 32 ... Display unit,
100 ... reception means, 101 ... acquisition means, 102 ... extraction means, 103 ... classification means,
104 ... calculation means, 105 ... output means, 106 ... display means,
111 ... Program storage unit, 112 ... Image data storage unit, 200 ... Read target,
200A ... 1st surface, 200B ... 2nd surface, 200a ... 1st rectangular image,
200b ... Second rectangular image, 201 ... Company name, 202 ... Department name, 203 ... Name,
204 ... Telephone number (direct contact), 205 ... Telephone number (representative contact),
206 ... logo mark, 207 ... email address, 210 ... dashed frame,
211 ... Dotted line frame, 212 ... Dashed line frame, 230 ... Result table, 240 ... Thumbnail,
240a ... 1st thumbnail, 240b ... 2nd thumbnail, 241 ... Highlight,
250 ... Matching table, 260 ... Scroll box, 320 ... Display surface,
C ... Degree of relevance, Rc ... Degree of matching of character information, Rl ... Degree of matching layout,
Rg ... Matching degree of graphic information, α, β, γ ... Coefficient

Claims (7)

Acquisition to acquire the first surface image from the first scanned image including the first surface to be read and to acquire the second surface image from the second scanned image including the second surface to be read. Means and
The first surface image and the second surface image associated with the first surface image and the second surface image are associated with the degree of showing the relationship between the feature information of the first surface image acquired by the acquisition means and the feature information of the second surface image. and an output means for outputting the set,
The output means has feature information including the position of the component of the first surface image in the first surface image and the second component of the second surface image as a degree of showing the relevance. Outputs the degree of matching with the feature information including the position in the surface image,
Image processing device. A display means for displaying the first surface image or the second surface image by changing the display mode according to the degree is further provided.
The image processing apparatus according to claim 1. The display means displays the first surface image or the second surface image in an order according to the degree.
The image processing apparatus according to claim 2. The display means emphasizes and displays the first surface image or the second surface image according to the degree.
The image processing apparatus according to claim 2 or 3. The output means comprehensively evaluates the degree of matching for each component of the image acquired by the acquisition means and outputs the degree.
The image processing apparatus according to claim 1. Wherein said component comprises information indicating the position of the character and the character, or the information indicating the position of the graphical information and graphic information,
The image processing apparatus according to claim 1 or 5. Computer,
Acquisition to acquire the first surface image from the first scanned image including the first surface to be read and to acquire the second surface image from the second scanned image including the second surface to be read. Means and
The first surface image and the second surface image associated with the first surface image and the second surface image are associated with the degree of showing the relationship between the feature information of the first surface image acquired by the acquisition means and the feature information of the second surface image. It is a program to function as an output means to output a set of
The output means has feature information including the position of the component of the first surface image in the first surface image and the second component of the second surface image as a degree of showing the relevance. A program that outputs the degree of matching with feature information including the position in a surface image.

Images