JP6330458B2 - Image processing apparatus and computer program - Google Patents

Description

  The present invention relates to a technique for determining a target area in a second image similar to a reference area in the first image.

  A technique for generating output image data representing an image obtained by combining a first image and a second image is known. For example, in the technique disclosed in Patent Document 1, an original having a size that cannot be read at one time is read in two using a scanner, and scan data representing the first image and the second image are read. Is obtained. Then, composite image data representing a composite image obtained by combining the first image and the second image is generated using the two scan data. The position where the first image and the second image are combined is determined using pattern matching. Pattern matching is a process of searching for a region similar to a partial region in the first image in the second image.

JP 2003-23530 A

  However, in the above technique, it cannot be said that the range in the second image to be searched for pattern matching is sufficiently optimized. For this reason, there is a possibility that the processing time for determining the target area in the second image similar to the reference area in the first image becomes long.

  An object of the present invention is to provide a new technique capable of reducing a processing time for determining a target area in a second image similar to a reference area in the first image.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following application examples.

Application Example 1 An image processing apparatus, an acquisition unit that acquires first image data representing a first image and second image data representing a second image, and the first image A reference area determining unit that determines a reference area, which is a partial area in the first image, using the data; the second image data; position information indicating a position of the reference area; and the reference A search region determination unit that determines a search region that is a partial region in the second image using external shape information regarding the external shape of the region, and a correspondence in the second image that is similar to the reference region The reference area and the corresponding area overlap using a corresponding area determining unit that determines an area from within the search area of the second image, the first image data, and the second image data. As described above, the synthesized image obtained by synthesizing the first image and the second image. And a generation unit for generating a composite image data representing the image processing apparatus.

According to the above configuration, a search area that is arranged at an appropriate position and has an appropriate size is determined using the position information and the external shape information of the reference area, and thus a search that is a partial area in the second image. Using the region, the corresponding region can be determined efficiently. Therefore, it is possible to reduce the processing time for determining the target area in the second image similar to the reference area in the first image.
[Application Example 2]
An image processing apparatus according to Application Example 1,
The search area determination unit
Obtaining information indicating a first reference position, which is a reference position of the first image, and information indicating a second reference position, which is a reference position of the second image;
An image processing apparatus that determines a position of the search area based on a relationship between a position of the reference area and the first reference position and the second reference position.
[Application Example 3]
An image processing apparatus according to Application Example 2,
The first reference position is information indicating a reference position of the reference area,
The second reference position is information indicating a reference position of the search area,
The information indicating the first reference position and the information indicating the second reference position are respectively information indicating the position in the first direction and a position in the second direction perpendicular to the first direction. Including information indicating
The first direction is a direction in which the first image and the second image are arranged in the composite image,
When the position of the reference area is shifted in the first direction with respect to the first reference position, the search area determining unit determines that the position of the search area is relative to the second reference position. When the position of the reference area is shifted in the second direction with respect to the first reference position, the position of the search area is shifted to the second reference position. An image processing apparatus that determines a position of the search region so as to be shifted in the second direction with respect to the image.
[Application Example 4]
The image processing apparatus according to any one of Application Examples 1 to 3,
The search area determination unit
Obtaining information indicating the outer shape of the first reference area serving as the reference area of the reference area and information indicating the outer shape of the second reference area serving as the reference of the search area;
An image processing apparatus that determines an outer shape of the search region based on a relationship between an outer shape of the reference region and an outer shape of the first reference region and an outer shape of the second reference region.
[Application Example 5]
An image processing apparatus according to Application Example 4,
The search area determination unit
Image processing for determining a length of the search region in the first direction using a ratio between a length of the reference region in the first direction and a length of the first reference region in the first direction apparatus.
[Application Example 6]
An image processing apparatus according to Application Example 5,
The first direction is a direction in which the first image and the second image are arranged in the composite image to be generated;
The search area determination unit
An image processing apparatus that determines a length of the search region in a second direction perpendicular to the first direction as a length of the reference region in the second direction.
[Application Example 7]
The image processing apparatus according to Application Example 5 or 6,
The search region determination unit is an image processing device that determines the length of the search region in the first direction to a value smaller than the length of the second reference region in the first direction.
[Application Example 8]
The image processing apparatus according to any one of Application Examples 4 to 7,
The search region determination unit is configured to determine whether the search region determined based on a relationship between an outer shape of the reference region and an outer shape of the first reference region and an outer shape of the second reference region is the second shape. An image processing device that corrects the search area so that the corrected search area is included in the second image when an area outside the first image is included.
[Application Example 9]
An image processing apparatus according to any one of Application Examples 4 to 8,
The second reference region is disposed along an edge in a first direction of the second image;
The first reference region is disposed along an end of the first image in a direction opposite to the first direction;
The image processing apparatus, wherein the first direction is a direction in which the first image and the second image are arranged in the composite image to be generated.
[Application Example 10]
The image processing apparatus according to any one of Application Examples 1 to 9,
The search area determination unit
An image processing apparatus that determines the length of the search region in the first direction to be longer as the length of the reference region in the first direction is longer.
[Application Example 11]
The image processing apparatus according to any one of Application Examples 1 to 10,
The position information includes coordinates that specify the position of the reference region,
The external shape information includes a length in a first direction indicating the size of the reference region and a length in a second direction perpendicular to the first direction.
[Application Example 12]
An image processing apparatus according to any one of Application Examples 1 to 11,
The reference area determining unit determines the reference area based on a variation in values of a plurality of pixels in the first image;
The image processing apparatus, wherein the search area determination unit determines the search area according to the reference area determined based on variation.
[Application Example 13]
An image processing apparatus according to any one of Application Examples 1 to 12,
The first image data is image data obtained by optically reading a first area of a document,
The image processing apparatus, wherein the second image data is image data obtained by optically reading a second area of the document.

  The present invention can be realized in various forms, for example, an image reading apparatus, an image processing apparatus, a method for controlling the image reading apparatus, a computer program for realizing these apparatuses or methods, and the like. It can be realized in the form of a recording medium on which a computer program is recorded.

It is a block diagram which shows the structure of the image processing system in an Example. FIG. 11 is a sequence diagram showing an operation of the image processing system 1000. It is a figure which shows an example of the original document used by a present Example. It is a figure which shows an example of UI screen. It is a figure which shows an example of a scanned image. 3 is a diagram illustrating an example of a composite image 30. FIG. It is a flowchart of a reference area determination process. It is explanatory drawing of a reference area determination process. It is a figure explaining a variation pixel and a non-variation pixel. It is a flowchart of a reference area arrangement process. It is a figure showing an example of reference field RA1 determined based on variation field VA. It is a flowchart of a search area | region determination process. It is explanatory drawing of a search area | region determination process. It is a flowchart of a corresponding area determination process. It is a figure explaining the several candidate area | region which should be specified in search area | region SA1.

A. Example:
A-1: Configuration of Image Processing System 1000 FIG. 1 is a block diagram illustrating a configuration of an image processing system in the embodiment. The image processing system 1000 includes a server 400 as an image processing apparatus and a multifunction device 200. The server 400 is connected to the Internet 70, and the multi-function device 200 is connected to the Internet 70 via a LAN (Local Area Network) 80. As a result, the server 400 and the multifunction device 200 can communicate with each other via the LAN 80 and the Internet 70. Further, the personal computer 500 of the user of the multifunction device 200 may be connected to the LAN 80.

  The server 400 includes a CPU 410, a volatile storage device 420 such as a DRAM, a nonvolatile storage device 430 such as a hard disk drive or a flash memory, and a communication unit 480 including an interface for connecting to a network such as the Internet 70. I have. The volatile storage device 420 is provided with a buffer area 421 for temporarily storing various intermediate data generated when the CPU 410 performs processing. The nonvolatile storage device 430 stores a computer program 431 and a UI data group 433.

  The computer program 431 and the UI data group 433 are installed in the server 400 by being uploaded to the server 400 via the Internet 70 by an administrator of the server 400, for example. Alternatively, the computer program 431 and the UI data group 433 may be provided in a form stored in, for example, a DVD-ROM, and may be installed in the server 400 by the administrator of the server 400. The CPU 410 implements image processing to be described later by executing the computer program 431.

  The MFP 200 includes a CPU 210, a volatile storage device 220 such as a DRAM, a nonvolatile storage device 230 such as a flash memory and a hard disk drive, a printer unit 240, a scanner unit 250, and an operation unit 260 such as a touch panel and buttons. A display unit 270 such as a liquid crystal display, and a communication unit 280 that communicates with an external device. For example, the communication unit 280 includes an interface for connecting to a network such as the LAN 80 and an interface for connecting to an external storage device (for example, a USB memory).

  The volatile storage device 220 is provided with a buffer area 221 for temporarily storing various data generated when the CPU 210 performs processing. A control program 231 is stored in the non-volatile storage device 230.

  The printer unit 240 executes printing using a printing method such as an inkjet method or a laser method. The scanner unit 250 generates scan data representing a color image or a gray image by optically reading a document using a photoelectric conversion element (for example, CCD, CMOS). The scanner unit 250 includes a so-called flatbed type document table. The size of the document table in the longitudinal direction is slightly smaller than 297 mm, which is the length in the longitudinal direction of the A4 size that defines the dimensions of paper defined by ISO (International Organization for Standardization) 216 (for example, several Cm) Long size. The size of the manuscript table in the short direction is the size in the short direction of the letter size that prescribes the paper dimensions defined in ANSI / ASME (Abbreviation of American National Standards Institute / American Society of Mechanical Engineers) Y14.1. The size is slightly longer (for example, several centimeters) than the length 215.9 mm. In this embodiment, the maximum size of a document that can be read at one time is larger than the A4 size and smaller than the A3 size (the paper size defined by ISO). Specifically, the scanner unit 250 is a document in which the length in the longitudinal direction is slightly longer than the length in the longitudinal direction of A4 size, and the length in the lateral direction is slightly longer than the length in the lateral direction of the letter size. Is generated, and image data representing an image of the size of the document is generated. Therefore, as will be described later, the A3 size document can be read in two steps so that the vicinity of the center in the longitudinal direction of the A3 size document overlaps.

  The CPU 210 executes control of the multifunction device 200 by executing the control program 231. For example, the CPU 210 controls the printer unit 240 and the scanner unit 250 to execute a copy process, a print process, a scan process, and the like. Furthermore, the CPU 210 can access the server 400 and execute a service use process that uses a service provided by the server 400.

A-2: Operation of Image Processing System 1000 FIG. 2 is a sequence diagram showing the operation of the image processing system 1000. The processing in the sequence diagram is started when the multifunction device 200 receives an instruction to use an image generation service provided by the server 400 from a user. As will be described in detail later, this image generation service is a service that combines a plurality of scan images represented by a plurality of scan data. The details of the plurality of scan data are described later. For example, the plurality of scan data are generated by reading a document having a size larger than the size that can be read at one time in a plurality of times.

  When the process is started, in S5, the CPU 210 of the multifunction device 200 transmits a service start request to the server 400. When receiving the service start request, the CPU 410 of the server 400 selects UI data necessary for providing the image generation service from the UI data group 433 (FIG. 1), and transmits the UI data to the multifunction device 200 (S10). ). Specifically, the UI data includes screen data representing a user interface screen (hereinafter referred to as UI screen) and control data. This control data includes, for example, various data necessary for the MFP 200 to perform predetermined processing (specifically, scan processing in S15 described later) using the UI screen. For example, the control data is used to perform processing (for example, transmission of scan data to the server 400) to be executed by the multifunction device 200 based on a user instruction received via a UI screen (for example, FIG. 4). Necessary information (for example, transmission address of scan data) is included.

  In S15, the CPU 210 executes a scan process for generating a plurality of scan data based on the received UI data. In the scan process, the CPU 210 generates two pieces of scan data by reading the document prepared by the user in two steps. The scan data according to the present embodiment is RGB image data including RGB component values (for example, values of 256 gradations from 0 to 255) for each pixel.

  FIG. 3 is a diagram illustrating an example of a document used in this embodiment. The size of the original 10 in FIG. 3 is about twice the size (A3 size in this embodiment) that is readable by the scanner unit 250 at one time (in this embodiment, a size slightly larger than the A4 size). .

  FIG. 4 is a diagram illustrating an example of a UI screen. First, the CPU 210 displays the UI screen UG1 of FIG. For example, the UI screen UG1 includes a message MS1 for prompting appropriate placement of the document 10 on the document table, a scan button SB, and a cancel button CB. The user places the document 10 on the document table and presses the scan button SB so that the left half 10L (FIG. 3) of the left side of the document 10 can be read according to the UI screen UG1. In response to the pressing of the scan button SB, the CPU 210 controls the scanner unit 250 to read the original, thereby generating left scan data.

  FIG. 5 is a diagram illustrating an example of a scanned image. FIG. 5A shows a left scan image 20L represented by the left scan data. The left-side scanned image 20L includes a left-side original image HIL that shows an approximately half area 10L (FIG. 3) on the left side of the original 10, and a margin WBL.

  Next, the CPU 210 displays a predetermined UI screen (not shown) on the display unit 270. Similar to the UI screen UG1, this UI screen includes a message for prompting appropriate placement of the document 10 on the document table, a scan button, and a cancel button. In accordance with the UI screen, the user places the document 10 on the document table and presses the scan button so that the half area 10R (FIG. 3) on the right side of the document 10 can be read. The CPU 210 generates right-side scan data by controlling the scanner unit 250 and reading a document in response to pressing of the scan button.

  FIG. 5B shows a right scan image 20R represented by the right scan data. The right-side scanned image 20R includes a right-side original image HIR that shows an approximately half area 10R (FIG. 3) on the right side of the original 10, and a margin WBR.

  Here, the image representing the lateral center CA of the document 10 in FIG. 3 is included in both the area along the right edge of the left scan image 20L and the area along the left edge of the right scan image 20R. Yes. That is, as shown by hatching in FIG. 5, the left-side scan image 20L includes an image CIL that represents the center portion CA of the document 10, and the right-side scan image 20R includes an image CIR that represents the center portion CA of the document 10. . This is because, for example, the central portion CA of the original 10 is read by both the generation of the right scan data and the generation of the left scan data according to a UI screen or a manual of the multifunction device 200. This is realized by instructing the user to place the document on the document table. Note that the image CIL in the left scan image 20L and the image CIR in the right scan image 20R both represent the center CA of the document 10, but the position of the document 10 placed on the document table by the user, the scanner Depending on the characteristics of the photoelectric conversion element of the portion 250, a slight difference in area, color, and the like may occur.

  In S20 of FIG. 2, the CPU 210 transmits right scan data representing the right scan image 20R and left scan data representing the left scan image 20L to the server 400. As a result, in S25, the CPU 410 of the server 400 acquires the right side scan data and the left side scan data, and stores them in the buffer area 421.

  In S30, the CPU 410 executes a reference area determination process using the right side scan data. The reference area determination process is a process of determining a reference area RA1 that is a partial area of the right scan image 20R represented by the right scan data. Details of the reference area determination process will be described later.

  In S32, the CPU 410 executes a search area determination process. The search area determination process is a process of determining a search area SA1, which is a partial area in the left scan image 20L, using the left scan data. Details of the search area determination process will be described later.

  In S35, the CPU 410 executes a corresponding area determination process. The corresponding area determination process is a process for determining the corresponding area CP in the left scan image 20L corresponding to the reference area RA1 in the right scan image 20R. The corresponding area CP corresponding to the reference area RA1 can be defined as follows. A part of the document 10 shown in the reference area RA1 (FIG. 5B) in the right-side scan image 20R is defined as a specific portion SPT (FIG. 3). The corresponding area CP corresponding to the reference area RA1 is an area representing the specific portion SPT of the document 10 in the left scan image 20L. The corresponding area CP is determined from the search area SA1 determined in the left scan image 20L by the search area determination processing in S32. Details of the corresponding area processing will be described later.

  In S40, the CPU 410 executes a composition process. In the composition process, a composite image representing a composite image 30 in which the right original image HIR in the right scan image 20R and the left original image HIL in the left scan image 20L are combined using the right scan data and the left scan data. Data is generated.

  FIG. 6 is a diagram illustrating an example of the composite image 30. As shown in FIG. 6, in the composite image 30, the right original image HIR and the left original image HIL are such that the reference area RA1 in the right original image HIR and the corresponding area CP in the left original image HIL overlap. Is synthesized. In the composite image 30, for example, the pixel value in the right original image HIR (right scan image 20R) is preferentially adopted as the pixel value in the region where the right original image HIR and the left original image HIL overlap each other. The As a result, the right original image HIR and the left original image HIL are combined to generate a combined image 30 representing the original 10 in FIG.

  If the corresponding area CP corresponding to the reference area RA1 cannot be determined in the corresponding area determination process described later, for example, the right edge of the left scanned image 20L and the left edge of the right scanned image 20R are mechanically processed. Composite image data representing a composite image obtained by combining the two scan images is generated (not shown in the drawing) such that the side of the image is in contact with each other.

  In S <b> 45 of FIG. 2, the CPU 410 transmits the generated composite image data to the multifunction device 200. When the composite image data is received, the CPU 210 of the multifunction device 200 stores the received composite image data in the nonvolatile storage device 230 and notifies the user that the composite image data has been received. The composite image data is used for the user. For example, the multifunction device 200 can print the composite image 30 using the composite image data based on a user instruction.

  According to the image processing system 1000 described above, a plurality of image data (specifically, right side scan data and left side scan data) obtained by reading different areas from one original 10 (FIG. 3) are obtained. By using this, composite image data representing one original 10 can be generated.

  Next, the reference area determination process shown in S30 of FIG. 2 will be described. FIG. 7 is a flowchart of the reference area determination process. FIG. 8 is an explanatory diagram of the reference area determination process. In S105, the CPU 410 determines the arrangement range FA in the right scan image 20R. The reference area RA1 is determined by the reference area determination process inside the determined arrangement range FA. The position, shape, and size (that is, the number of pixels in the vertical and horizontal directions) of the arrangement range FA are determined in advance.

  Specifically, as shown in FIG. 5B, the arrangement range FA is the end where the image CIR representing the central portion CA of the document 10 is located, among the four ends of the right-side scan image 20R. In the example, it is arranged along the left end of the right scanned image 20R.

  The lateral length of the arrangement range FA (that is, the lateral length) is preferably narrower than the lateral width of the image CIR representing the central portion CA of the document 10 in the right-side scan image 20R. The length in the vertical direction of the arrangement range FA (that is, the length in the longitudinal direction) is equal to the length in the vertical direction of the right scan image 20R.

  In S110, the CPU 410 determines the size of the unit area to be arranged in the arrangement range FA. In the present embodiment, the size of the unit area is determined to be a predetermined value. For example, the horizontal size of one unit region is, for example, about 1 to 2 pixels, and the vertical size of one unit region is 1/6 of the number of pixels in the vertical direction of the right scan image 20R. About 1/3.

  In S115, the CPU 410 selects one unit area of interest from the plurality of unit areas in the arrangement range FA. FIG. 8A shows a plurality of unit areas within the arrangement range FA. For the sake of explanation, the coordinates of the pixels in the arrangement range FA have the X axis with the right direction as the positive direction and the Y axis with the lower direction as the positive direction. This is expressed using a two-dimensional coordinate system 1).

  FIG. 8A shows four unit areas UAa to UAd. The upper end and the left end of the unit area UAa coincide with the upper end and the left end of the arrangement range FA. The lower end and the left end of the unit area UAb coincide with the lower end and the left end of the arrangement range FA. The upper end and right end of the unit area UAc coincide with the upper end and right end of the arrangement range FA. The lower end and the right end of the unit area UAd coincide with the lower end and the right end of the arrangement range FA.

  First, the unit area UAa at the upper left end is selected as the first target unit area. Then, as indicated by arrows in FIG. 8A, unit regions whose positions are shifted downward by one pixel are sequentially selected. When the lower left end unit area UAb is selected, a partial image shifted rightward by one pixel from the position of the upper left end unit area UAa is selected. Thereafter, partial images whose positions are shifted downward by one pixel are sequentially selected. The partial image selected last is the unit area UAd at the lower right end.

  When one target unit region is selected, in S120, the CPU 410 classifies the plurality of pixels in the target unit region into a variation pixel and a non-variation pixel, and the number of variation pixels (the number of variation pixels). ) Calculate VCu.

  FIG. 9 is a diagram for explaining a variation pixel and a non-variation pixel. The CPU 410 selects a plurality of pixels in the target unit region one by one as the target pixel TP, and uses the pixels in the region FL corresponding to three horizontal pixels × three pixels centered on the target pixel TP, A variation value V of TP is calculated. First, as shown in the equation of FIG. 9, the CPU 410 determines the difference ΔVn between the value of the target pixel TP (R0, G0, B0) and the values of the surrounding eight pixels (Rn, Gn, Bn). Is calculated. n is a number from 1 to 8 that identifies eight pixels around the pixel of interest TP, and is a number assigned to each pixel in the region FL of FIG. The CPU 410 calculates the total value of the calculated eight differences ΔVn as the variation value V of the target pixel TP, as shown by an equation in FIG. When the variation value V of the target pixel TP is greater than or equal to the predetermined threshold Vth, the target pixel TP is classified as a variation pixel, and when the variation value V is less than the predetermined threshold Vth, the target pixel TP is a non-variable pixel. are categorized.

  In S125, the CPU 410 compares the variation pixel number VCu in the unit area of interest with a predetermined threshold value VCth. When the variation pixel number VCu in the unit area of interest is equal to or greater than the threshold value VCth (S130: YES), in S135, the CPU 410 determines the coordinates UP (X1, Y1) of the upper left pixel of the unit unit area as the internal pixel. Is stored as information indicating a unit region (hereinafter also referred to as a variation unit region) having a relatively large variation in the value of. If the variation pixel number VCu in the unit area of interest is less than the threshold value VCth (S130: NO), the CPU 410 skips S135 and proceeds to S140.

  In S140, the CPU 410 determines whether or not all unit areas (FIG. 8A) in the arrangement range FA have been processed as the target unit area. If there is an unprocessed unit image (S140: NO), the CPU 410 returns to S115. When all the unit areas have been processed (S140: YES), in S145, the CPU 410 determines whether or not the coordinates P (X1, Y1) of one or more unit areas are stored in S135 described above. to decide.

  If the unit area coordinates UP (X1, Y1) are not stored (S145: NO), in S150, the CPU 410 determines the standard reference area RA0 (default area) as the reference area RA1. The reference reference area RA0 will be described later. When the unit area coordinates UP (X1, Y1) are stored (S145: YES), in S155, the CPU 410 executes a reference area arrangement process.

  FIG. 10 is a flowchart of the reference area arrangement process. First, when the reference area arrangement process in FIG. 10 is started, a plurality of variation unit areas are specified in the arrangement area FA (FIG. 8B). In FIG. 8B, the black circle indicates the position of the upper left pixel (also referred to as region specifying pixel) of the specified variation unit region.

  In S205, the CPU 410 selects one or more variation unit regions (hereinafter referred to as a proximity variation unit region group) that are close to each other from the plurality of specified variation unit regions. Specifically, the CPU 410 scans the arrangement range FA using the determination frame JH having a predetermined size (FIG. 8B), and the determination frame having the maximum number of area specifying pixels included therein. Determine the position of JH.

  FIG. 8B shows a determination frame JH in which the number of area specifying pixels included therein is the maximum. The determination frame JH shown in FIG. 8B includes five area specifying pixels UP1 to UP5. Further, the CPU 410 specifies an area specifying pixel whose distance from these five area specifying pixels UP1 to UP5 is shorter than the reference distance Dth. In the example of FIG. 8B, two area specifying pixels P6 and P7 whose distances from the area specifying pixels UP1 to UP5 are closer than the reference distance Dth are specified. As a result, in the example of FIG. 8B, seven variation unit regions corresponding to the seven region specifying pixels UP1 to UP7 are selected as the proximity variation unit region group.

  In S210, the CPU 410 calculates the number of pixels PC in an area VA defined by the proximity variation unit area group (hereinafter also referred to as a variation area VA). FIG. 8C is a diagram showing an example of the variation area VA defined by the proximity variation unit area group. The variation area VA includes a plurality of pixels included in at least one unit area among one or more variation unit areas included in the proximity variation unit area group.

  In S215 to S240, the CPU 410 determines the reference region RA1 by comparing the calculated pixel number PC with the three pixel number threshold values PCth1 to PCth3 (PCth1 <PCth2 <PCth3).

  First, in S215, the CPU 410 determines whether or not the pixel number PC is less than the first pixel number threshold PCth1. When the number of pixels PC is less than the first pixel number threshold PCth1 (S215: YES), in S230, the CPU 410 sets an area having a size (default size) of a reference reference area RA0 (described later) as a reference area. Determine as RA1. In this case, since the variation area VA is excessively small, a default size area centered on the position of the center of gravity of the variation area VA is determined as the reference area RA1 so that the reference area RA1 is not excessively small.

  When the pixel number PC is greater than or equal to the first pixel number threshold PCth1 (S215: NO), in S220, the CPU 410 determines whether or not the pixel number PC is less than the second pixel number threshold PCth2. To do. When the pixel number PC is less than the second pixel number threshold PCth2 (S220: YES), in S240, the CPU 410 determines a circumscribed rectangle of the variation area VA as the reference area RA1. FIG. 11 is a diagram illustrating an example of the reference area RA1 determined based on the variation area VA. FIG. 11A shows an example in which the circumscribed rectangle of the specific variation area VA1 is determined as the specific reference area RA11.

  When the pixel number PC is equal to or larger than the second pixel number threshold PCth2 (S220: NO), in S225, the CPU 410 determines whether or not the pixel number PC is less than the third pixel number threshold PCth3. To do. When the pixel number PC is less than the third pixel number threshold PCth3 (S225: YES), in S235, the CPU 410 determines the inscribed rectangle of the variation area VA as the reference area RA1. FIG. 11B shows an example in which the inscribed rectangle of the specific variation area VA2 is determined as the specific reference area RA12.

  When the pixel number PC is equal to or greater than the third pixel number threshold value PCth3 (S225: NO), in S230, the CPU 410 determines a default size region as the reference region RA1. In this case, since the variation area VA is excessively large, an area having a default size is determined as the reference area RA1 so that the reference area RA1 is not excessively large. When the reference area RA1 is determined, the reference area determination process is terminated. As can be understood from the above description, in the reference area determination process of the present embodiment, a partial image area having a relatively large variation within the arrangement range FA is determined as the reference area RA1. FIG. 5B shows an example of the reference area RA1 determined within the arrangement range FA of the right-side scan image 20R.

  Next, the search area determination process in S32 of FIG. 2 will be described. FIG. 12 is a flowchart of search area determination processing. FIG. 13 is an explanatory diagram of the search area determination process. FIG. 13A shows an example of a reference search area SA0 that is an area serving as a reference for the search area SA1. FIG. 13B shows an example of a reference reference area RA0 that is a reference area of the reference area RA1. FIG. 13C shows an example of the search area SA1 to be determined by the search area determination process. FIG. 13D shows an example of the reference area RA1 determined by the above-described reference area determination process (FIG. 7).

  In S305 of FIG. 12, the CPU 410 acquires the position information and the outline information of the reference area RA1 determined by the reference area determination process of FIG. As described above, the reference area RA1 is an area determined in the right scan image 20R. Specifically, the position information of the reference area RA1 is upper left coordinates (Rx1, Ry1) indicating the position of the upper left pixel of the reference area RA1 (FIG. 13D). Specifically, the external shape information of the reference region RA1 is the width Rw1 and the height Rh1 of the reference region RA1 (FIG. 13D). The width means the length in the horizontal direction, and the height means the length in the vertical direction.

  As shown in FIG. 13D, the upper left coordinates (Rx1, Ry1) of the reference area RA1 have an X axis with the right direction as the positive direction and a Y axis with the lower direction as the positive direction. It is expressed using a two-dimensional coordinate system in which the coordinates of the upper left pixel of 20R are (0, 0). Further, the width Rw1 and the height Rh1 of the reference region RA1 are represented by the number of pixels.

  In S310, the CPU 410 determines the position information and outer shape information of the reference reference area RA0, specifically, the upper left coordinates (Rx0, Ry0) indicating the position of the upper left pixel of the reference reference area RA0, the width Rw0, and the height Rh0. Are acquired (FIG. 13B). The upper left coordinates (Rx0, Ry0) of the reference reference area RA0 are expressed using the same coordinate system as the upper left coordinates (Rx1, Ry1) of the reference area RA1. The width Rw0 and the height Rh0 of the reference reference area RA0 are represented by the number of pixels.

  The reference reference area RA0 is an area defined in advance on the right-side scan image 20R, and predetermined values are acquired for the position information and the outline information. In the present embodiment, the reference reference area RA0 is a rectangular area in contact with the left end and the upper end of the right scanned image 20R. Therefore, the upper left coordinate of the reference reference area RA0 matches the upper left coordinate of the right scan image 20R. That is, the upper left coordinates (Rx0, Ry0) = (0, 0).

  Further, the width Rw0 of the standard reference region RA0 is, for example, 5 to 20 pixels. The height Rh0 of the reference reference area RA0 is 1/4 to 1/2 of the height of the right-side scan image 20R.

  In S315, the CPU 410 determines the position information and outline information of the reference search area SA0, specifically, the upper right coordinates (Sx0, Sy0) indicating the position of the upper right pixel of the reference search area SA0, the width Sw0, and the height Sh0. Are acquired (FIG. 13A). The upper right coordinates (Sx0, Sy0) of the reference search area SA0 have an X axis with the right direction as the positive direction and a Y axis with the lower direction as the positive direction, and the coordinates of the upper left pixel of the left scan image 20L are ( (0, 0) is represented using a two-dimensional coordinate system.

  The reference search area SA0 is an area defined in advance on the left-side scan image 20L, and predetermined values are acquired for the position information and the outline information. In the present embodiment, the reference search area SA0 is a rectangular area in contact with the right end and the upper end of the left scan image 20L. Therefore, the upper right coordinate of the reference search area SA0 matches the upper right coordinate of the right scan image 20R.

  Further, the width Sw0 of the reference search area SA0 is, for example, ¼ to ½ of the width of the left scan image 20L. The height of the reference search area SA0 is the same as the height of the reference reference area RA0 described above. That is, the height of the reference search area SA0 is 1/4 to 1/2 of the height of the left scan image 20L.

  The search area SA1 to be determined is determined by the processes of S320 to S360. Specifically, as shown in FIG. 13C, the search area SA1 is a rectangular area arranged on the left scan image 20L. The search area SA1 is defined by upper right coordinates (Sx1, Sy1) indicating the position of the upper right pixel of the search area SA1, the width Sw1 of the search area SA1, and the height Sh1 of the search area SA1. Therefore, the search area SA1 is determined by determining the upper right coordinates (Sx1, Sy1), the width Sw1, and the height Sh1 by the processes of S320 to S360.

  In S320, the CPU 410 determines the Y coordinate Sy1 of the upper right coordinates (Sx1, Sy1) of the search area SA1. Specifically, the Y coordinate Sy1 of the upper right coordinates (Sx1, Sy1) is determined to be the same value as the Y coordinate Ry1 of the upper left coordinates (Rx1, Ry1) of the reference area RA1 (Sy1 = Ry1).

  In S325, the CPU 410 determines the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) of the search area SA1. Specifically, the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) is determined as a value obtained by adding (Rx1-Rx0) to the X coordinate Sx0 of the upper right coordinate (Sx0, Sy0) of the reference search area SA0 ( Sx1 = Sx0 + (Rx1-Rx0)). (Rx1-Rx0) represents an amount by which the reference region RA1 is displaced in the right direction when viewed from the standard reference region RA0 (hereinafter referred to as a lateral displacement amount of the reference region RA1). Therefore, the X coordinate Sx1 of the upper right coordinates (Sx1, Sy1) of the search area SA1 is equal to the X coordinate Sx0 of the upper right coordinates (Sx0, Sy0) of the reference search area SA0 by the amount of lateral deviation (Rx1-Rx0) of the reference area RA1. The position is shifted in the same direction.

  In S335, the CPU 410 determines the height Sh1 of the search area SA1. Specifically, the height Sh1 of the search area SA1 is determined to be the same value as the height Rh1 of the reference area RA1 (Sh1 = Rh1).

  In S340, the CPU 410 determines the width Sw1 of the search area SA1. Specifically, the width Sw1 of the search area SA1 is determined to be a value obtained by multiplying the width Sw0 of the reference search area SA0 by the ratio (Rw1 / Rw0) of the width Rw1 of the reference area RA1 to the width Rw0 of the reference reference area RA0. (Sw1 = (Rw1 / Rw0) × Sw0).

  In S345, the CPU 410 determines whether or not the width Sw1 of the search area SA1 determined in S340 satisfies a predetermined condition. The predetermined conditions are Sw1 <(Sw0 + Sx1-Sx0) and (Sx1-Sx0)> 0. This condition is obtained when the left end X coordinate (Sx1-Sw1) of the search region SA1 is on the right side of the left end X coordinate (Sx0-Sw0) of the reference search region SA0, that is, ((Sx0-Sw0) <( Sx1-Sw1)). In other words, if this condition is not satisfied, it means that the left end of the search area SA1 is located on the left side of the left end of the reference search area SA0, and the search area SA1 extends excessively to the left side. .

  When the predetermined condition is not satisfied (S345: NO), in S350, the CPU 410 corrects the width Sw1 of the search area SA1. Specifically, the left end X coordinate (Sx0-Sw0) of the reference search area SA0 and the left end X coordinate (Sx1-Sw1) of the search area SA1 are the same, that is, (Sx0-Sw0) = The value of the width Sw1 is corrected so as to be (Sx1-Sw1) (Sw1 = Sw0 + Sx1-Sx0).

  If the predetermined condition is satisfied (S345: YES), the CPU 410 skips S350 and advances the process to S355.

  In S355, the CPU 410 determines whether or not the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) of the search area SA1 is located in the left scan image 20L. Specifically, it is determined whether or not the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) of the search area SA1 is larger than the X coordinate of the upper right coordinate of the left scan image 20L. In the present embodiment, the upper right coordinate of the left scan image 20L matches the upper right coordinate (Sx0, Sy0) of the reference search area SA0. When Sx1 is larger than the X coordinate (Sx0 in this embodiment) of the right end of the left scan image 20L, the CPU 410 determines that Sx1 is not located in the left scan image 20L.

  When the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) of the search area SA1 is not located in the left scan image 20L (S355: NO), in S360, the CPU 410 sets the value of Sx1 to the left scan image 20L. To the right end X coordinate (in this embodiment, the X coordinate Sx0 of the upper right coordinate (Sx0, Sy0) of the reference search area SA0), and the search area determination process ends. FIG. 13C illustrates the corrected upper right coordinates (Sx1a, Sy1) (Sx1a = Sx0).

  When the X coordinate Sx1 of the upper right coordinate (Sx1, Sy1) of the search area SA1 is located in the left scan image 20L (S355: YES), the CPU 410 skips S360 and performs the search area determination process. finish. In the example of FIG. 13C, the hatched area is finally determined as the search area SA1.

  Next, the corresponding area determination process in S35 of FIG. 2 will be described. FIG. 14 is a flowchart of the corresponding area determination process. In S410, the CPU 410 specifies one attention candidate area from among a plurality of candidate areas to be specified in the search area SA1.

  FIG. 15 is a diagram for explaining a plurality of candidate areas to be specified in the search area SA1. The shape and size of the candidate area are determined according to the reference area RA1 (FIG. 5B) determined in the right-side scan image 20R by the reference area determination process of FIG. First, the frame NP having the same shape and size as the reference area RA1 is arranged with respect to the search area SA1 so as to overlap at least a part of the search area SA1. And the area | region which overlaps search area | region SA1 among the area | regions in frame NP is pinpointed as an attention candidate area | region.

  A frame NPa in FIG. 15A is a frame arranged on the rightmost side and on the upper side. The coordinates of the upper left pixel Pa of the frame NPa are (k, 1). Note that the coordinates (k, 1) of the pixel Pa are coordinates used for convenience of the description of the reference area determination process, and the coordinate system used to define the search area SA1 in the search area determination process described above. It has nothing to do with the coordinates. The same applies to the coordinates of pixels Pb to Pd described later. The position in the horizontal direction (X-axis direction) of the pixel Pa is the same as the horizontal position of the rightmost pixel in the search area SA1. The position of the pixel Pa in the vertical direction (Y-axis direction) is positioned above the uppermost pixel of the search area SA1 by a predetermined length (specifically, a length corresponding to 50 to 100 pixels). Accordingly, as indicated by hatching in FIG. 15A, a partial area below a plurality of pixels located on the line of one pixel at the left end of the area within the frame NPa is a search area SA1. It overlaps with.

  A frame NPb in FIG. 15B is a frame arranged on the rightmost side and the lower side. The coordinates of the upper left pixel Pb of the frame NPb are (k, m). The horizontal position of the pixel Pb is the same as the horizontal position of the rightmost pixel in the search area SA1. The vertical position of the pixel Pb is located below the lower end pixel of the search area SA1 by a predetermined length. Therefore, as shown by hatching in FIG. 15B, among the regions in the frame NPb, the upper partial region of the plurality of pixels located on the leftmost pixel line is the search region SA1. overlapping.

  A frame NPc in FIG. 15C is a frame arranged on the leftmost side and the upper side. The coordinates of the upper left pixel Pc of the frame NPc are (1, 1). The horizontal position of the pixel Pc is the same as the horizontal position of the leftmost pixel in the search area SA1. The vertical position of the pixel Pc is located above the upper end pixel of the search area SA1 by a predetermined length. Therefore, as indicated by hatching in FIG. 15C, a partial region on the lower side of the region in the frame NPc overlaps the search region SA1.

  A frame NPd in FIG. 15D is a frame arranged on the leftmost side and the lower side. The coordinates of the upper left pixel Pd of the frame NPd are (1, m). The horizontal position of the pixel Pd is the same as the horizontal position of the leftmost pixel in the search area SA1. The vertical position of the pixel Pc is located below the lower end pixel of the search area SA1 by a predetermined length. Therefore, as indicated by hatching in FIG. 15D, an upper partial region of the region in the frame NPc overlaps the search region SA1.

  The coordinates of the upper left pixel of the frame NP arranged to specify a plurality of candidate areas are sequentially selected from (k × m) coordinates represented by (p, q). Here, p is an arbitrary integer from 1 to k, and q is an arbitrary integer from 1 to m. Accordingly, (k × m) frames NP can be arranged in the search area SA1. In other words, (k × m) candidate areas can be specified in the search area SA1.

  As indicated by arrows in FIGS. 15A to 15C, the upper right end frame NPa (FIG. 15A) is arranged from among these (k × m) frames NP. It is first selected as the power frame NP. Then, when the next candidate region of interest is specified, the frames NP whose positions are shifted downward by one pixel are sequentially selected. When the lower right end frame NPb (FIG. 15B) is selected, a frame NP that is shifted leftward by one pixel from the position of the upper right end frame NPa is selected. Thereafter, the frames NP whose positions are shifted downward one pixel at a time are sequentially selected. The frame NP selected last is the frame NPd at the lower left end (FIG. 15D).

  As in the frames NPa to NPd in FIGS. 15A to 15D, in most cases, a part of the frame NP is located outside the search area SA1, and therefore the size of the attention candidate area is as shown in FIG. It becomes smaller than the reference area RA1 in (B).

  When one attention candidate area is specified, in S415, the CPU 410 selects one pixel as the attention pixel among all the pixels in the attention candidate area.

  In S420, the CPU 410 calculates a difference ΔVP between the value of the target pixel in the target candidate region and the value of the pixel in the reference region RA1 corresponding to the target pixel. A pixel in the reference area RA1 corresponding to the target pixel includes an image in the target candidate area and an image in the reference area RA1, and a frame NP used for specifying the target candidate area and an outer edge of the reference area RA1. The pixels in the reference area RA1 that overlap with the target pixel when they are overlapped so as to match. The values of the two pixels for which the difference ΔVP is to be calculated are (R1, G1, B1) and (R2, G2, B2). The difference ΔVP is represented by the sum of absolute values of differences between the three types of component values. That is, the difference ΔVP is represented by the sum of the absolute value of (R1−R2), the absolute value of (G1−G2), and the absolute value of (B1−B2).

  In subsequent S425, CPU 410 determines whether or not calculated difference ΔVP is equal to or smaller than a predetermined reference value TH1. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1 (S425: YES), in S430, the CPU 410 counts up the number of similar pixels SC. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1, it is possible to determine that the target pixel in the target candidate region and the pixel in the reference region RA1 corresponding to the target pixel are similar pixels. is there.

  When the difference ΔVP is larger than the predetermined reference value TH1 (S425: NO), the CPU 410 skips S430 and advances the process to S435. The reference value TH1 is 10 to 40, for example.

  In S435, the CPU 410 determines whether or not all the pixels in the attention candidate area have been processed as the attention pixels. When there is an unprocessed pixel (S435: NO), the CPU 410 returns to S415 and selects an unprocessed pixel as a target pixel. When all the pixels have been processed (S435: YES), the CPU 410 advances the process to S440.

  In S440, the CPU 410 calculates the similarity (SC / Nt) between the reference area RA1 and the attention candidate area. The similarity (SC / Nt) is a ratio of the number of similar pixels SC to the total number Nt of pixels in the target candidate region. As the similarity (SC / Nt) is larger, the reference area RA1 and the attention candidate area are more similar.

  In S445, CPU 410 determines whether or not the similarity (SC / Nt) is equal to or higher than threshold value TH2. That is, the CPU 410 determines whether or not the current attention candidate area is similar to the reference area RA1.

  When the similarity (SC / Nt) is equal to or higher than the threshold value TH2, that is, when it is determined that the current candidate region of interest is similar to the reference region RA1 (S445: YES), in S450, the CPU 410 The current attention candidate region is determined as a corresponding region CP similar to the reference region RA1, and the corresponding region determination process is ended.

  If the similarity (SC / Nt) is less than the threshold value TH2 (S445: NO), in S455, has the CPU 410 processed all candidate areas (FIG. 15) in the search area SA1 as attention candidate areas? Judge whether or not. If there is an unprocessed candidate area (S455: NO), the CPU 410 initializes the number of similar pixels SC in S460, returns to S410, and selects the unprocessed candidate area as the target candidate area. . When all candidate areas have been processed (S455: YES), the CPU 410 ends the corresponding area determination process without determining the corresponding area CP corresponding to the reference area RA1.

  FIG. 5A shows the determined corresponding region CP in the left scan image 20L.

  According to the embodiment described above, in the search area determination process, the position information indicating the position of the reference area RA1 (specifically, the upper left coordinates (Rx1, Ry1) in FIG. 13D) and the reference area RA1 The search area SA1 is determined using the external shape information (specifically, the width Rw1 and the height Rh1 in FIG. 13D) (S320 to S340 in FIG. 12). In the corresponding area determining process (FIGS. 14 and 15), the corresponding area CP is determined from the determined search area SA1. As a result, the corresponding region CP can be determined efficiently. Therefore, it is possible to reduce the processing time for generating the composite image data.

  More specifically, the position of the right original image HIR in the right scan image 20R is shifted in the vertical direction with respect to the left original image HIL in the left scan image 20L due to, for example, the specifications of the scanner unit 250 of the multifunction device 200. The amount falls within a predetermined assumed range. Therefore, if the height Sh1 of the search area SA1 is excessively large with respect to the height Rh1 of the reference area RA1, an area where the corresponding area CP does not exist is excessively large in the search area SA1. Then, the frequency of performing the process of searching for the corresponding area CP (S415 to S445 of the corresponding area determining process in FIG. 14) using the candidate area that cannot be the corresponding area CP as the target candidate area is increased. Then, there is a possibility that the processing time for generating the composite image data becomes longer than necessary. Further, in the left side scan image 20L and the right side scan image 20R, the width in the horizontal direction of the central portion CA of the document 10 included in common falls within a predetermined assumed range, so that the search is performed with respect to the width Rw1 of the reference region RA1. Even if the width Sw1 of the area SA1 is excessively large, similarly, the processing time may be longer than necessary. In addition, when the position of the search area SA1 is not an appropriate position with respect to the position of the reference area RA1, a search is performed to ensure that the corresponding area CP to be searched is included in the search area SA1. Since the size (width and height) of the area SA1 must be increased, the processing time may be longer than necessary. According to the above-described embodiment, the search area SA1 that is arranged at an appropriate position and has an appropriate size is determined using the position information and the outline information of the reference area RA1, so that such inconvenience is avoided, The processing time for generating the composite image data can be reduced.

  Further, when the position of the search area SA1 is not an appropriate position with respect to the position of the reference area RA1, there is a high possibility that the corresponding area CP to be searched is not included in the search area SA1. As a result, the corresponding area CP corresponding to the reference area RA1 cannot be searched, and there is a possibility that composite image data representing a composite image in which two images are combined with high accuracy cannot be generated. According to the present embodiment, the search area SA1 that is arranged at an appropriate position and has an appropriate size is determined using the position information and the outer shape information of the reference area RA1, and therefore the corresponding area CP to be searched is included. Thus, an appropriate search area SA1 can be determined. As a result, it is possible to generate composite image data representing a composite image in which two images are combined with high accuracy.

  Further, in the search area determination process of the above embodiment, the upper left coordinates (Rx0, Ry0) of the standard reference area RA0 are acquired as information indicating the first standard position, which is the standard position of the reference area RA1 (S310). . Also, the upper right coordinates (Sx0, Sy0) of the reference search area SA0 are acquired as information indicating the second reference position, which is the reference position of the search area SA1 (S315). Then, the position of the search area SA1 is determined based on the relationship between the position of the reference area RA1 and the first reference position and the second reference position. As a result, an appropriate search area can be determined according to the position of the reference area RA1 and the positions of the standard reference area RA0 and the standard search area SA0.

  Specifically, the coordinates (Sx1, Sy1) defining the position of the search area SA1 are set such that the position of the reference area RA1 is the first reference position (ie, the reference position) as shown in FIGS. The position of the search area SA1 is shifted to the right with respect to the second reference position (that is, the position of the reference search area SA0), and When the position of the reference area RA1 is shifted upward with respect to the first reference position, the position of the search area SA1 is set so that the position of the search area SA1 is shifted upward with respect to the second reference position. Is determined. This is because when the reference area RA1 is determined at a position relatively distant from the left end of the right scan image 20R, that is, a position shifted to the right from the position of the standard reference area RA0, the corresponding area CP is determined as the left scan image. This is because there is a high possibility of being in a position closer to the right end in 20L. This is because, when the reference area RA1 is determined on the relatively upper side in the right scan image 20R, the corresponding area CP is likely to exist on the relatively upper side in the left scan image 20L. As a result, an appropriate search area SA1 can be determined according to the relationship in the vertical and horizontal directions between the position of the reference area RA1 and the position of the reference reference area RA0.

  Furthermore, in the search area determination process of the above embodiment, information (specifically, width Rw0 and height Rh0) indicating the outer shape of the reference reference area RA0 and information (specifically, information indicating the outer shape of the reference search area SA0). , Width Sw0 and height Sh0) are acquired (S310, S315). Then, the outline of the search area SA1 is determined based on the relationship between the outline of the reference area RA1 and the outline of the standard reference area RA0 and the outline of the standard search area SA0. Specifically, as shown in FIGS. 13A to 13D, as the width Rw1 of the reference region RA1 is longer than the width Rw0 of the reference reference region RA0, the width Sw1 of the search region SA1 is larger than that of the reference search region SA0. The width is longer than Sw0. Then, as the height Rh1 of the reference area RA1 is larger than the height Rh0 of the standard reference area RA0, the height Sh1 of the search area SA1 is made larger than the height Sh0 of the standard search area SA0. As a result, an appropriate outer shape of the search region SA1 can be determined according to the outer shape of the reference region RA1, the outer shape of the standard reference region RA0, and the outer shape of the standard search region SA0.

  More specifically, the width Sw1 of the search area SA1 is determined using a ratio (Rw1 / Rw0) between the width Rw1 of the reference area RA1 and the width Rw0 of the reference reference area RA0 (S340). As a result, an appropriate width of the search area SA1 can be determined in accordance with the width relationship between the reference area RA1 and the reference reference area RA0. Further, the height Sh1 of the search area SA1 is determined as the height Rh1 of the reference area RA1 (S335). As a result, the height Sh1 of the search area SA1 can be appropriately determined according to the height Rh1 of the reference area RA1. In other words, the processing time for generating the composite image data while suppressing the height Sh1 and the width Sh1 of the search area SA1 from becoming excessively large or excessively small with respect to the reference area RA1. Reduction and improvement in the accuracy of combining two images in a composite image can be realized.

  Further, the width Sw1 of the search area SA1 is corrected so that the left end of the search area SA1 is not positioned to the left of the left end of the reference search area SA0 (S345 and S350 in FIG. 3). In other words, the width Sw1 of the search area SA1 is determined to be smaller than the width Sw0 of the reference search area SA0. As a result, the width Sw1 of the search area SA1 can be suppressed from becoming excessively long, and the processing time for generating the composite image data can be suppressed from becoming excessively long.

  Further, when the search area SA1 determined based on the relationship between the outer shape of the reference area RA1 and the outer shape of the standard reference area RA0 and the outer shape of the standard search area SA0 includes an area outside the left scan image 20L. The search area SA1 is corrected so that the corrected search area SA1 is included in the left-side scan image 20L (S355 and S360 in FIG. 3). As a result, an appropriate search area SA1 included in the left scan image 20L can be determined.

  Further, as shown in FIG. 13B, the reference reference area RA0 is arranged along the left end (the opposite direction to the X-axis direction) of the right-side scan image 20R. As shown in FIG. 13A, the reference search area SA0 is arranged along the right end of the left scan image 20L. Here, the X-axis direction can be said to be a direction in which the left scan image 20L and the right scan image 20R are arranged in the composite image to be generated in the present embodiment. According to the above configuration, it is possible to determine an appropriate search area SA1 for determining a corresponding area for generating composite image data using an appropriate reference area.

  Further, in the reference area determination process (FIGS. 7 to 9), the reference area RA1 is determined based on the variation in the values of the plurality of pixels in the right-side scan image 20R. For example, the position, size, and size of the reference area RA1 are each determined based on the values of a plurality of pixels in the arrangement range FA of the right-side scan image 20R. Specifically, the reference region RA1 is determined so as to include a unit region in which the pixel value variation is relatively large (FIG. 8 and the like). In the corresponding area determination process (FIG. 14), a plurality of candidate areas are specified according to the reference area determined based on the variation. As a result, an appropriate corresponding region CP corresponding to the reference region RA1 is easily determined.

  More specifically, an area where the variation in pixel values is relatively small has poor characteristics. For this reason, when the region where the variation in pixel values is relatively small is determined as the reference region RA1, it may be difficult to determine the corresponding region CP in the left-side scan image 20L. For example, an erroneous area in the left scanned image 20L is easily determined as the corresponding area CP. The region where the pixel variation is relatively small includes, for example, a region indicating the background and a region where the color change inside the object is poor.

  On the other hand, a region having a relatively large variation in pixel values includes a relatively large number of edges and the like, and thus is likely to include a characteristic portion. For this reason, when a region having a relatively large variation in pixel values is determined as the reference region RA1, the corresponding region CP in the left-side scan image 20L can be easily determined. In other words, it is possible to determine the corresponding region CP corresponding to the corresponding region CP with higher accuracy by using the region having a relatively large variation as a reference than using the region having a relatively small variation as a reference. Alternatively, it is possible to determine an appropriate corresponding region CP even if the size of the corresponding region CP is smaller than the region where the variation is relatively larger than the region where the variation is relatively small. Become.

B. Modification (1) In the above embodiment, the upper left coordinates (Rx1, Ry1) are used as the position information of the reference area RA1, and the width Rw1 and the height Rh1 are used as the outer shape information of the reference area RA1. Instead, the coordinates of two pixels located at the two vertices forming the diagonal of the reference area RA1 may be used as information that serves as both position information and outer shape information of the reference area RA1. Specifically, a combination of an upper left coordinate representing the upper left pixel and a lower right coordinate representing the lower right pixel may be used. Further, the coordinates of the four pixels located at the four vertices of the reference area RA1 may be used as information that serves as both position information and outer shape information of the reference area RA1. Any position information described above can identify the position of the reference area RA1 on the right-side scan image 20R. In addition, any of the above-described outline information can specify the width and height indicating the size of the reference area RA1.

  Note that the position information and outline information of the reference reference area RA0 and the position information and outline information of the reference search area SA0 are also located at the two vertices forming the diagonal of the rectangle, similar to the position information and outline information of the reference area RA1. Alternatively, the coordinates of two pixels or four pixels located at four vertices of a rectangle may be used.

(2) In the above embodiment, in order to determine the search area SA1, in addition to the position information and the outline information of the reference area RA1, the position information and the outline information of the standard reference area RA0 and the reference search area SA0 are used. . Instead, the CPU 410 may determine the reference area RA1 using only the position information and the outer shape information of the reference area RA1 without using the position information and the outer shape information of the standard reference area RA0 and the standard search area SA0. good. For example, the CPU 410 determines the height of the search area SA1 to be the same as the height of the reference area RA1. Then, the CPU 410 determines the width of the search area SA1 to be a predetermined number of times (for example, 20 times) the width of the reference area RA1. Then, the CPU 410 determines the vertical position of the search area SA1 as the same position as the vertical position of the reference area RA1. Then, the CPU 410 determines the horizontal position of the search area SA1 so that the right end position of the search area SA1 is the same position as the right end of the left scan image 20L.

(3) In the above embodiment, in order to determine the search area SA1, the relationship between the position of the standard reference area RA0 and the position of the reference area RA1 and the position of the standard search area SA0 are used. Instead, the CPU 410 determines the relationship between the reference position of the right scan image 20R (for example, the position of the upper left pixel of the right scan image 20R) and the position of the reference area RA1, and the reference position of the left scan image 20L. The search area SA1 may be determined using the position (for example, the position of the upper right pixel of the left scan image 20L). For example, the CPU 410 determines that the position of the upper left pixel of the reference area RA1 with respect to the position of the upper left pixel of the right scan image 20R and the position of the upper right pixel of the search area SA1 with respect to the position of the upper right pixel of the left scan image 20L. The search area SA1 may be determined so as to have a mirror image relationship.

(4) Let two images represented by two scan data be a first image and a second image. In the above embodiment, the vicinity of the left end of the first image (specifically, the right scan image 20R) and the vicinity of the right end of the second image (specifically, the left scan image 20L) are combined. In addition, a composite image is generated. Instead, for example, in accordance with one original used for generating two pieces of scan data, the vicinity of the right end of the first image and the vicinity of the left end of the second image are combined so as to be combined. An image may be generated. Alternatively, a composite image may be generated such that the vicinity of the lower end of the first image and the vicinity of the upper end of the second image are combined, and the vicinity of the upper end of the first image and the vicinity of the second image A composite image may be generated so that the vicinity of the lower end is combined.

  Then, such a combination mode of the first image and the second image may be determined in advance, and for each of the four combination modes described above, the determination of the reference region RA1 is performed. The determination of the corresponding area CP may be executed. In the latter case, combined image data representing a combined image in which the first image and the second image are combined is generated in a combination mode in which the similarity between the reference region RA1 and the corresponding region CP is the highest. Also good.

  The reference area RA1 and the search area SA1 to be determined differ depending on the mode of the four combinations. Further, the standard reference area RA0 and the standard search area SA0 to be used are different depending on the four coupling modes. For example, consider a case where a composite image is generated such that the vicinity of the lower end of the first image and the vicinity of the upper end of the second image are combined. In this case, for example, the reference area RA1 is determined near the lower end of the first image, and the search area SA1 is determined near the upper end of the second image. For example, the standard reference area RA0 is arranged along the lower end of the first image, and the standard search area SA0 is arranged along the upper end of the second image. For example, the width of the search area SA1 is determined to be the same as the width of the reference area RA1, and the height of the search area SA1 is equal to the ratio between the height of the reference area RA1 and the height of the standard reference area RA0. The value is determined by multiplying the height of SA0.

(5) In the above embodiment, the height Sh1 of the search area SA1 is made the same as the height Rh1 of the reference area RA1 (S335 in FIG. 12). In the above-described embodiment, this is based on the assumption that the amount of vertical deviation between the right original image HIR in the right scan image 20R and the left original image HIL in the left scan image 20L is relatively small. This is because the height Sh0 of the area SA0 and the height Rh0 of the standard reference area RA0 are equal to each other (Sh0 = Rh0 (FIG. 13)). For example, if it is assumed that the amount of vertical deviation between the right original image HIR in the right scan image 20R and the left original image HIL in the left scan image 20L is relatively large, the reference search area SA0 The height Sh0 can be larger than the height Rh0 of the reference reference region RA0 (Sh0> Rh0).

  In this case, the height Sh1 of the search area SA1 is determined to be a value obtained by multiplying the height ratio (Rh1 / Rh0) between the reference area RA1 and the standard reference area RA0 by the height Sh0 of the standard search area SA0. (Sh1 = (Rh1 / Rh0) × Sh0). Alternatively, the height Sh1 of the search area SA1 may be determined to be a value obtained by adding a margin of a predetermined length M determined in consideration of an assumed vertical shift amount to the height Rh1 of the reference area RA1. (Sh1 = Sh0 + M). In general, the height Sh1 of the search area SA1 is preferably determined so as to increase as the height Rh1 of the reference area RA1 increases.

(6) The position information of the reference area RA1 may be information different from the coordinates. For example, the position information of the reference area RA1 may be information representing the vertical position of the reference area RA1 in three stages (specifically, upper, middle, and lower). The external shape information of the reference area RA1 may be information different from the width and the height. For example, the outer shape information of the reference area RA1 may be information representing the size of the reference area RA1 in three levels (specifically, large, medium, and small). In this case, for example, in the reference area determination process, the CPU 410 obtains a combination of the three-stage position information and the three-stage outline information based on the variation in the values of the plurality of pixels in the right-side scan image 20R. One reference region RA1 is determined from nine types of reference regions corresponding to the nine types of patterns. Then, the CPU 410 may determine one search area SA1 from nine types of search areas corresponding to the nine types of patterns corresponding to the determined reference area RA1.

(7) In the above-described embodiment, composite image data representing a composite image obtained by combining two scan images is generated using two scan data. The present invention is not limited to this, and the composite image data may be generated using an arbitrary number of scan data. For example, composite image data representing a composite image obtained by combining four scan images may be generated using the four scan image data.

(8) In the above embodiment, the two pieces of image data used for generating the composite image data are two pieces of scan data generated by reading a document by the scanner unit 250 of the multifunction device 200. Instead of this, two pieces of image data may be generated by photographing a plurality of areas of a document with a digital camera.

(9) In the above embodiment, the two scan data used for generating the composite image data are the right side scan data and the left side scan data obtained by reading one document 10. Alternatively, the two scan data may be two scan data obtained by reading two originals, respectively.

(10) The processing (for example, the processing of S25 to S40 in FIG. 2) executed by the CPU 410 of the server 400 in the above embodiment may be executed by the CPU 210 of the multifunction device 200, for example. In this case, the server 400 is unnecessary, and the multi-function device 200 may execute the processing of FIG. 2 alone. Further, the processing executed by the CPU 410 of the server 400 may be executed by the CPU (not shown) of the personal computer 500 (FIG. 1) connected to the multifunction device 200. For example, the CPU of the personal computer 500 may execute these processes by executing a scanner driver program installed in the personal computer 500. Further, the server 400 may be configured with one computer as in the present embodiment, or may be configured with a computer system including a plurality of computers.

(11) In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware. Also good.

  200 ... multifunction device, 210 ... CPU, 220 ... volatile storage device, 221 ... buffer area, 230 ... nonvolatile storage device, 231 ... control program, 240 ... printer , 250 ... scanner unit, 260 ... operation unit, 270 ... display unit, 280 ... communication unit, 400 ... server, 410 ... CPU, 420 ... volatile storage device 421 ... buffer area, 430 ... nonvolatile storage device, 431 ... computer program, 433 ... UI data group, 480 ... communication unit, 500 ... personal computer, 1000 ... Image processing system

Claims (13)

An image processing apparatus,
An acquisition unit that acquires first image data representing a first image and second image data representing a second image;
A reference area determining unit that determines a reference area, which is a partial area in the first image, using the first image data;
A search area, which is a partial area in the second image, is determined using the second image data, position information indicating the position of the reference area, and outline information regarding the outline of the reference area. A search area determination unit to perform,
A corresponding area determining unit that determines a corresponding area in the second image similar to the reference area from within the search area of the second image;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap with each other is obtained. A generating unit that generates composite image data to be represented;
Equipped with a,
The search area determination unit
Obtaining information indicating the outer shape of the first reference area serving as the reference area of the reference area and information indicating the outer shape of the second reference area serving as the reference of the search area;
And the relationship between the contour of the outer shape and the first reference region of the reference region, and the outer shape of the second reference area, based on, that determine the outer shape of the search area, the image processing apparatus. The image processing apparatus according to claim 1,
The search area determination unit
Obtaining information indicating a first reference position, which is a reference position of the first image, and information indicating a second reference position, which is a reference position of the second image;
An image processing apparatus that determines a position of the search area based on a relationship between a position of the reference area and the first reference position and the second reference position. The image processing apparatus according to claim 2,
The first reference position is information indicating a reference position of the reference area,
The second reference position is information indicating a reference position of the search area,
The information indicating the first reference position and the information indicating the second reference position are respectively information indicating the position in the first direction and a position in the second direction perpendicular to the first direction. Including information indicating
The first direction is a direction in which the first image and the second image are arranged in the composite image,
When the position of the reference area is shifted in the first direction with respect to the first reference position, the search area determining unit determines that the position of the search area is relative to the second reference position. When the position of the reference area is shifted in the second direction with respect to the first reference position, the position of the search area is shifted to the second reference position. An image processing apparatus that determines a position of the search region so as to be shifted in the second direction with respect to the image. The image processing apparatus according to any one of claims 1 to 3 ,
The search area determination unit
Image processing for determining a length of the search region in the first direction using a ratio between a length of the reference region in the first direction and a length of the first reference region in the first direction apparatus. The image processing apparatus according to claim 4 ,
The first direction is a direction in which the first image and the second image are arranged in the composite image to be generated;
The search area determination unit
An image processing apparatus that determines a length of the search region in a second direction perpendicular to the first direction as a length of the reference region in the second direction. The image processing apparatus according to claim 4 or 5 , wherein
The search region determination unit is an image processing device that determines the length of the search region in the first direction to a value smaller than the length of the second reference region in the first direction. An image processing apparatus according to any one of claims 1 to 6,
The search region determination unit is configured to determine whether the search region determined based on a relationship between an outer shape of the reference region and an outer shape of the first reference region and an outer shape of the second reference region is the second shape. if from the image including the outer regions, so that the search region after correction is included in the second image, modifying the search area, the image processing apparatus. An image processing apparatus according to any one of claims 1 to 7,
The second reference region is disposed along an edge in a first direction of the second image;
The first reference region is disposed along an end of the first image in a direction opposite to the first direction;
The image processing apparatus, wherein the first direction is a direction in which the first image and the second image are arranged in the composite image to be generated. An image processing apparatus according to any one of claims 1-8,
The search area determination unit
An image processing apparatus that determines the length of the search region in the first direction to be longer as the length of the reference region in the first direction is longer. An image processing apparatus according to any one of claims 1 to 9 ,
The position information includes coordinates that specify the position of the reference region,
The external shape information includes a length in a first direction indicating the size of the reference region and a length in a second direction perpendicular to the first direction. The image processing apparatus according to any one of claims 1 to 10 ,
The reference area determining unit determines the reference area based on a variation in values of a plurality of pixels in the first image;
The image processing apparatus, wherein the search area determination unit determines the search area according to the reference area determined based on variation. An image processing apparatus according to any one of claims 1 to 1 1,
The first image data is image data obtained by optically reading a first area of a document,
The image processing apparatus, wherein the second image data is image data obtained by optically reading a second area of the document. A computer program,
An acquisition function for acquiring first image data representing a first image and second image data representing a second image;
A reference area determination function for determining a reference area, which is a partial area in the first image, using the first image data;
A search area, which is a partial area in the second image, is determined using the second image data, position information indicating the position of the reference area, and outline information regarding the outline of the reference area. A search area determination function to
A corresponding area determination function for determining a corresponding area in the second image similar to the reference area from within the search area of the second image;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap with each other is obtained. A generating function for generating composite image data to be represented;
Is realized on a computer ,
The search area determination function is:
Obtaining information indicating the outer shape of the first reference area serving as the reference area of the reference area and information indicating the outer shape of the second reference area serving as the reference of the search area;
And the relationship between the contour of the outer shape of the reference region of the first reference area, said the outer shape of the second reference area, based on a computer program that determines the outer shape of the search area.

Images