JP6897405B2 - Image processing equipment and computer programs - Google Patents

Description

The present specification relates to image processing for identifying a document area in a scanned image.

Conventionally, scanned image data representing a scanned image including a document image has been generated by using an image scanning device such as a scanner. For example, the image reading device disclosed in Patent Document 1 calculates a straight line corresponding to the edge of the document region by using the image data generated by scanning the document and using the Hough transform or the least squares method. The tilt of the document is calculated based on the straight line. The image reading device determines the document area by passing through the pixels that are considered to be the edges of the document area and calculating a straight line having the calculated inclination.

Japanese Unexamined Patent Publication No. 2009-218953

However, in the above technique, for example, it cannot be said that the method for specifying the manuscript area is sufficiently devised. Therefore, depending on the type of image, there is a possibility that the original area cannot be specified accurately.

The present specification discloses a technique capable of accurately identifying a manuscript area in a target image based on scanned image data.

The techniques disclosed herein have been made to solve at least a portion of the above problems and can be realized as the following application examples.

[Application Example 1] The target is an image processing apparatus using an image data acquisition unit that acquires scanned image data generated by optically scanning a document and target image data based on the scanned image data. A line-of-sight determination unit that determines a straight line of interest in a target image indicated by image data, wherein the target image includes a manuscript image indicating the manuscript, and the straight line of attention is parallel to a specific outer edge of the manuscript image. Using the attention straight line determining unit and the target image data, which are straight lines, a plurality of pixels located on the attention straight line are each composed of N types of pixels (N) including the original pixels constituting the original image. Is an integer of 3 or more), the number of the manuscript pixels among the N types of pixels, and N1 types of pixels including the manuscript pixels (N1 is an integer satisfying 3 ≦ N1 ≦ N). The number calculation unit for calculating the number and the number of N2 types of pixels (N2 is an integer satisfying 2 ≦ N2 <N1) including the original image, and whether or not the straight line of interest satisfies the first condition. The first condition determination unit for determining, the first condition is that the ratio of the original image pixel to the N1 type of image located in the attention straight line is equal to or more than the first threshold value. The first condition determination unit and the second condition determination unit for determining whether or not the attention straight line satisfies the second condition different from the first condition, and the second condition is the above. Each of the M straight lines (M is an integer of 2 or more) parallel to the straight line of interest and continuous in the direction intersecting the straight line of interest satisfies the specific condition, and the M straight lines include the straight line of interest. The specific condition is that the ratio of the original image pixel to the N2 types of pixels located on the target straight line is equal to or greater than the second threshold value. When the first outer edge straight line determining unit which determines the outer edge straight line located at the outer edge of the original image based on the attention straight line and the attention straight line satisfy the second condition when the first condition is satisfied. A second outer edge straight line determining unit that determines the outer edge straight line based on the attention straight line, and an area specifying unit that uses the outer edge straight line to specify a document area corresponding to the original document image in the target image. An image processing device.

According to the above configuration, it can be determined whether or not the straight line of interest satisfies the first condition and whether or not the straight line of interest satisfies the second condition. Then, when the straight line of interest satisfies the first condition and when the straight line of interest satisfies the second condition, the outer edge straight line located at the outer edge of the original image is determined based on the straight line of interest. As a result, the outer edge straight line can be appropriately determined. Therefore, the original area in the target image based on the scanned image data can be accurately specified by using the outer edge straight line.

The techniques disclosed in the present specification can be realized in various forms, for example, a multifunction device, a scanner, an image processing method, a function of these devices, or a computer program for realizing the above method. It can be realized in the form of a recording medium or the like on which the computer program is recorded.

It is a figure which shows the structure of the image processing system as one Example. It is a flowchart of image processing of this Example. It is a figure which shows an example of the image used in image processing. It is a flowchart of contour identification processing. It is explanatory drawing of the contour specifying process. It is a figure which shows the scanned image OI. It is an enlarged view of a part area AR of the scanned image OI of FIG. It is a flowchart which shows the processing step of determination of a start straight line. It is a flowchart which shows the specific processing step of a document area AA. It is a flowchart of a repair process. It is a figure which shows an example of a specific document area.

A. Example:
A-1. Configuration of Image Processing System FIG. 1 is a diagram showing a configuration of an image processing system as an embodiment. This image processing system includes an image processing device 100 and a scanner 400. The image processing device 100 and the scanner 400 are connected to the network NT and can communicate with each other.

The image processing device 100 is, for example, a personal computer (for example, a desktop computer or a tablet computer). The image processing device 100 includes a CPU 110 as a controller of the image processing device 100, a volatile storage device 120, a non-volatile storage device 130, a display unit 140 for displaying an image, an operation unit 150 for accepting an operation by a user, and the like. It has an interface 190 and. These elements are connected to each other via a bus.

The CPU 110 is an arithmetic unit (processor) that processes data. The volatile storage device 120 is, for example, a DRAM, and the non-volatile storage device 130 is, for example, a flash memory. The non-volatile storage device 130 stores the program 132. By executing the program 132, the CPU 110 causes the scanner 400 to read the document and executes a process of generating output image data representing the scanned document (details will be described later). The program 132 may be, for example, a scanner driver for controlling the scanner 400. The CPU 110 temporarily stores various intermediate data used for executing the program 132 in a storage device (for example, either a volatile storage device 120 or a non-volatile storage device 130). Program 132 is provided, for example, by the manufacturer of scanner 400.

The display unit 140 is a device for displaying an image, for example, a liquid crystal display. Alternatively, other types of devices that display images, such as LED displays and organic EL displays, may be employed. The operation unit 150 is a device that receives an operation by the user, and is, for example, a touch panel that is superposed on the display unit 140. Alternatively, other types of devices operated by the user, such as buttons and levers, may be employed. The user can input various instructions to the image processing device 100 by operating the operation unit 150.

The interface 190 is an interface for communicating with other devices (for example, a USB interface, a wired LAN interface, and an IEEE 802.11 wireless interface). In this embodiment, the interface 190 is a wired or wireless network interface and is connected to the network NT.

FIG. 1 shows a perspective view of the scanner 400. In the figure, the first direction Dp1 and the second direction Dp2 indicate a horizontal direction, and the third direction Dp3 indicates a vertically upward direction. The first direction Dp1 and the second direction Dp2 are perpendicular to each other. Further, the third direction Dp3 is also referred to as an upward direction Dp3.

The scanner 400 is a so-called flatbed type image reading device. The scanner 400 includes a main body 490 and a cover 450 attached to the upper Dp3 side of the main body 490 so as to be openable and closable. FIG. 1 shows a scanner 400 with a cover 450 opened toward Dp3 in the upward direction.

The main body 490 includes a control unit 410, an image sensor 420, a moving device 430 that moves the image sensor 420 in parallel with the first direction Dp1, and a document base 440.

The platen 440 is provided on the upward Dp3 side of the main body 490 (FIG. 1). The platen 440 appears by opening the cover 450 upward Dp3. The document table 440 is a substantially rectangular table surrounded by two sides parallel to the first direction Dp1 and two sides parallel to the second direction Dp2, and is configured by using a transparent plate (for example, a glass plate). There is. The surface of the platen 440 on the third direction Dp3 side is the mounting surface Us on which the document to be read is placed.

The image sensor 420 is arranged below the platen 440 (opposite the third direction Dp3), that is, below the platen 440. The image sensor 420 is a one-dimensional image sensor that optically reads a document, and has a configuration in which a plurality of photoelectric conversion elements (simply also referred to as optical elements) such as a CCD or CMOS are arranged side by side in the second direction Dp2. Have. The image sensor 420 optically reads the document on the platen 440 to output a signal representing the scanned document.

The mobile device 430 includes a power source (eg, an electric motor). The moving device 430 uses the power source to move the image sensor 420 in a direction along the mounting surface Us of the platen 440 (a direction parallel to the first direction Dp1 in FIG. 1).

The control unit 410 is a computer including, for example, a CPU 411 that performs data processing, a volatile storage device 412 (for example, DRAM), a non-volatile storage device 413 (for example, a flash memory), and an interface 419. Each element of the control unit 410 is connected to each other via a bus. The non-volatile storage device 413 stores the program 414 in advance. The CPU 411 controls the image sensor 420 and the mobile device 430 by executing the program 414. Interface 419 is an interface for communicating with other devices (for example, USB interface, wired LAN interface, IEEE 802.11 wireless interface). In this embodiment, the interface 419 is a wired or wireless network interface and is connected to the network NT.

The CPU 411 controls the moving device 430 to move the image sensor 420 along the platen 440 from the position of the end of the platen 440 in the direction opposite to the first direction Dp1 to the position of the end of the platen 440 on the side of the first direction Dp1. To move. The CPU 411 controls the image sensor 420 while moving the image sensor 420 to optically read the document placed on the platen 440. The CPU 411 generates scanned image data indicating the scanned image based on the signal output from the image sensor 420. The CPU 411 transmits the generated scanned image data to the image processing device 100.

A-2. Outline of image processing FIG. 2 is a flowchart of image processing of this embodiment. In this embodiment, the CPU 110 of the image processing device 100 executes image processing as a scanner driver. In this image processing, scanned image data is acquired by causing the scanner 400 to scan the original, and the acquired scanned image data is used to represent a repaired image (described later) that has been subjected to tilt correction, missing repair, and the like. Finished image data is generated. The user inputs an image processing start instruction by operating the operation unit 150. The CPU 110 starts the process of FIG. 2 in response to the input start instruction.

In S10, the CPU 110 acquires the read image data. Specifically, the CPU 110 transmits an instruction to read the document to the scanner 400. The CPU 411 of the control unit 410 of the scanner 400 generates scanned image data by reading the document placed on the platen 440 in response to an instruction from the image processing device 100, and obtains the generated scanned image data as an image. It is transmitted to the processing device 100. The CPU 110 acquires the scanned image data transmitted from the scanner 400. The data format of the scanned image data may be various formats. In this embodiment, the scanned image data is RGB image data including RGB values of a plurality of pixels, or monochrome image data including brightness values of a plurality of pixels. The RGB value of one pixel indicates the color of that pixel, and for example, the three component values of red (R), green (G), and blue (B) (hereinafter, R value, G value, and B value). Also called). In this embodiment, the number of gradations of each component value is 256 gradations. The luminance value is a value indicating the luminance of the pixel, for example, as a value of 256 gradations.

FIG. 3 is a diagram showing an example of an image used in image processing. FIG. 3A shows an example of the scanned image OI. The shape of the scanned image OI is rectangular.

The scanned image OI is an image generated by scanning a rectangular original. The scanned image OI includes a manuscript image Ad showing the manuscript and a background image Ab located outside the manuscript image Ad. The background image Ab indicates an area other than the original outside the original image Ad. The background image Ab is an image outside the original image Ad, that is, an image located between the outer edge of the original image Ad and the outer edges (four sides) of the scanned image OI. The background image Ab has a specific background color. In this embodiment, the background image Ab shows the facing surface Bs (FIG. 1) facing the platen 440 of the cover 450. Therefore, the specific background color is the color of the facing surface Bs. In this embodiment, the color of the facing surface Bs is darker than white and lighter gray than black.

The shape of the original image Ad is an incomplete substantially rectangular shape in which the region Da corresponding to the upper right corner of the rectangle of the original and the region Db corresponding to the lower right corner are missing. The chipping of the upper right area Da is caused by the fact that the original is read with the corresponding portion of the original protruding from the readable area on the platen 440. The chipping of the lower right region Db is caused by the fact that the original is read on the platen 440 with the corresponding corners of the original bent. Further, the substantially rectangle of the original image Ad is inclined with respect to the rectangle of the outer shape of the scanned image OI. This inclination is due to the fact that the original is read on the platen 440 with the original tilted.

The manuscript image Ad includes objects Ob1 and Ob2 such as characters and drawings, and a ground color region Sa. The ground color area Sa is an area indicating the ground color of the original.

In S15, the CPU 110 uses the scanned image data to execute a reduction process for reducing the image to generate reduced image data indicating the reduced image MI. A known process is used for the reduction process. For example, the reduced image MI has a number of pixels corresponding to 100 dpi (dot per inch) in length × 100 dpi in width. When the vertical and horizontal resolutions of the scanned image data are 100 dpi or less, the process of S15 is omitted. It can also be said that FIG. 3A is a diagram showing a reduced image MI.

In S20, the CPU 110 performs an edge extraction process for extracting the edge in the reduced image MI on the reduced image data to generate the edge extraction data. Specifically, the CPU 110 applies a so-called Sobel filter to the value of each pixel of the reduced image data to calculate the edge strength Se. The CPU 110 generates edge extraction data in which these edge strength Se are set as the values of a plurality of pixels.

The formula (1) for calculating the edge strength is shown below. The gradation value P (x, y) in the formula (1) represents the brightness of a specific pixel position (x, y) in the reduced image MI. The position x indicates the pixel position in the direction D1 (vertical direction), and the position y indicates the pixel position in the direction D2 (horizontal direction). The edge strength Se (x, y) at the pixel position (x, y) in the reduced image MI is calculated using the values of nine pixels in three rows and three columns adjacent to each other with the pixel position (x, y) as the center. Will be done. The first and second terms of the calculation formula are absolute values of the sum of the gradation values of the pixels at the nine positions multiplied by the corresponding coefficients. The first term is the derivative of the gradation value in the direction D1 (that is, the derivative in the horizontal direction), and the second term is the derivative of the gradation value in the direction D2 (that is, the derivative in the vertical direction). The calculated edge strength Se (x, y) is normalized to a value of 256 gradations in the range of 0 to 255.

…（１）

… (1)

In the edge extraction image indicated by the generated edge extraction data, the position corresponding to the edge in the reduced image MI, for example, the pixel corresponding to the boundary between the original image Ad and the background image Ab in the reduced image MI of FIG. 3 (A). Or, the value of the pixel corresponding to the contour of the objects Ob1 and Ob2 becomes larger than the value of the pixel at another position.

In S25, the CPU 110 binarizes the edge extraction data generated in S20 to generate edge binary image data indicating the edge binary image EI. For example, in the edge extraction data, the CPU 110 specifies a pixel whose pixel value (that is, edge strength) is equal to or greater than a threshold value (for example, 128) as an edge pixel PXe, and a pixel whose pixel value is less than the threshold value is defined as an edge pixel PXe. It is specified as a non-edge pixel PXn. In the edge binary image data, the value of the edge pixel PXe is set to "1", and the value of the non-edge pixel PXn is set to "0".

FIG. 3B shows an example of the edge binary image EI. In this edge binary image EI, it can be seen that the pixels corresponding to the boundary between the original image Ad and the background image Ab in the reduced image MI and the pixels corresponding to the contours of the objects Ob1 and Ob2 are specified as the edge pixels PXe. .. Since each pixel in the reduced image MI corresponds to one or more pixels of the scanned image OI, specifying the edge pixel PXe and the non-edge pixel PXn in the reduced image MI is in the scanned image OI. Is equivalent to identifying the corresponding edge and non-edge pixels of.

In this embodiment, since the edge pixel PXe is specified using the reduced image data, the number of pixels to be processed can be reduced as compared with the case where the scanned image data is used, for example. Therefore, the processing time and memory capacity required to identify the edge pixel PXe can be reduced.

In S30, the CPU 110 executes the contour specifying process. The contour identification process uses edge binary image data to identify a plurality of contour pixels PXo constituting the contour of the original image from the plurality of edge pixels PXe specified in the reduced image MI. This is a process of generating contour image data showing a contour image OLI showing the contour of the original image Ad.

FIG. 3C shows an example of the contour image OLI. In this example, a plurality of contour pixels PXo constituting the contour of the original image Ad indicated by the six straight lines L1 to L6 are specified in the contour image OLI. The contour image OLI is an image composed of a plurality of pixels corresponding to a plurality of pixels of the reduced image MI on a one-to-one basis. The contour image data is binary data, and the value of each pixel of the contour image data indicates whether or not the corresponding pixel of the reduced image MI is the contour pixel PXo. Since each pixel in the reduced image MI corresponds to one or more pixels in the scanned image OI, specifying the contour pixel PXo in the reduced image MI identifies the corresponding contour pixel in the scanned image OI. Is equivalent to doing. Details of the contour identification process will be described later.

In S40, the CPU 110 executes the document area specifying process. The manuscript area specifying process is a process of specifying the manuscript area AA corresponding to the manuscript image Ad by using the contour image data. As shown in FIG. 3A, the document area AA is a rectangular area obtained by compensating for the above-mentioned areas Da and Db that are missing in the document image Ad.

In S45, the CPU 110 executes a repair process on the read image data using the specific result of the document area AA to generate the repaired image data indicating the repaired image RI. FIG. 3D shows an example of the repaired image RI. The repaired image RI includes a repaired manuscript image Dr and a repaired background image Abr. The manuscript image Adr has a rectangle corresponding to the manuscript area AA and is not tilted with respect to the rectangle of the outer shape of the repaired image RI. Further, the background image Abr outside the original image Adr is painted with a designated color (for example, white) designated in advance. Details of the repair process will be described later.

In S50, the CPU 110 outputs the repaired image data. In this embodiment, the CPU 110 stores the repaired image data in a storage device (for example, a non-volatile storage device 130) inside the image processing device 100. Instead, the CPU 110 may store the repaired image data in an external storage device (for example, a USB memory (not shown)) connected to the image processing device 100. Further, the CPU 110 may output the repaired image data in a form of printing the repaired image RI on the printer by transmitting the repaired image data to the printer, for example. Any format can be adopted as the data format of the repaired image data. In this embodiment, the CPU 110 outputs the repaired image data in JPEG format.

According to the image processing described above, the repaired image data showing the repaired image RI showing the original is output by using the scanned image data. As a result, it is possible to provide the user with image data that appropriately indicates the original.

A-3. Contour identification process The contour identification process of S30 of FIG. 2 will be described. As described above, the contour specifying process is a process of specifying the contour pixel PXo using the edge binary image data. FIG. 4 is a flowchart of the contour specifying process. FIG. 5 is an explanatory diagram of the contour specifying process.

In S100 of FIG. 4, the CPU 110 prepares the initial data of the contour image data indicating the contour image OLI in the volatile storage device 120. The initial data is binary image data indicating an image having the same size as the reduced image MI and the edge binary image EI, and the values of all the pixels are set to "0" as the initial values.

In S105, the CPU 110 has one attention end from the four ends corresponding to the four sides of the outer shape of the edge binary image EI, that is, the upper end TU, the lower end TB, the left end TL, and the right end TR (FIG. 5). Select. In S110, the CPU 110 selects one pixel of interest from the plurality of pixels on the end of interest.

In S115, the CPU 110 searches from the attention end toward the inside of the edge binary image EI, starting from the attention pixel, and identifies the edge pixel PXe to be searched first. Specifically, the direction toward the inside of the edge binary image EI is a direction perpendicular to the attention end and toward the other end of the edge binary image EI facing the attention end. As shown in FIG. 5, when the pixel PXt1 located on the upper end TU is the pixel of interest, the search is performed downward, and the edge pixel PXe1 to be searched first is specified. Similarly, when the pixels PXt2, PXt3, and PXt4 located on the right end TR, the lower end TB, and the left end TL are the pixels of interest, the search is performed toward the left side, the upper side, and the right side, respectively, and the search is performed first. The edge pixels PXe2, PXe3, and PXe4 to be formed are specified.

In S118, the CPU 110 determines whether or not the edge pixel has been specified. Specifically, as a result of searching from the pixel of interest in the search direction, if the edge pixel PXe1 is searched before reaching the limit position of the search, for example, another end facing the end of interest, the edge pixel PXe1 is used. Judged as identified. If the edge pixel PXe1 is not searched even when the search limit position is reached, it is determined that the edge pixel PXe1 is not specified.

When the edge pixel PXe is specified (S118: YES), in S120, the CPU 110 records the specified edge pixel PXe as the contour pixel PXo in the contour image data. Specifically, the pixel value in the contour image data corresponding to the specified edge pixel PXe is changed to "1".

If the edge pixel PXe is not specified (S118: NO), S120 is skipped. Therefore, in this case, the contour pixel PXo corresponding to the pixel of interest is not recorded.

In S125, the CPU 110 determines whether or not all the pixels on the attention end are processed as the attention pixels. If there are unprocessed pixels (S125: NO), the CPU 110 returns the processing to S110. When all the pixels are processed (S125: YES), the CPU 110 proceeds to S130.

In S130, the CPU 110 determines whether or not the ends TU, TB, TL, and TR of all the edge binary image EIs have been processed as the attention ends. If there is an unprocessed end (S130: NO), the CPU 110 returns the process to S105. When all the edges have been processed (S130: YES), the CPU 110 ends the contour specifying process. The contour identification process generates contour image data showing the contour image OLI shown in FIG. 3C.

According to the contour specifying process of this embodiment, the edge pixels PXe are searched from the edge of the edge binary image EI toward the inside, and the specific edge pixels PXe (for example, PXe1 to PXe4) searched first are contoured. Specified as pixel PXo. Therefore, the contour of the original image Ad can be easily and appropriately specified in the edge binary image EI.

A-4. Original area specifying process The document area specifying process of S40 of FIG. 2 will be described. As described above, the manuscript area specifying process is a process of specifying the manuscript area AA corresponding to the manuscript image Ad by using the contour image data. As shown in FIG. 2, the manuscript area specifying process includes determining the start straight line of S42 and specifying the manuscript area AA of S44.

FIG. 6 is a diagram showing a scanned image OI. FIG. 7 is an enlarged view of a part of the region AR of the scanned image OI of FIG. The region AR is a region including a part of the outer edge LPe on the left side of the original image Ad in the scanned image OI. The background image Ab described above exists outside the outer edge LPe of the original image Ad. Then, a shadow image SH showing a shadow exists between the outer edge LPe of the original image Ad and the background image Ab (FIG. 7). In FIG. 6, the shadow image SH is omitted in order to avoid complication of the drawing. The shadow image SH has a color (for example, black) darker than the color of the ground color region Sa of the original image Ad (for example, white) and the color of the background image Ab (for example, gray). Such a shadow image SH is, for example, a thin image generated due to a step between the facing surface Bs (FIG. 1) of the cover 450 and the original document to be read.

The start straight line Lt is a straight line indicating the start position of the search for specifying the outer edge LPe of the original image Ad in the specification of the document region AA of S44, which will be described later. The start straight line Lt is determined to be substantially parallel to the outer edge LPe on the left side of the original image Ad and outside the original image Ad. More specifically, the starting straight line Lt is determined at the position of the outer edge LSe of the shadow image SH. In the determination of the start straight line of S42, four start straight lines Lt, Ut, Rt, and Bt (FIG. 6) substantially parallel to each of the outer edges LPe, UPe, RPe, and BPe of the original image Ad are determined.

FIG. 8 is a flowchart showing a processing step of determining the starting straight line. In S310 of FIG. 8, a plurality of candidate straight lines are specified on the reduced image MI by using the contour image data. Specifically, the straight line represented by the contour pixel PXo is adopted as the candidate straight line. As a method of specifying the candidate straight line, a Hough transform using the contour pixel PXo is used. Since the Hough transform is a known process, its description will be omitted. As a result of the Hough transform, in the example of FIG. 3C, six candidate straight lines L1 to L6 corresponding to the contour of the original image Ad are specified. Specifically, information indicating the slopes and positions of the candidate straight lines L1 to L6 is acquired.

In S320, the CPU 110 selects four provisional straight lines parallel to the outer edge of the original image Ad from the six candidate straight lines L1 to L6. Specifically, the CPU 110 excludes straight lines located at the four ends (sides) of the outer shape of the contour image OLI from the six candidate straight lines L1 to L6. For example, in the example of FIG. 3A, the candidate straight line L2 located at the upper end of the contour image OLI is excluded. The straight lines (eg, L2) located at the four ends of the contour image OLI are located at the corresponding ends of the scanned image OI. It is considered that such a straight line L2 corresponds to the area Da (FIG. 3A) on the document that protrudes from the readable area at the time of reading, and does not indicate the edge of the document. The CPU 110 extracts two sets of two lines parallel to each other from the remaining candidate straight lines L1 and L3 to L6. In the example of FIG. 3A, a pair of straight lines L3 and L6 and a pair of straight lines L2 and L4 are extracted. These four straight lines L2, L3, L4, and L6 are selected as the four provisional straight lines. Since the inclinations of the four provisional straight lines L2, L3, L4, and L6 indicate the inclination of the original image Ad, the inclination of the original image Ad is determined by specifying the four provisional straight lines L2, L3, L4, and L6. Is identified.

In S322, the CPU 110 determines the reference straight lines L2C, L3C, and L4C corresponding to the four provisional straight lines L2, L3, L4, and L6 selected on the reduced image MI on the scanned image OI (FIG. 6) before reduction. , L6C is determined.

As shown in FIG. 7 for the reference straight line L3C substantially parallel to the outer edge LPe on the left side of the original image Ad, the reference straight lines L2C, L3C, L4C, and L6C are along the outer edge of the original image Ad when viewed from the original image Ad. It is located further outside the shadow image SH. In other words, the reference straight lines L2C, L3C, L4C, and L6C are located further outside the starting straight lines Lt, Ut, Rt, and Bt to be determined when viewed from the original image Ad. The reason for this is as follows.

The edge pixels specified by the edge binary image EI include pixels that form a boundary between the shadow image SH and the background image Ab and belong to the background image Ab. Then, the contour pixel PXo (FIG. 3 (C)) in the contour image OLI specified by using the edge binary image EI is viewed from the original image Ad as can be seen from the contour identification process (FIGS. 4 and 5). The edge pixel PXe located on the outer side is preferentially selected. Therefore, the provisional straight lines L2, L3, L4, and L6 specified in the reduced image MI based on the contour pixel PXo are located in the background image Ab. Then, the reference straight lines L2C, L3C, L4C, and L6C corresponding to the provisional straight lines L2, L3, L4, and L6 specified by the reduced image MI equivalent to 100 dpi are the read image OI before reduction (for example, equivalent to 300 to 1200 dpi). It is decided by. Therefore, the reference straight lines L2C, L3C, L4C, and L6C may be located several pixels outside the shadow image SH in the scanned image OI before reduction (FIG. 7).

As can be seen from the above description, in S322, it is a reference straight line (for example, the reference straight line L3C) along a specific outer edge (for example, outer edge LPe) of the original image Ad, and the original image Ad is from the specific outer edge of the original image Ad. A distant reference line is identified outside of.

In S325, the CPU 110 selects one reference straight line to be processed from the four reference straight lines L2C, L3C, L4C, and L6C. For example, the reference straight line L3C on the left side, which is substantially parallel to the outer edge LPe on the left side of the original image Ad, is selected as the reference straight line to be processed (FIG. 7). In S330, the CPU 110 sets one reference straight line to be processed as a straight line of interest.

In S335, the CPU 110 classifies the plurality of pixels located on the straight line of interest into a background pixel having a background color and a non-background pixel having a color different from the background color. Specifically, the CPU 110 classifies pixels having a color value within the background color range indicating a specific background color (in this embodiment, the color of the facing surface Bs of the cover 450) as the background pixel, and classifies the background color range. Pixels with outside color values are classified as non-background pixels. In this embodiment, since the specific background color is gray, the background color range is a range of color values indicating gray. The background color range is, for example, a range of color values in which the saturation C satisfies the saturation range CR1 and the luminance Y satisfies the luminance range YR1.

In this embodiment, the color of the facing surface Bs is gray (achromatic color), so the saturation range CR1 is the range of saturation C satisfying C ≦ Cth. The threshold value Cth is 32, for example, when the saturation C is represented by 256 gradations of 0 to 255.

The luminance range YR1 is, for example, a range centered on the luminance Ybs of the color of the facing surface Bs. Specifically, the luminance range YR1 is a range of luminance Y satisfying (Ybs−ΔY) ≦ Y ≦ (Ybs + ΔY). ΔY is 32, for example, when the luminance Y is represented by 256 gradations of 0 to 255. White (luminance 255) and black (luminance 0) are outside the brightness range YR1.

For example, in FIG. 7, the pixels constituting the background image Ab are classified into background pixels. Further, the pixels constituting the shadow image SH and the original image Ad are classified into non-background pixels.

In S340, the CPU 110 determines whether or not there are K or more consecutive non-background pixels on the line of interest. K is an integer of 2 or more, for example 20. If there are no consecutive K or more non-background pixels on the straight line of interest (S340: NO), the process proceeds to S342.

In S342, the CPU 110 determines whether or not the straight line of interest has reached the limit position of the search. FIG. 6 shows the search limit position LLa corresponding to the lower start straight line Bt. The limit position LLa is a position separated from the start straight line Bt in the direction of the arrow BM by a predetermined width Wa. It can be said that the arrow AMa is a direction perpendicular to the start straight line Bt and is a direction approaching the center of the original image Ad. The predetermined width Wa is, for example, the number of pixels corresponding to a width of 3 to 10 cm. When the search limit position is reached (S342: YES), the CPU 110 determines in S344 the reference straight line to be processed as the start straight line. If the search limit position has not been reached (S342: NO), the CPU 110 proceeds to S345.

In S345, the CPU 110 sets a straight line located on the original image Ad side by one pixel from the current straight line of interest as a new straight line of interest. For example, when the reference straight line L3C in FIG. 7 is the current straight line of interest, the adjacent straight line Lnx in FIG. 7 is set as a new straight line of interest. The adjacent straight line Lnx is a straight line parallel to the reference straight line L3C and adjacent to the reference straight line L3C toward the outer edge LPe of the original image Ad. In this way, the search for the start straight line is performed in the direction of the arrow AM in FIG. 7 while shifting the straight line of interest toward the outer edge of the original image Ad by one pixel.

When there are K or more consecutive non-background pixels on the line of interest (S340: YES), in S350, the CPU 110 determines the current line of interest as the start line corresponding to the reference line to be processed. To do. In the example of FIG. 7, for example, the start straight line Lt corresponding to the reference straight line L3C on the left side is determined at the position of the outer edge LSe on the left side of the shadow image SH.

In S355, the CPU 110 determines whether or not all the reference straight lines L2C, L3C, L4C, and L6C have been processed as processing targets. If there is an unprocessed reference line (S355: NO), the CPU 110 returns the process to S325. When all the reference straight lines have been processed (S355: YES), the determination of the start straight line ends. At this point, the four start lines Lt, Ut, Rt, and Bt (FIG. 6) corresponding to the four reference lines L2C, L3C, L4C, and L6C have been determined.

When the start straight lines Lt, Ut, Rt, and Bt are determined, the document region AA is specified in S44 of FIG. FIG. 9 is a flowchart showing a specific processing step of the document area AA.

In S400, the CPU 110 selects one start straight line to be processed from among the four start straight lines Lt, Ut, Rt, and Bt. For example, the left start straight line Lt substantially parallel to the left outer edge LPe of the original image Ad is selected as the start straight line to be processed (FIG. 7). In S405, the CPU 110 sets one start straight line to be processed as a straight line of interest. That is, one starting straight line to be processed is the first straight line of interest for the specific processing of the document area AA in FIG.

In S410, the CPU 110 classifies the plurality of pixels located on the straight line of interest into background pixels, original pixels, and shadow pixels. Specifically, the CPU 110 classifies pixels having a color value within the background color range indicating the above-mentioned specific background color as background pixels. Further, the CPU 110 classifies pixels having a color value within the document color range as document pixels, and classifies pixels having a color value within the shadow color range as shadow pixels.

The original color range is a range of color values in which the luminance Y satisfies the luminance range YR2 or the saturation C satisfies the saturation range CR2. The luminance range YR2 is a range of luminance Y higher than the upper limit value (Ybs + ΔY) of the above-mentioned luminance range YR1 that defines the background color range. That is, the luminance range YR2 is a range of luminance Y satisfying Y> (Ybs + ΔY). The saturation range CR2 is a range of saturation C higher than the upper limit value Cth of the above-mentioned saturation range CR1 that defines the background color range. That is, the saturation range CR2 is a range of saturation C that satisfies C> Cth.

The shadow color range is a range of color values in which the luminance Y satisfies the luminance range YR3 and the saturation C satisfies the saturation range CR1. The luminance range YR3 is a range of luminance Y lower than the lower limit value (Ybs−ΔY) of the above-mentioned luminance range YR1 that defines the background color range. That is, the luminance range YR3 is a range of luminance Y satisfying Y <(Ybs−ΔY). The saturation range CR1 is the same range as the above-mentioned saturation range CR1 that defines the background color range, that is, the range of saturation C that satisfies C ≦ Cth. As described above, since the shadow pixel has a color different from that of the background pixel and the original pixel, it is also called a different color pixel.

In S415, the CPU 110 calculates the ratio Ra of the original pixels to all the pixels on the straight line of interest. That is, the CPU 110 calculates the total number of pixels PNt on the straight line of interest and the number PNp of the original pixels on the straight line of interest, and calculates the ratio Ra (Ra = (PNp / Pnt)).

In S420, the CPU 110 determines whether or not the calculated ratio Ra is equal to or greater than a predetermined threshold value THa. The threshold THa is 5% in this example. When the calculated ratio Ra is less than the predetermined threshold value THA (S420: NO), the CPU 110 proceeds to S425. When the calculated ratio Ra is equal to or greater than the predetermined threshold value THa (S420: YES), the CPU 110 proceeds to S460.

In S425, the CPU 110 calculates the ratio Rb of the original pixel to the non-background pixel on the straight line of interest. That is, the CPU 110 calculates the number of shadow pixels PNs on the line of interest and the number of original pixels PNp, and calculates the ratio Rb (Pb = {PNp / (PNs + PNp)}).

In S430, the CPU 110 determines whether or not the calculated ratio Rb is equal to or greater than a predetermined threshold value THb. The threshold THb is 5% in this example. When the calculated ratio Rb is less than the predetermined threshold value THb (S430: NO), the CPU 110 proceeds to S435. When the calculated ratio Rb is equal to or higher than the predetermined threshold value THb (S430: YES), the CPU 110 proceeds to S440.

In S435, the CPU 110 resets the continuous count value Cc, that is, sets the continuous count value Cc to 0. The continuous count value Cc is a variable for counting the number of continuous lines of interest in which the ratio Rb is equal to or greater than the threshold value THb in the direction intersecting the straight lines of interest (direction of arrow AM in FIG. 7).

In S440, the CPU 110 counts up the continuous count value Cc, that is, adds 1 to the continuous count value Cc.

In S445, the CPU 110 determines whether or not the continuous count value Cc is a predetermined value M. The predetermined value M is, for example, the number of pixels corresponding to a width of 2 mm. When the continuous count value Cc is M, it means that there are M consecutive lines of interest in which the ratio Rb is equal to or higher than the threshold value THb. When the continuous count value Cc is M (S445: YES), the CPU 110 proceeds to S470. If the continuous count value Cc is less than M (S445: NO), the CPU 110 proceeds to S450.

In S450, the CPU 110 determines whether or not the straight line of interest has reached the limit position of the search. FIG. 6 shows the search limit position LL corresponding to the start straight line Lt on the left side. The limit position LL is a position separated from the start straight line Lt in the direction of the arrow AM by a predetermined width W. It can also be said that the arrow AM is a direction perpendicular to the start straight line Lt and is a direction approaching the center of the original image Ad. The predetermined width W is, for example, the number of pixels corresponding to a width of 3 to 10 cm. When the search limit position is reached (S455: YES), the CPU 110 determines in S475 the start straight line as the outer edge straight line. If the search limit position has not been reached (S455: NO), the CPU 110 proceeds to S455.

In S455, the CPU 110 sets a straight line located on the center side of the original image Ad by one pixel from the current straight line of interest as a new straight line of interest. In other words, a straight line parallel to the current straight line of interest and adjacent to the current straight line of interest toward the center of the original image Ad is set as a new straight line of interest. After S455, the CPU 110 returns the process to S410. In this way, from the start straight line (for example, the start straight line Lt), a straight line indicating the outer edge (for example, the outer edge LPe) of the original image Ad is searched toward the center of the original image Ad.

In S460, the CPU 110 determines whether or not the continuous count value Cc is 1 or more. In the straight line of interest immediately before the current straight line of interest, when the ratio Rb is equal to or greater than the threshold value THb (S430: YES), the continuous count value Cc is 1 or more. When the continuous count value Cc is 1 or more (S460: YES), the CPU 110 proceeds to S470. When the continuous count value Cc is less than 1, that is, zero (S460: NO), in S465, the CPU 110 determines the current straight line of interest as the outer edge straight line corresponding to the start straight line to be processed.

In S470, the CPU 110 determines the straight line of interest at the start of counting the continuous count value Cc as the outer edge straight line. That is, when the continuous count value Cc is 1 or more, at least two or more straight lines of interest having a ratio Rb of the threshold value THb or more are continuous. In this step, the first straight line of interest among the two or more continuous straight lines of interest is determined as the outer edge straight line. For example, if the continuous count value Cc is M, the first straight line of interest among the M consecutive straight lines of interest is determined as the outer edge straight line. As described above, the M continuous straight lines are sequentially selected as the straight lines of interest from the side away from the center of the original image Ad (outside) toward the side toward the center of the original image Ad (inside). Of the M continuous straight lines, the outermost straight line when viewed from the original image Ad is determined as the outer edge straight line.

In S480, the CPU 110 determines whether or not all the start straight lines Lt, Ut, Rt, and Bt have been processed as processing targets. If there is an unprocessed start straight line (S480: NO), the CPU 110 returns the process to S400. When all the start straight lines have been processed (S480: YES), the process proceeds to S485. At this point, the four outer edge straight lines corresponding to the four start straight lines Lt, Ut, Rt, and Bt, in other words, the four outer edge lines located at the four outer edge LPe, UPe, RPe, and BPe of the original image Ad. The outer edge straight line has been determined. For example, in the example of FIG. 7, the outer edge straight line Lf located at the outer edge LPe on the left side of the original image Ad is determined. The outer edge straight line Lf is substantially parallel to the outer edge LPe of the corresponding original image Ad, and is parallel to the corresponding start straight line Lt.

In S485, the CPU 110 identifies the document area AA based on the four determined outer edge straight lines. Specifically, the rectangular inner region formed by the four outer edge straight lines is specified as the document region AA. For example, the CPU 110 identifies the manuscript area AA defined by the four intersections CP1 to CP4 by specifying the four intersections CP1 to CP4 (FIG. 6) of the four outer edge straight lines.

A-5. Repair process The repair process of S45 in FIG. 2 will be described. As described above, the repair process is a process executed on the scanned image data using the specific result of the document area AA. FIG. 10 is a flowchart of the repair process.

In S510 of FIG. 10, a tilt correction process for correcting the tilt of the original image Ad is executed on the scanned image data. Specifically, for example, the angle of the left side of the document area AA (the outer edge straight line Lf on the left side) with respect to the vertical direction of the scanned image OI is calculated. Then, by rotating the scanned image OI by the angle, the rotation process of making the left side of the document region AA parallel to the vertical direction of the scanned image OI is executed.

In S520, the CPU 110 draws a missing portion of the original image Ad in the tilt-corrected scanned image OI. Specifically, in the tilt-corrected scanned image OI, a region outside the contour of the original image Ad specified by using the contour image OLI and inside the original region AA, specifically, a region inside the original region AA. , The regions Da and Db of FIG. 6 are specified. In the tilt-corrected scanned image OI, the values of the plurality of pixels in the regions Da and Db are replaced with values indicating the color of the ground color region Sa of the original image Ad. The value indicating the color of the ground color region Sa is, for example, the average value of the values of a plurality of pixels located along the contour of the original image Ad specified by using the contour image OLI and within the original image Ad. Used.

In S530, the CPU 110 changes the color of the background image Ab to the designated color in the read image OI in which the missing portion has been drawn. Specifically, in the tilt-corrected scanned image OI, the values of the plurality of pixels located outside the original area AA are replaced with values indicating a color (for example, white) specified in advance. As a result, the repaired image data showing the repaired image RI of FIG. 3D is generated.

According to the document area specifying process (FIGS. 8 and 9) of the present embodiment described above, the CPU 110 uses the scanned image data to determine the straight line of interest in the scanned image OI (FIG. 6) (S405, S455). ). The straight line of interest is a straight line parallel to any of the outer edges LPe, UPe, RPe, and BPe of the original image (FIGS. 6 and 7). The CPU 110 uses the scanned image data to classify the plurality of pixels located on the straight line of interest into three types of pixels including the original pixel, specifically, the original pixel, the background pixel, and the shadow pixel (S410). .. The CPU 110 has three types of pixels, the number of original pixels PNp, the number of three types of pixels including the original pixels (that is, the total number of pixels on the straight line of interest PNT), and the number of two types of pixels including the original pixels. The number (specifically, the sum of the number PNs of shadow pixels and the number PNp of original pixels ((PNs + PNp))) is calculated, and the ratios Ra and Rb are calculated (S415, S430). The CPU 110 determines whether or not the first condition for determining whether or not the straight line of interest is an outer edge straight line is satisfied (S420). The first condition is that the ratio Ra is equal to or higher than the threshold value THa, as described above. The CPU 110 determines whether or not the straight line of interest satisfies the second condition different from the first condition (S430 to S445). The second condition is M straight lines including the straight line of interest, and each of the M straight lines parallel to the straight line of interest and continuous in the direction intersecting the straight line of interest (direction of arrow AM in FIG. 7) is a specific condition. Satisfy (S430-S445). The specific condition is that the ratio Rb is equal to or higher than the threshold value THb (S430). When the straight line of interest satisfies the first condition (S420: YES), the CPU 110 determines an outer straight line (for example, the outer straight line Lf of FIG. 7) located at the outer edge of the original image Ad based on the straight line of interest (S460-). S470). Further, the CPU 110 determines the outer edge straight line based on the straight line of interest (S470) when the straight line of interest satisfies the second condition (S430: YES and S445: YES). The CPU 110 uses the outer edge straight line to specify the document area AA corresponding to the document image Ad. As a result, it can be determined whether or not the straight line of interest satisfies the first condition and whether or not the straight line of interest satisfies the second condition. Then, when the straight line of interest satisfies the first condition and when the straight line of interest satisfies the second condition, the outer edge straight line located at the outer edge of the original image Ad is determined based on the straight line of interest. As a result, the outer edge straight line can be appropriately determined. Therefore, the original region AA can be accurately specified by using the outer edge straight line.

FIG. 11 is a diagram showing a specific example of the document area. FIG. 11A shows an enlarged view ARa of a scanned image including a manuscript image Ada, a shadow image Sha, and a background image Aba showing a manuscript with a label sticker. This enlarged view ARa is shown by enlarging the vicinity of the outer edge LPea on the left side of the original image Ada. The manuscript image Ada includes a label image LB showing a label sticker protruding outward. In this example, the manuscript area AA is preferably specified so as to include the entire manuscript image Ada (including the label image LB), and the manuscript area AA such that a part of the manuscript image Ada protrudes is specified. Not preferred. Therefore, in the scanned image of FIG. 11 (A), the outer edge straight line that defines the document region AA is preferably determined at the position of the straight line La1 of FIG. 11 (A). For example, if the outer edge straight line is determined at the position of the straight line La2, the label image LB protrudes outside the document area AA, which is not preferable.

In the case of the scanned image of FIG. 11A, only the first condition described above is used. That is, when the ratio Ra of the original pixels to all the pixels on the straight line of interest is equal to or greater than the threshold value THa, the straight line of interest is specified as the outer edge straight line. In this case, the outer edge straight line is determined at the position of the straight line La2. For example, when the straight line La1 is a straight line of interest, most of the pixels on the straight line La1 are classified as background pixels indicating the background image Aba.

On the other hand, when the second condition described above is used, that is, the ratio Rb of the original pixel to the non-background pixels (original pixel + shadow pixel) on the straight line in each of the continuous M straight lines including the straight line of interest Rb. When is equal to or greater than the threshold THb, it is assumed that the outermost straight line seen from the original image Ada is specified as the outer edge straight line among the M straight lines. In this case, the outer edge straight line can be determined at the position of the straight line La1.

FIG. 11B shows an enlarged view ARb of a scanned image including a manuscript image Adb, a shadow image SHb, and a background image Abb showing a manuscript having a plurality of black ruled lines (for example, RL1 and RL2) in the vicinity of the edges. Is. The enlarged view ARb shows the vicinity of the outer edge LPeb on the left side of the original image Adb in an enlarged manner. In this example, the outer edge straight line is preferably determined at the position of the straight line Lb1 in order to identify the manuscript region AA including the entire manuscript image Adb.

In the case of the scanned image of FIG. 11B, only the above-mentioned second condition is used. In this case, when the straight line of interest is on the ruled line RL1 shown in the original image Adb, the ratio Rb is less than the threshold value THb. This is because the pixels on the ruled line RL1 are black, so that they are not classified as original pixels but are classified as shadow pixels. Therefore, when the M straight lines including the straight line Lb1 include a straight line on the ruled line RL1, the continuous count value Cc is reset, and as a result, the straight line Lb1 is not determined as the outer edge straight line.

On the other hand, when the first condition described above is used, when the straight line Lb1 is the straight line of interest, the ratio Ra is equal to or higher than the threshold value THa, so that the straight line Lb1 is appropriately determined as the outer edge straight line.

As can be seen from the above description, in the present embodiment, the outer edge straight line is determined by using the first condition and the second condition in combination. Therefore, for example, the scanned image data showing various types of scanned images can be obtained. When used, an appropriate manuscript area AA can be identified. For example, whether it is a scanned image including the original image Ada of FIG. 11 (A) or a scanned image including the original image Adb of FIG. 11 (B), an appropriate document area AA is provided on these scanned images. Can be identified.

Here, in the second condition, the reason why the ratio Rb is required to be equal to or higher than the threshold value THb for each of the M straight lines including the straight line of interest as well as the straight line of interest will be described. The ratio Rb uses the number of non-background pixels (PNp / Pnt) as the denominator instead of the total number of pixels PNT on the straight line of interest. Therefore, even when the number of original pixels PNp on the straight line of interest is relatively small, the ratio Rb tends to be equal to or higher than the threshold value THb. Therefore, if the judgment is made by paying attention only to the straight line of interest, for example, there is a high possibility that the outer edge straight line is erroneously determined at a place irrelevant to the outer edge of the original image Ad due to the influence of noise or the like. In the second condition of this embodiment, since the determination is made by paying attention to each of the M straight lines including the straight line of interest, it is possible to suppress the determination of an erroneous outer edge straight line.

Further, according to the above embodiment, the scanned image OI includes a background image Ab and a shadow image SH having a color different from the background color as an image showing a region other than the original image Ad (FIG. 7). The CPU 110 calculates the ratio Ra using three types of pixels of the original pixel, the background pixel, and the shadow pixel, and uses the two types of pixels of the original pixel and the shadow pixel without using the background pixel. Calculate Rb. As a result, the original region AA can be accurately specified by using the ratio Ra and the ratio Rb.

Further, according to the above embodiment, the CPU 110 can determine the current straight line of interest as the outer edge straight line when the straight line of interest satisfies the first condition (S420: YES) (S465). As a result, an appropriate outer edge straight line can be determined.

Further, according to the above embodiment, when the straight line of interest satisfies the first condition (S420: YES) and the continuous count value Cc is 1 or more (S460: YES), the CPU 110 has a continuous count value. The straight line of interest at the start of counting Cc is determined as the outer edge straight line (S470). When the continuous count value Cc is 1 or more, the ratio Rb is the threshold value THb in each of the K straight lines (K is an integer less than M) continuous in the direction opposite to the arrow AM in FIG. 7 from the straight line of interest. It means that it is the above. The direction opposite to the arrow AM is a direction that intersects the straight line of interest and is a direction away from the center of the original image Ad. Determining the straight line of interest at the start of counting of the continuous count value Cc as the outer edge straight line means that the straight line located at the end of the continuous K straight lines in the direction opposite to the arrow AM is determined as the outer edge straight line. means. According to this configuration, among the K continuous straight lines, the straight line located at the end in the direction away from the center of the original image Ad is determined as the outer edge straight line. As a result, an appropriate manuscript area AA including the entire manuscript image Ad can be determined. As described above, it is preferable that the outer edge straight line is determined so that the original image Ad does not protrude outside the original area AA. For this reason, when there are a plurality of straight lines that may be outer edge straight lines, it is appropriate to determine the straight line away from the center of the original image Ad as the outer edge straight line among the plurality of straight lines. it is conceivable that.

Further, according to the above embodiment, when the straight line of interest satisfies the second condition (S430: YES and S445: YES), the CPU 110 has the arrow AM of FIG. 7 out of the continuous M straight lines. A straight line located at the end in the opposite direction is determined as the outer edge straight line (S470). As a result, an appropriate manuscript area AA including the entire manuscript image Ad can be determined.

Further, according to the above embodiment, the CPU 110 acquires analysis information for identification for determining the straight line of interest by analyzing the reduced image data. Specifically, the CPU 110 generates edge binary image data using the reduced image data (S20 and S25 in FIG. 2), and generates contour image data using the edge binary image data (FIG. 2). S30), four provisional straight lines L2, L3, L4, and L6 are determined on the reduced image MI using the contour image data (S310 to S320 in FIG. 8). The information indicating the provisional straight lines L2, L3, L4, and L6, specifically, the information acquired by the Hough transform in S310 can be said to be the analysis information for identifying the straight line of interest. Since the reduced image data is analyzed in this way, the processing time for determining the straight line of interest, and by extension, the processing time for determining the outer edge straight line can be reduced. Further, even when the analysis is performed using the reduced image data, the outer edge straight line can be appropriately determined, so that the document region AA in the scanned image OI can be accurately specified.

Further, according to the above embodiment, the CPU 110 specifies the inclination of the original image Ad by specifying the four provisional straight lines L2, L3, L4, and L6 in S320 of FIG. When specifying the document area AA in FIG. 9, the CPU 110 determines a straight line of interest parallel to any of the reference straight lines L2C, L3C, L4C, and L6C corresponding to the four provisional straight lines L2, L3, L4, and L6. (S405, S455). As a result, a straight line of interest having an inclination corresponding to the inclination of the original image Ad is appropriately determined.

Further, according to the above embodiment, the CPU 110 identifies a reference straight line L3C (FIG. 7) away from the specific outer edge LPe (FIG. 7) of the document image Ad outside the document image Ad (S322 in FIG. 8). A straight line satisfying the search condition is searched from the reference straight line L3C toward the specific outer edge LPe (S330 to S345). The search condition is that there are K or more consecutive non-background pixels on the target straight line (S340). Then, the CPU 110 first determines a straight line satisfying the search condition as a starting straight line, that is, as the first straight line of interest in specifying the document area AA in FIG. 9 (S350 in FIG. 8). According to this configuration, after the reference straight line L3C is specified, the straight line of interest is determined from the reference straight line L3C toward the specific outer edge LPe using a relatively simple search condition. The processing time can be reduced. For example, in the determination of the first condition and the second condition used for specifying the document area AA in FIG. 9, a plurality of pixels on the line of interest are defined as three types of pixels, a background pixel, a document pixel, and a shadow pixel. However, in the determination of the above search conditions, it is sufficient to classify the pixels into two types of pixels, a background pixel and a non-background pixel.

Further, according to the above embodiment, in S450 of FIG. 9, when the straight line of interest reaches the limit position of the search (S450: YES), the starting straight line is determined as the outer edge straight line (S475). When the straight line of interest reaches the limit position of the search, it is considered difficult to determine the outer edge straight line at the position of the outer edge of the original image Ad using the first condition and the second condition. In this case, since the start straight line is determined as the outer edge straight line, it is possible to increase the possibility that the outer edge straight line is determined outside the original image Ad. As a result, an appropriate document area AA can be determined so that a part of the document image Ad does not protrude.

Further, according to the above embodiment, in S342 of FIG. 8, when the straight line of interest reaches the limit position of the search (S342: YES), the reference straight line is determined as the starting straight line (S344). When the straight line of interest reaches the limit position of the search, it is considered difficult to determine the start condition under the determination condition of S340. In this case, since the reference straight line is determined as the start straight line, the start straight line can be reliably determined outside the original image Ad. If the start straight line is erroneously determined inside the original image Ad, there is a high possibility that the outer edge straight line cannot be appropriately determined by the process of FIG. In this embodiment, it is possible to avoid such inconveniences and increase the possibility that the outer edge straight line can be appropriately determined by the process of FIG.

B. Modification example:
(1) In the identification of the document area AA of the above embodiment (FIG. 9), the plurality of pixels located on the straight line of interest are classified into three types of pixels (S410). Instead of this, these plurality of pixels may be classified into four types of pixels, for example, a document pixel, a background pixel, a shadow pixel, and another pixel. For example, when it is known that the color of the ground color region Sa of the document is white, the pixels having the color value indicating white are classified as the document pixels, and the pixels having the color value indicating the background color are classified as the background pixels. Pixels having a color indicating black may be classified as shadow pixels, and pixels that are neither of them may be classified into other pixels. Further, in this case, the ratio Ra used for determining the first condition may be the ratio of the original pixel to the three types of pixels excluding the other pixels. The ratio Rb used for determining the second condition may be the ratio of the original pixels to the two types of pixels, the shadow pixels and the original pixels.

Further, the plurality of pixels located in the straight line of interest are classified into five types of pixels, the ratio Ra is the ratio of the original pixel to the four types of pixels, and the ratio Rb is three types of them. It may be the ratio of the original pixel to the pixel of. The pixels used for calculating the ratio Ra may include original pixels, for example, may not include at least one of background pixels and shadow pixels, or may include other types of pixels (for example, noise pixels). Further, the pixels used for calculating the ratio Rb may include original pixels, and may not include shadow pixels or may include background pixels, for example.

Generally speaking, a plurality of pixels located on the straight line of interest are classified into N types of pixels including original pixel (N is an integer of 3 or more), and the ratio Ra is the original pixel among the N types of pixels. Is the ratio of the original pixel to the number of N1 type pixels (N1 is an integer satisfying 3 ≦ N1 ≦ N) including, and the ratio Rb is N2 type pixels (N2) including the original pixel among the N types of pixels. May be the ratio of the original pixels to 2 ≦ N2 <an integer satisfying N1).

(2) In the above embodiment, the scanned image data is used to specify the document region AA corresponding to the document image Ad in the scanned image OI indicated by the scanned image data. Instead of this, a manuscript region corresponding to the manuscript image in the image represented by other image data generated using the scanned image data may be specified. The other image data may be, for example, enlarged image data obtained by executing an enlargement process for enlarging the image on the read image data. Generally, the target image data based on the scanned image data (for example, the scanned image data itself or the enlarged image data) may be used to specify the manuscript area in the target image indicated by the target image data. ..

(3) In S465 of FIG. 9 of the above embodiment, the current straight line of interest is determined as the outer edge straight line, but the present invention is not limited to this. For example, in order to further reduce the possibility that the original image Ad protrudes from the original area AA, it is a straight line separated from the center of the original image Ad by a predetermined pixel (for example, 1 or 2 pixels) from the current straight line of interest. A straight line parallel to the straight line of interest may be determined as the outer edge straight line.

(4) In FIG. 9 of the above embodiment, S460 may be omitted. In this case, for example, when the straight line of interest satisfies the first condition (S420: YES), the CPU 110 may always determine the current straight line of interest as the outer edge straight line in S465.

(5) In FIG. 9 of the above embodiment, when the straight line of interest satisfies the second condition (S430: YES and S445: YES), the straight line of interest at the start of counting the continuous count value Cc is set as the outer edge straight line. It is determined (S470). Instead, for example, a straight line of interest or a straight line that is separated from the center of the original image Ad by a predetermined pixel (for example, 1 or 2 pixels) from the current straight line of interest and is parallel to the straight line of interest. , May be determined as an outer edge straight line.

(6) The processing of S325 to S355 in FIG. 8 of the above embodiment may be omitted. In this case, in S400 of FIG. 9, one start straight line is selected from the four reference straight lines L2C, L3C, L4C, and L6C determined in S322, and in S410, the selected start line is selected. A straight line (reference straight line) is set as the first straight line of interest.

(7) In S310 of FIG. 8 of the above embodiment, the slope and position of the candidate straight line are determined by using the Hough transform. Alternatively, the least squares method may be used to determine the slope and position of the candidate straight line. Further, for example, assuming that the original image Ad is not tilted, the four reference straight lines determined in S322 may be straight lines located on the four sides of the outer shape of the scanned image OI. In this case, S310 and S320 in FIG. 8 may be omitted.

(6) The image processing step of FIG. 2 can be omitted or modified as appropriate. For example, S15 in FIG. 2 of the above embodiment is omitted, and the reduced image data may not be generated. In this case, the processes S20 to S30 of FIG. 2 may be executed using the unreduced scanned image data.

Further, in S20 and S25, edge binary image data is generated using the scanned image data, but instead, simple binary image data may be generated. The simple binary image data is, for example, binary image data obtained by classifying a plurality of pixels in the scanned image OI into a background pixel having a background color and a non-background pixel having another color. is there. In this case, in S115 of the contour specifying process of FIG. 4, non-background pixels are searched instead of edge pixels, and in S120, the first searched non-background pixels are recorded as contour pixels.

(8) The repair process of FIG. 10 can be changed as appropriate. For example, one or two steps may be omitted out of the three steps S510 to S530. Further, instead of the repair process, the specific result of the document area AA may be used for another image process, for example, a trimming process.

(9) The device for optically reading the document may be various other devices capable of optically reading the document, instead of the scanner 400 described with reference to FIG. For example, the scanner 400 may be a type of scanner that includes an automatic document feeder for transporting documents and reads the conveyed documents with a stopped image sensor 420. Further, the device that optically reads the document may be a digital camera that optically reads the document by using a two-dimensional image sensor.

Further, the CPU 110 of the image processing device 100 is not shown, which is connected to the non-volatile storage device 130 or the image processing device 100, instead of acquiring the scanned image data from the scanner 400 connected to the image processing device 100. The scanned image data stored in advance in an external storage device (for example, a USB memory) may be acquired.

(10) The image processing of FIG. 2 may be executed by a device of a type different from that of a personal computer. For example, the image processing may be executed by the CPU 411 of the scanner (for example, the scanner 400). Image processing may be performed by a server device connected to the network. The image processing may be partially shared and executed by a plurality of devices (for example, a computer) capable of communicating with each other via a network. In this case, the entire plurality of devices corresponds to the image processing device.

In each of the above embodiments, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part or all of the configuration realized by the software may be replaced with the hardware. May be good.

Further, when a part or all of the functions of the present invention are realized by a computer program, the program is provided in a form stored in a computer-readable recording medium (for example, a non-temporary recording medium). be able to. The program may be used while being stored on the same or different recording medium (computer-readable recording medium) as it was provided. The "computer-readable recording medium" is not limited to a portable recording medium such as a memory card or a CD-ROM, but is connected to an internal storage device in the computer such as various ROMs or a computer such as a hard disk drive. It may also include an external storage device.

Although the present invention has been described above based on Examples and Modifications, the above-described embodiments of the invention are for facilitating the understanding of the present invention and do not limit the present invention. The present invention can be modified and improved without departing from the spirit and claims, and the present invention includes equivalents thereof.

100 ... image processing device, 110 ... CPU, 120 ... volatile storage device, 130 ... non-volatile storage device, 132 ... program, 140 ... display unit, 150 ... operation unit, 190 ... interface, 400 ... scanner, 410 ... control unit 411 ... CPU, 412 ... Volatile storage device, 413 ... Non-volatile storage device, 414 ... Program, 419 ... Interface, 420 ... Image sensor, 430 ... Mobile device, 440 ... Document stand, 450 ... Cover, PXe ... Edge pixel , AA ... original area, SH ... shadow image, EI ... edge binary image, RI ... repaired image, OI ... scanned image, MI ... reduced image, LL ... limit position, NT ... network, Sa ... ground color area, Ab ... Background image, Cc ... Continuous count value, Ad ... Original image, Ob1, Ob2 ... Object, OLI ... Contour image

Claims (9)

It is an image processing device
An image data acquisition unit that acquires scanned image data generated by optically scanning a document, and an image data acquisition unit.
A focus straight line determining unit that determines a straight line of interest in the target image indicated by the target image data using the target image data based on the scanned image data, and the target image includes a manuscript image showing the manuscript. , The attention straight line is a straight line parallel to a specific outer edge of the original image.
Using the target image data, a classification unit that classifies a plurality of pixels located on the straight line of interest into N types of pixels (N is an integer of 3 or more) including the original pixels constituting the original image. ,
Among the N types of pixels, the number of the manuscript pixels, the number of N1 types of pixels including the manuscript pixels (N1 is an integer satisfying 3 ≦ N1 ≦ N), and N2 types of pixels including the manuscript pixels ( N2 is the number of 2 ≦ N2 <an integer satisfying N1), and the number calculation unit for calculating
A first condition determination unit for determining whether or not the straight line of interest satisfies the first condition, and the first condition is that of the original pixel among the N1 type pixels located on the straight line of interest. The first condition determination unit, wherein the ratio is equal to or higher than the first threshold value.
A second condition determination unit for determining whether or not the line of interest satisfies a second condition different from the first condition, and the second condition is parallel to the straight line of interest and the straight line of interest. Each of the M straight lines (M is an integer of 2 or more) continuous in the direction intersecting with the specific condition is satisfied, the M straight lines include the attention straight line, and the specific condition is the target. The second condition determination unit, wherein the ratio of the original pixel to the N2 types of pixels located in a straight line is equal to or greater than the second threshold value.
When the attention straight line satisfies the first condition, the first outer edge straight line determining unit that determines the outer edge straight line located at the outer edge of the original image based on the attention straight line,
A second outer edge straight line determining unit that determines the outer edge straight line based on the attention straight line when the attention straight line satisfies the second condition.
A region specifying portion that specifies a document region corresponding to the document image in the target image using the outer edge straight line, and a region specifying portion.
An image processing device. The image processing apparatus according to claim 1.
The target image includes, as an image showing a region other than the original image, a background image having a specific background color and a different color image having a color different from the specific background color.
The N1 type of pixel includes the original pixel, the background pixel constituting the background image, and the different color pixel constituting the different color image.
An image processing apparatus in which the N2 types of pixels include the original pixel and the different color pixel, and do not include the background pixel. The image processing apparatus according to claim 1, further
The target image includes a background image having a specific background color as an image showing a region other than the original image.
The line of interest determination unit
A reference straight line that is a reference straight line along a specific outer edge of the original image and is separated from the specific outer edge of the original image to the outside of the original image is specified.
A straight line satisfying a predetermined condition is searched from the reference straight line toward the specific outer edge.
First, a straight line satisfying the predetermined condition is determined as the first straight line of interest, and then the straight line is determined.
The predetermined condition is an image processing apparatus in which a specific number of pixels are continuous on a straight line of interest, and the specific number of pixels having a color different from the specific background color is present. The image processing apparatus according to any one of claims 1 to 3.
The first outer edge straight line determining unit is an image processing device that determines the straight line of interest as the outer edge straight line when the straight line of interest satisfies the first condition. The image processing apparatus according to any one of claims 1 to 4.
In the first outer edge straight line determination unit, each of the K straight lines (K is an integer less than M) continuous from the attention straight line in a specific direction while the attention straight line satisfies the first condition is specified. When the condition is satisfied, the straight line located at the end in the specific direction among the K continuous straight lines is determined as the outer edge straight line.
An image processing device in which the specific direction intersects the straight line of interest and is away from the center of the original image. The image processing apparatus according to any one of claims 1 to 5.
When the attention straight line satisfies the second condition, the second outer edge straight line determination unit is in the direction intersecting the attention straight line among the continuous M straight lines and is the center of the manuscript image. An image processing device that determines a straight line located at an end in a direction away from the outer edge straight line as the outer edge straight line. The image processing apparatus according to any one of claims 1 to 6, further comprising.
A reduction processing unit that executes a reduction process for reducing an image using the scanned image data to generate reduced image data indicating the reduced image, and a reduction processing unit.
An analysis unit that acquires analysis information for identification for determining the straight line of interest by analyzing the reduced image data, and an analysis unit.
With
The target image is a scanned image indicated by the scanned image data on which the reduction process has not been executed.
The attention straight line determination unit determines the attention straight line in the read image by using the analysis information.
The first outer edge straight line determining unit and the second outer edge straight line determining unit are image processing devices that determine the outer edge straight line in the scanned image. The image processing apparatus according to any one of claims 1 to 7.
A tilt specifying unit for specifying the tilt of the original image in the target image by analyzing the scanned image data is provided.
The attention straight line determining unit is an image processing device that determines the attention straight line having an inclination corresponding to the inclination of the specified original image. It ’s a computer program,
An image data acquisition function that acquires scanned image data generated by optically scanning a document, and
It is a attention straight line determination function for determining a line of interest in a target image indicated by the target image data using the target image data based on the scanned image data, and the target image includes a manuscript image showing the manuscript. The attention straight line is a straight line parallel to a specific outer edge of the original image.
Using the target image data, a classification function for classifying a plurality of pixels located on the straight line of interest into N types of pixels (N is an integer of 3 or more) including the original pixels constituting the original image. ,
Among the N types of pixels, the number of the manuscript pixels, the number of N1 types of pixels including the manuscript pixels (N1 is an integer satisfying 3 ≦ N1 ≦ N), and N2 types of pixels including the manuscript pixels ( N2 is the number of 2 ≦ N2 <an integer that satisfies N1), and the number calculation function that calculates
It is a first condition determination function for determining whether or not the straight line of interest satisfies the first condition, and the first condition is the original pixel occupying the N1 type of pixel located on the straight line of interest. The first condition determination function that the ratio is equal to or higher than the first threshold value, and
It is a second condition determination function for determining whether or not the attention straight line satisfies a second condition different from the first condition, and the second condition is parallel to the attention straight line and the attention straight line. Each of the M straight lines (M is an integer of 2 or more) continuous in the direction intersecting with the specific condition is satisfied, the M straight lines include the attention straight line, and the specific condition is the target. The second condition determination function, wherein the ratio of the original pixel to the N2 types of pixels located on a straight line is equal to or greater than the second threshold value.
A first outer edge straight line determination function that determines an outer edge straight line located at the outer edge of the original image based on the attention straight line when the attention straight line satisfies the first condition.
A second outer edge straight line determination function that determines the outer edge straight line based on the attention straight line when the attention straight line satisfies the second condition.
An area identification function for specifying a document area corresponding to the document image in the target image using the outer edge straight line, and
A computer program that makes a computer realize.

Images