JP7804476B2 - Image processing device, image processing method and program - Google Patents

Description

This disclosure relates to an image processing device, etc.

Conventionally, when a user uses an image processing device such as a multifunction peripheral, they may transport a document using an automatic document feeder (ADF) and then have that document read by an image reading device such as a scanner. At this time, dust or debris on the glass or mirror can sometimes cause so-called scanner streaks in the scanned image.

Various techniques have been proposed for detecting and correcting scanner streaks in scanned images. For example, one proposed technique determines line noise pixels based on the displacement of RGB component readouts relative to neighboring pixels in the document feed direction when the pixels differ by a predetermined amount or more in the RGB component readouts. The RGB component readouts of the line noise pixels are then corrected based on the RGB component readouts of the surrounding pixels (see, for example, Patent Document 1). Another proposed technique calculates the density change rate in the main scanning direction, identifies potential streak pixels based on the density change rate between the target pixel and a pixel a predetermined distance away from the target pixel, and determines that a streak image has occurred if there are consecutive potential streak pixels in the sub-scanning direction that exceed a threshold (see, for example, Patent Document 2). Another proposed technique detects the background color, selects foreign matter to be corrected based on its density, and performs density correction using interpolation based on that density (see, for example, Patent Document 3).

Patent No. 4591531 Patent No. 5008918 JP 2016-066836 A

Scanner streaks can sometimes be caused by foreign matter such as paper dust on the glass. Furthermore, paper dust and other particles can move when the document passes through the document feeder. Scanner streaks caused by such moving foreign matter can change location with each scan. Therefore, image processing devices need to detect abnormal pixel areas that make up scanner streaks from the scanned image in real time and correct these abnormal pixel areas. However, this has not been taken into consideration in the prior art, such as the patent documents mentioned above.

In consideration of the above-mentioned problems, the present disclosure aims to provide an image processing device and the like that can appropriately detect abnormal pixel areas and correct the detected abnormal pixel areas.

To solve the above-mentioned problems, the image processing device disclosed herein includes an acquisition unit that acquires an image, and a control unit. The control unit detects a target pixel as a candidate abnormal pixel based on pixel changes in multiple pixels in the main scanning direction to which the target pixel belongs, in the image acquired by the acquisition unit. If the pixels detected as candidate abnormal pixels have continuity in the sub-scanning direction, the control unit detects an area in the sub-scanning direction that includes the pixels detected as candidate abnormal pixels as an abnormal pixel area, and corrects the abnormal pixel based on the detection result of the abnormal pixel area.

The image processing method disclosed herein also includes the steps of: acquiring an image; detecting a pixel of interest in the image as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel belongs; if the pixels detected as candidate for an abnormal pixel have continuity in the sub-scanning direction, detecting an area in the sub-scanning direction that includes the pixel detected as the candidate for an abnormal pixel as an abnormal pixel area; and correcting the abnormal pixel based on the detection result of the abnormal pixel area.

The program disclosed herein also has the following features: it causes a computer to acquire an image; detect a target pixel of the image as a candidate for an abnormal pixel based on pixel changes in multiple pixels in the main scanning direction to which the target pixel belongs; if the pixels detected as candidate for an abnormal pixel have continuity in the sub-scanning direction, it detects an area in the sub-scanning direction that includes the pixel detected as a candidate for an abnormal pixel as an abnormal pixel area; and corrects the abnormal pixel based on the detection result of the abnormal pixel area.

This disclosure makes it possible to provide an image processing device or the like that can appropriately detect abnormal pixel areas and correct the detected abnormal pixel areas.

1 is a perspective view of an appearance of an image forming apparatus according to a first embodiment. FIG. 2 is a diagram for explaining the functional configuration of the image forming apparatus according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data configuration of parameter information in the first embodiment. FIG. 4 is a diagram illustrating an example of a data configuration of region separation information according to the first embodiment. FIG. 10 is a diagram illustrating an example of a data configuration of a provisional scanner line determination result in the first embodiment. FIG. 10 is a diagram illustrating an example of a data configuration of a scanner line determination result in the first embodiment. FIG. 4 is a flowchart showing the flow of main processing in the first embodiment. FIG. 10 is a flowchart showing the flow of first scanner line detection processing in the first embodiment. FIG. 10 is a flowchart showing the flow of first scanner line detection processing in the first embodiment. FIG. 10 is a flowchart showing the flow of second scanner line detection processing in the first embodiment. FIG. 10 is a flowchart showing the flow of scanner streak correction processing in the first embodiment. 10A and 10B are diagrams for explaining a case where scanner streaks occur. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a diagram illustrating an example of operation in the first embodiment. FIG. 10 is a flowchart showing the flow of scanner streak correction processing in the second embodiment. FIG. 10 is a diagram illustrating an example of operation in the second embodiment. FIG. 10 is a diagram illustrating an example of operation in the second embodiment. FIG. 10 is a diagram illustrating an example of operation in the second embodiment. FIG. 10 is a diagram illustrating an example of operation in the second embodiment.

One embodiment for carrying out the present disclosure will be described below with reference to the drawings. Note that the following embodiment is an example for explaining the present disclosure, and the technical scope of the invention described in the claims is not limited to the following description.

1. First embodiment
First, a first embodiment will be described. In the first embodiment, a case will be described in which an image processing apparatus according to the present disclosure is applied to an image forming apparatus 10. The image forming apparatus 10 is an information processing apparatus having a copy function, a scan function, a document print function, etc., and is also called an MFP (Multi-Function Printer/Peripheral, or multifunction device).

[1.1 Functional configuration]
The functional configuration of an image forming apparatus 10 of this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is an external perspective view of the image forming apparatus 10, and Fig. 2 is a block diagram showing the functional configuration of the image forming apparatus 10.

As shown in FIG. 2, the image forming device 10 is configured with a control unit 100, an image input unit 120, an image forming unit 130, a display unit 140, an operation unit 150, a memory unit 160, and a communication unit 190.

The control unit 100 is a functional unit for controlling the entire image forming apparatus 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage unit 160, and is configured, for example, by one or more arithmetic units (CPUs (Central Processing Units)). The control unit 100 may also be configured as a SoC (System on a Chip) that has multiple functions among those described below.

The control unit 100 functions as an image processing unit 102 and an area separation processing unit 104 by executing programs stored in the memory unit 160.

The image processing unit 102 performs various image-related processes. For example, the image processing unit 102 performs sharpening and tone conversion processes on the image of the original document read and input by the image input unit 120.

The region separation processing unit 104 performs region separation processing. Region separation processing is processing that separates an image into specified regions. For example, the region separation processing unit 104 separates the image into regions of various types, such as character regions, line regions, halftone dot regions, and background regions. Note that the types of regions separated by the region separation processing unit 104 are not limited to those described above, and regions such as halftone dot character regions may also be separated. Furthermore, the region separation processing unit 104 may separate regions such as regions containing black text (black text regions) and regions containing chromatic text, such as red (color text regions), from the image by determining the color of the text contained in the text regions.

The image input unit 120 is an image acquisition unit that acquires images and inputs the acquired images to the image forming device 10. For example, the image input unit 120 is composed of an automatic document feeder (ADF) and a scanner device that reads images from documents. The scanner device converts images into electrical signals using an image sensor such as a charge coupled device (CCD) or contact image sensor (CIS), and then quantizes and encodes the electrical signals. When the scanner device reads an image from a document, the image from the document is input to the image forming device 10 as digital data.

The image forming unit 130 forms (prints) an image on a recording medium such as recording paper. The image forming unit 130 is configured, for example, by a printing device such as a laser printer that uses electrophotography. The image forming unit 130 feeds recording paper from the paper feed tray 132 in FIG. 1, forms an image on the surface of the recording paper, and ejects the recording paper from the paper ejection tray 134, for example.

The display unit 140 displays various types of information. The display unit 140 is composed of a display device such as an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, or a micro LED (Light Emitting Diode) display.

The operation unit 150 accepts operation instructions from a user using the image forming apparatus 10. The operation unit 150 is composed of input devices such as key switches (hard keys) and touch sensors. The touch sensor can detect input by contact (touch) using any common detection method, such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitance method. The image forming apparatus 10 may also be equipped with a touch panel in which the display unit 140 and the operation unit 150 are integrally formed.

The storage unit 160 stores various programs and data necessary for the operation of the image forming device 10. The storage unit 160 is configured, for example, by a storage device such as a semiconductor memory such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

The memory unit 160 reserves a parameter information memory area 162 as a memory area, and further stores region separation information 164, provisional scanner line determination result information 166, and scanner line determination result information 168.

The parameter information storage area 162 stores information on various parameters (parameter information). For example, as shown in FIG. 3, the parameter information includes a parameter name (e.g., "Th1") and a parameter value for that parameter name (e.g., "80").

In this embodiment, four types of parameter values, Th0, Th1, Th2, and Th3, are stored. The parameter values may be predetermined or may be set by the user. Furthermore, the parameter values may be individual values for each image forming device.

Th0 is the threshold value for the pixel value when determining that the pixel of interest is a white pixel. Th1 and Th2 are threshold values for the density difference between the pixel of interest and its surrounding pixels. Th0, Th1, and Th2 are used in the first scanner line detection process, which detects candidate abnormal pixels. The first scanner line detection process will be described later.

In this embodiment, abnormal pixels refer to pixels that make up scanner lines. Also, in this embodiment, pixels that are candidates for abnormal pixels are also referred to as provisional scanner lines.

Th3 is a threshold for the continuity in the sub-scanning direction of pixels that are candidates for abnormal pixels, and is used in the second scanner line detection process described below.

The region separation information 164 is information indicating the results of the region separation process (region separation result). Region separation information 164 may be any information that can identify the range of a region and the type of region within that range. For example, as shown in Figure 4, it may be a table that stores values indicating the type of region for each pixel of the image.

Here, the position of a pixel in an image is expressed by coordinates. Coordinates are expressed as (x, y), where the top left pixel of the image is the origin, x is the number of pixels in the main scanning direction from the origin to the pixel of interest, and y is the number of pixels in the sub-scanning direction.

The value indicating the type of area is, for example, an integer value. In this case, the integer value corresponds to the type of area, with "0" indicating a character area, "1" a line area, "2" a halftone dot area, and "3" a halftone dot character area. For example, P100 in Figure 4 has a value of "1" for the pixel at position (x, y) = (10, 3) in the image, indicating that the pixel in question is included in the line area.

The provisional scanner line determination result information 166 is information that indicates, for each pixel in the image, the determination result as to whether the pixel is a candidate for an abnormal pixel.

The provisional scanner line determination result information 166 is, for example, a table that stores a value indicating whether or not each pixel of an image is a candidate for an abnormal pixel, as shown in Fig. 5. In this embodiment, the value stored in the provisional scanner line determination result information 166 is one of the following values.
Wh: Indicates that the pixel is not a candidate for an abnormal pixel and is a white pixel.
×: Indicates that the pixel is not a candidate for an abnormal pixel and is a pixel other than white.
1R, 1G, 1B: Indicates that the pixel is a candidate for an abnormal pixel.

Note that "1R" indicates a pixel whose red component density difference from surrounding pixels exceeds Th1, and whose other color component density difference is less than Th2. "1G" indicates a pixel whose green component density difference from surrounding pixels exceeds Th1, and whose other color component density difference is less than Th2. "1B" indicates a pixel whose blue component density difference from surrounding pixels exceeds Th1, and whose other color component density difference is less than Th2.

For example, P102 in Figure 5 indicates that the value for the pixel at position (x,y) = (10,3) in the image is "X", which means that the pixel is not a candidate for an abnormal pixel.

Scanner streak determination result information 168 is information indicating the result of determining whether or not scanner streaks have occurred on each line (pixel row) in the sub-scanning direction of the image. For example, as shown in FIG. 6, scanner streak determination result information 168 is a table that stores a line number (e.g., "0") indicating the position (row) of the line in the sub-scanning direction of the image, in association with a value (e.g., "Yes") indicating the determination result of whether or not scanner streaks have occurred on that line. The value indicating whether or not scanner streaks have occurred may be any information that can distinguish the presence of scanner streaks. Therefore, either "Yes" or "No" may be stored, or either "0" or "1" may be stored, or either a value other than Null or Null may be stored. For example, P104 in FIG. 6 indicates that scanner streaks have occurred on the eighth line of the image, since the value corresponding to line number "8" is "Yes."

The communication unit 190 communicates with external devices via a LAN (Local Area Network) or WAN (Wide Area Network). The communication unit 190 is composed of a communication device or communication module, such as a NIC (Network Interface Card) used in wired/wireless LANs.

[1.2 Processing flow]
The flow of processing executed by the image forming apparatus 10 in this embodiment will be described with reference to Figures 7 to 11. The processing shown in Figures 7 to 11 is executed by the control unit 100 reading out a program stored in the storage unit 160. The processing shown in Figures 7 to 11 is also executed when a user performs an operation to start execution of a job that reads a document, such as a copy job or a scan job.

[1.2.1 Main Processing]
First, the flow of the main processing will be described with reference to FIG. 7 . The control unit 100 acquires an input image (e.g., a scanned image of a document) (step S100). The input image is an image input to the image forming apparatus 10. For example, the control unit 100 controls the image input unit 120 to read a document conveyed by the document feeder and acquire a scanned image of the document as the input image. The control unit 100 may acquire the input image from an external device via the communication unit 190, by reading an image stored in the storage unit 160, or from a storage medium such as a USB (Universal Serial Bus) memory.

Next, the control unit 100 (segment separation processing unit 104) performs segmentation processing on the input image (step S102). The segmentation processing unit 104 stores the results of the segmentation processing in the storage unit 160 as segmentation information 164.

Next, the control unit 100 executes a first scanner line detection process (step S104). The first scanner line detection process is a process that detects pixels from the input image that are candidates for abnormal pixels based on pixel changes in multiple pixels in the main scanning direction. For example, the control unit 100 provisionally determines, for each pixel in the input image, whether or not it is a candidate for an abnormal pixel that constitutes a scanner line, based on changes in pixel value (density or luminance) of multiple pixels in the main scanning direction, changes in hue, and changes in the region separation results. Details of the first scanner line detection process will be described later. The control unit 100 stores the results of the first scanner line detection process in the memory unit 160 as provisional scanner line determination result information 166.

Next, the control unit 100 executes a second scanner line detection process (step S106). The second scanner line detection process is a process that detects lines in the sub-scanning direction where scanner lines occur, based on whether pixels detected as candidate abnormal pixels have continuity in the sub-scanning direction by evaluating the provisional scanner line determination result information 166. Details of the second scanner line detection process will be described later. The control unit 100 stores the results of the second scanner line detection process in the memory unit 160 as scanner line determination result information 168. In this embodiment, the region formed by the pixel rows that make up the lines in the sub-scanning direction where scanner lines occur is called an abnormal pixel region. In other words, the second scanner line detection process is a process that detects abnormal pixel regions.

Next, the control unit 100 executes a scanner line correction process to correct scanner lines that have appeared in the input image based on the scanner line determination result information 168 (step S108). As a result, the control unit 100 corrects abnormal pixels contained in the abnormal pixel region based on the detection result of the abnormal pixel region. The scanner line correction process will be described later.

Next, the control unit 100 outputs the input image corrected in step S108 (step S110). For example, the control unit 100 controls the image forming unit 130 to form and output the corrected input image. Note that the control unit 100 may output the corrected input image data by storing it in the storage unit 160 or by transmitting it to another device.

1.2.2 First Scanner Line Detection Processing
The flow of the first scanner line detection process will be described with reference to Figures 8 and 9. The first scanner line detection process may be performed while the input image is being acquired, or after the input image is acquired. In this embodiment, as an example of the first scanner line detection process, a case will be described in which pixels that are candidates for abnormal pixels are detected based on changes in pixel values (density) of multiple pixels in the main scanning direction.

First, the control unit 100 selects one pixel from the pixels of the input image as a pixel of interest (step S120), and sets a range of multiple pixels in the main scanning direction to which the pixel of interest belongs (step S122). For example, the control unit 100 sets a scanner line detection mask (e.g., 7 pixels in the main scanning direction x 1 pixel in the sub-scanning direction) for the input image, and sets the range in which the scanner line detection mask is set as the range of multiple pixels in the main scanning direction to which the pixel of interest belongs.

The scanner line detection mask is used to set a range that includes a pixel of interest, several consecutive pixels (e.g., three pixels) in the forward direction from the pixel of interest in the main scanning direction, and several consecutive pixels (e.g., three pixels) in the reverse direction from the pixel of interest in the main scanning direction. By using this scanner line detection mask, the control unit 100 sets a range of seven consecutive pixels that includes multiple pixels in the main scanning direction to which the pixel of interest belongs.

Next, the control unit 100 determines whether each pixel included in the range set in step S122, including the pixel of interest, is a white pixel, and sets the flag (Flg) value for white pixels to "Wh" (step S124). Flg is information that indicates the characteristics of each pixel included in the range set in step S122.

For example, when pixel values are expressed as RGB values and the range of values for the R, G, and B components is 0 to 255, the control unit 100 determines that a pixel whose R, G, and B component values all exceed Th0 is a white pixel. Note that the control unit 100 may use the value stored in the parameter information storage area 162 for Th0. Also, when the color component value is any value between 0 and 255, for example, the value of Th0 is "230." Note that when a white pixel is found, the control unit 100 may associate the position of the pixel with the Flg value "Wh" and temporarily store this in the storage unit 160.

Next, the control unit 100 determines whether the Flg value of the pixel of interest is "Wh" (step S126). If the Flg value of the pixel of interest is "Wh", the control unit 100 determines that the pixel of interest is a white pixel, not a candidate for an abnormal pixel (step S126; Yes → step S128). At this time, the control unit 100 stores "Wh" as the value of the position of the pixel of interest in the provisional scanner line determination result information 166.

On the other hand, if the Flg value of the pixel of interest is not "Wh", the control unit 100 executes the process shown in Fig. 9. First, the control unit 100 assigns XMIN to a variable X indicating the position of a pixel included in the range set in step S122 of Fig. 8 (step S132). Here, the relationship between the value of the variable X and the pixel position is as follows:
(1) When X=0, it indicates the position of the pixel of interest.
(2) If X<0, it indicates a position that is away from the pixel of interest in the reverse direction of the main scanning direction by the number of pixels equal to the absolute value of the value of X. For example, if X=-3, it indicates a position that is three pixels away from the pixel of interest in the reverse direction of the main scanning direction.
(3) If X>0, it indicates a position that is the number of pixels away from the pixel of interest in the forward direction of the main scanning direction, which is the value of X. For example, if X=3, it indicates a position that is three pixels away from the pixel of interest in the forward direction of the main scanning direction.

XMIN indicates the smallest possible value for X. For example, in step S122 of FIG. 8, if a range is set that includes three consecutive pixels in the forward direction from the target pixel in the main scanning direction and three consecutive pixels in the reverse direction from the target pixel in the main scanning direction, the possible range for X is -3 to 3, and XMIN is -3.

Next, the control unit 100 determines whether the value of variable X is 0 (step S134). If the value of variable X is not 0, the control unit 100 determines whether the pixel at the position indicated by the value of variable X (pixel at position X) is a white pixel (step S134; Yes → step S136). For example, if the Flg value for the pixel at position X is "Wh", the control unit 100 determines that the pixel is a white pixel.

If the pixel at position X is not a white pixel, the control unit 100 obtains the density difference for each color component between the pixel at position X and the pixel of interest (step S136; Yes → step S138). The density difference refers to the difference in the value of a specific color component between the two pixels.

For example, when the pixel value of the pixel at position X is ( _Rx , _Gx , _Bx ), the control unit 100 calculates the density difference for each color component using the following formula, thereby obtaining the density difference for each color component.
Concentration difference of R component=|R ₀ −R _X |
Concentration difference of B component=|G ₀ −G _X |
Density difference of G component=|B ₀ −B _X |
Here, when X=−3, the control unit 100 calculates the density difference for each color component using the following formula.
Concentration difference of R component=|R ₀ −R ₋₃ |
Concentration difference of B component=|G ₀ −G ₋₃ |
Density difference of G component=|B ₀ −B ₋₃ |

Next, the control unit 100 determines whether, of the density differences for each color component obtained in step S138, the density difference for one color component exceeds threshold value Th1 and the density differences for the other color components are less than threshold value Th2 (step S140).

For example, the control unit 100 determines whether any of the following conditions is met.
(1) |R ₀ −R _X |>Th1 and |G ₀ −G _X |<Th2 and |B ₀ −B _X |<Th2
(2) |R ₀ −R _X |<Th2 and |G ₀ −G _X |>Th1 and |B ₀ −B _X |<Th2
(3) |R ₀ −R _X |<Th2 and |G ₀ −G _X |<Th2 and |B ₀ −B _X |>Th1
Here, if X=−3, the control unit 100 determines whether or not any of the following conditions is met.
(1) |R ₀ -R _-3 |>Th1 and |G ₀ -G _-3 |<Th2 and |B ₀ -B _-3 |<Th2
(2) |R ₀ -R _-3 |<Th2 and |G ₀ -G _-3 |>Th1 and |B ₀ -B _-3 |<Th2
(3) |R ₀ −R ₋₃ |<Th2 and |G ₀ −G ₋₃ |<Th2 and |B ₀ −B ₋₃ |>Th1

Note that Th1 and Th2 may use the values stored in the parameter information storage area 162. Furthermore, if the color component values are any value between 0 and 255, for example, the value of Th1 is "80" (preferably any value between 50 and 150), and the value of Th2 is "15" (preferably any value between 5 and 20).

If the density difference of one color component exceeds threshold value Th1 and the density difference of the other color components is less than Th2, the control unit 100 sets the Flg value of the pixel at position X to "1" (step S140; Yes → step S142). The control unit 100 also sets information about the color component that exceeds threshold value Th1 for the pixel at position X, along with the Flg value. In this embodiment, a color component that exceeds threshold value Th1 is referred to as a specific color component. In this embodiment, the control unit 100 performs the process of step S140 to determine whether any one of the R, G, and B color components is a specific color component, and if any one color component is a specific color component, sets the Flg value to "1".

For example, when the specific color component is the R component, the control unit 100 associates the value of the variable X with the Flg value "1R" and temporarily stores them in the storage unit 160. Similarly, when the specific color component is the G component, the control unit 100 associates the value of the variable X with the Flg value "1G" and temporarily stores them in the storage unit 160. When the specific color component is the B component, the control unit 100 associates the value of the variable X with the Flg value "1B" and temporarily stores them in the storage unit 160.

On the other hand, unless the density difference of one color component exceeds threshold Th1 and the density difference of the other color components is less than Th2, the control unit 100 sets the Flg value of the pixel at position X to "0" (step S140; No → step S144). For example, the control unit 100 associates the value of variable X with the Flg value "0" and temporarily stores them in the storage unit 160.

Note that if the value of X is 0 (step S134; No) or if the pixel at position X is a white pixel (step S136; No), the control unit 100 skips steps S138 to S144.

Next, the control unit 100 determines whether the value of variable X is XMAX (step S146). XMAX indicates the maximum value that X can take. If the value of variable X is not XMAX, the control unit 100 assigns the value of X+1 to X and returns to step S134 (step S146; No → step S148 → step S134). In this way, the control unit 100 sets Flg values for pixels other than the pixel of interest that are included in the range of pixels set in step S122 of Figure 8.

On the other hand, if the value of variable X is XMAX, the control unit 100 determines whether there are one or more pairs of pixels on either side of the pixel of interest that have a Flg value of "1" and the same specific color element (step S146; Yes → step S150). "On either side of the pixel of interest" refers to pixels adjacent to the pixel of interest in the forward main scanning direction and pixels adjacent to the pixel of interest in the reverse main scanning direction. The control unit 100 determines whether there are any pixels with a Flg value of "1" and the same specific color element at any pixel adjacent to the pixel of interest in the forward main scanning direction and any pixel adjacent to the pixel of interest in the reverse main scanning direction. Then, if there are any pixels with a Flg value of "1" and the same specific color element, the control unit 100 determines that there are one or more pairs of pixels on either side of the pixel of interest that have a Flg value of "1" and the same specific color element.

For example, if there are pixels on both sides of the pixel of interest whose Flg value is "1R," the control unit 100 determines that there is a pair of pixels with Flg = 1 and the same specific color component.

If there is one or more pairs of pixels on either side of the pixel of interest that have a Flg value of "1" and the same specific color element, the control unit 100 determines that the pixel of interest is a candidate for an abnormal pixel as a provisional scanner line determination result for the pixel of interest (step S150; Yes → step S152). Here, the control unit 100 stores information on the specific color element of the pixels on either side of the pixel of interest that have a Flg value of "1" and the same specific color element in the provisional scanner line determination result information 166 for the pixel of interest.

For example, if there is one or more pairs of pixels with a Flg value of "1R" on either side of the pixel of interest, the control unit 100 stores "1R" as the position value of the pixel of interest in the provisional scanner line determination result information 166. This allows the control unit 100 to identify, from the "1R" information stored in the provisional scanner line determination result information 166, that the pixel in question is a candidate for an abnormal pixel, and that the density difference of the red component between the pixel in question and its surrounding pixels exceeds Th1. The information on the specific color element stored in the provisional scanner line determination result information 166 is the specific color component corresponding to the pixel detected as a candidate for an abnormal pixel.

On the other hand, if Flg = 1 and there are no pixels with the same specific color component on either side of the pixel of interest, the control unit 100 determines that the pixel of interest is not a candidate for an abnormal pixel, but is a non-white pixel, as a provisional scanner line determination result for the pixel of interest (step S150; No → step S154). In this case, the control unit 100 stores "X" as the value of the position of the pixel of interest in the provisional scanner line determination result information 166.

Returning to FIG. 8, after the control unit 100 executes the process of either step S128 in FIG. 8, step S152 in FIG. 9, or step S154 in FIG. 9, it determines whether or not all pixels in the input image have been determined to be candidates for abnormal pixels (step S130). If the control unit 100 has determined all pixels, it ends the first scanner line detection process (step S130; Yes). On the other hand, if the control unit 100 has not determined all pixels, it returns to step S120 (step S130; No → step S120). In this way, the control unit 100 detects candidate abnormal pixels from the input image.

The control unit 100 may also execute the first scanner line detection process while reading an image of a document. For example, once the control unit 100 has read one line or a predetermined number of lines of the image of the document in the main scanning direction, it may execute the first scanner line detection process on the image of the read line. In this case, in step S120 of FIG. 8, the control unit 100 selects a pixel of interest from the pixels included in the image of the read line, and in step S130 of FIG. 8, it determines whether or not it has performed a determination on all pixels included in the image of the read line. In this way, the control unit 100 can execute the first scanner line detection process in real time.

1.2.3 Second scanner streak detection process
The flow of the second scanner line detection process will be described with reference to Fig. 10. The second scanner line detection process may be executed after the first scanner line detection process, or may be executed in parallel with the first scanner line detection process when acquiring the input image.

First, the control unit 100 selects one line in the sub-scanning direction from the input image (step S160). For example, the control unit 100 sets a variable indicating the line in the sub-scanning direction to L, and assigns a predetermined value to L (e.g., 0, 1, 2, ... where L is in the range 0≦L≦number of pixels in the main scanning direction of the input image - 1). In the following explanation, the line in the sub-scanning direction selected in step S160 will be referred to as the "line of interest." In other words, the line of interest is a row of pixels with the sub-scanning direction as its longitudinal direction, and is composed of pixels whose x-coordinate value is L.

Next, the control unit 100 evaluates the provisional scanner line determination result for the line of interest. First, the control unit 100 determines whether there are any pixels on the line of interest that are candidates for abnormal pixels (step S162). For example, the control unit 100 reads the determination result for the Lth column from the provisional scanner line determination result information 166, and if any of "1R," "1G," or "1B" is included, it determines that there is a pixel that is a candidate for abnormal pixels.

If there is a candidate abnormal pixel, the control unit 100 determines whether there is a single color component whose density difference exceeds Th1 between the candidate abnormal pixel and its surrounding pixels (Step S162: Yes → Step S164). That is, the control unit 100 determines whether the specific color components corresponding to the pixels detected as candidate abnormal pixels included in the target line are the same. For example, if the values of the provisional scanner line determination result for the pixels included in the target line include two or more of "1R," "1G," and "1B," the control unit 100 determines that there is more than one color component whose density difference exceeds Th1. That is, the control unit 100 determines that although there is a candidate abnormal pixel on the target line, the color components whose density difference exceeds Th1 are not the same color component.

If there is more than one color component with a density difference exceeding Th1, the control unit 100 changes the provisional scanner line determination result for the candidate abnormal pixel included in the line of interest to a non-white pixel that is not a candidate abnormal pixel (step S164; No → step S166). That is, the control unit 100 changes the provisional scanner line determination result for the candidate abnormal pixel in the line of interest to "X." As a result, the control unit 100 excludes pixels on lines with more than one color component with a density difference exceeding Th1 (pixels detected as candidate abnormal pixels do not have the same specific color component) from the candidate abnormal pixels. Note that if the control unit 100 determines in step S164 that there is one color component with a density difference exceeding Th1, it skips the processing in step S166 (step S164; Yes).

Next, the control unit 100 determines whether there are any isolated candidate abnormal pixels on the line of interest (step S168). Here, an isolated candidate abnormal pixel refers to a candidate abnormal pixel that is not adjacent to other candidate abnormal pixels, or a candidate abnormal pixel where the number of consecutive candidate abnormal pixels is less than a predetermined threshold (e.g., 5).

If there is a pixel that is a candidate for an isolated abnormal pixel, the control unit 100 changes the provisional scanner line determination result for that pixel to a non-white pixel that is not a candidate for an abnormal pixel (step S168; Yes → step S170). That is, the control unit 100 changes the provisional scanner line determination result for the pixel that is a candidate for an isolated abnormal pixel within the line of interest to "X". This causes the control unit 100 to exclude the pixel that is a candidate for an isolated abnormal pixel from the list of candidate abnormal pixels. Note that if there is no pixel that is a candidate for an isolated abnormal pixel in step S168, the control unit 100 omits the processing in step S170 (step S168; No).

Next, the control unit 100 determines whether the candidate abnormal pixels on the line of interest are continuous in the sub-scanning direction (step S172). Consecutive abnormal pixels in the sub-scanning direction are present when the candidate abnormal pixels are intermittently located in the sub-scanning direction on the line of interest. Consecutive abnormal pixel pixels in the sub-scanning direction are present when the candidate abnormal pixels are arranged with white pixels sandwiched between them. In other words, the candidate abnormal pixels are continuous in the sub-scanning direction when the candidate abnormal pixels are alternately arranged with white pixels sandwiched between them, or when the candidate abnormal pixels are consecutive with white pixels sandwiched between them.

In step S172, the control unit 100 determines whether the number of consecutive white pixels and candidate abnormal pixels on the target line is equal to or exceeds the threshold value Th3. The value of Th3 may be a pixel count equivalent to 10% of the vertical pixel count of the input image (preferably, a pixel count equivalent to any ratio between 5% and 20% of the vertical pixel count of the input image), or a specific value (e.g., "200"). In this case, the control unit 100 reads the determination result for the target line from the provisional scanner line determination result information 166 and determines whether the number of consecutive pixels determined to be "Wh" and the number of consecutive pixels determined to be candidate abnormal pixels (either "1R," "1G," or "1B") exceeds the threshold value Th3. If the number of consecutive pixels determined to be "Wh" and the number of consecutive candidate abnormal pixels exceeds the threshold value Th3, the control unit 100 determines that the candidate abnormal pixels on the target line are continuous in the sub-scanning direction.

If the candidate abnormal pixels on the line of interest are continuous in the sub-scanning direction, the control unit 100 determines that the line of interest is a line where a scanner streak has occurred (step S172; Yes → step S174). This allows the control unit 100 to identify candidate abnormal pixels included in the line where a scanner streak has occurred as abnormal pixels. The control unit 100 can also identify a line (pixel row) in the sub-scanning direction that contains abnormal pixels as an abnormal pixel area. At this time, the control unit 100 stores the value of X, which indicates the row number of the line of interest, in association with "Yes" in the scanner streak determination result information 168.

On the other hand, if the candidate abnormal pixels on the line of interest are not continuous in the sub-scanning direction, the control unit 100 determines that the line of interest is a line on which scanner streaks do not occur (step S172; No → step S176). At this time, the control unit 100 stores the value of X, which indicates the number of columns on the line of interest, in association with "No" in the scanner streak determination result information 168. Note that even if the control unit 100 determines in step S162 that the line of interest does not have any candidate abnormal pixels, it also determines that the line of interest is a line on which scanner streaks do not occur (step S162; No → step S176).

Next, the control unit 100 determines whether all lines of the input image have been selected (step S178). If all lines of the input image have been selected, the control unit 100 ends the processing shown in FIG. 10 (step S178; Yes). On the other hand, if all lines of the input image have not been selected, the control unit 100 returns to step S160 (step S178; No → step S160).

When the control unit 100 executes the second scanner streak detection process in parallel with the first scanner streak detection process, it only needs to execute the second scanner streak detection process each time it reads a predetermined number (e.g., 200) of lines of the document image in the main scanning direction. In this way, the control unit 100 can determine in real time the lines on which scanner streaks have occurred.

[1.2.4 Scanner streak correction processing]
The flow of the scanner streak correction process will be described with reference to Fig. 11. First, the control unit 100 determines whether or not scanner streaks have appeared in the input image (step S180). For example, the control unit 100 reads the scanner streak determination result information 168, and if the scanner streak determination result includes one or more "Yes"s, it determines that scanner streaks have appeared in the input image. Note that if there are no scanner streaks in the input image, the control unit 100 ends the process shown in Fig. 11 (step S180; No).

If scanner streaks are present in the input image, the control unit 100 selects one line in the sub-scanning direction where the scanner streaks are present (step S180; Yes → step S182). For example, the control unit 100 obtains the position in the scanner streak determination result information 168 where the scanner streak determination result is "Yes," and selects the line in the sub-scanning direction at that position.

Next, the control unit 100 identifies the image pattern (region separation result) around the scanner line (step S184). For example, while referencing the region separation information 164, the control unit 100 identifies, for each pixel in the line selected in step S182, the image pattern of the pixel adjacent to either the left (opposite to the main scanning direction) or the right (forward to the main scanning direction) of the pixel as the image pattern around the scanner line at that pixel. Note that, for each pixel in the line selected in step S182, the control unit 100 may also identify the image pattern around the scanner line at that pixel based on the image pattern of a region of a predetermined size adjacent to the left or right of the pixel. For example, the control unit 100 sets two regions of 25 pixels, five vertically and five horizontally, to the left and right of a pixel (pixel of interest) in the line selected in step S182, so that they are adjacent to the pixel of interest. In this case, the control unit 100 counts the image patterns of the pixels within the set area (50 pixels in total) and identifies the most common image pattern as the image pattern surrounding the pixel of interest. Note that the control unit 100 may identify an image pattern surrounding the scanner line for each pixel within the line selected in step S182, or may identify an image pattern surrounding one scanner line for the selected line.

Next, the control unit 100 determines whether the surrounding image pattern is a halftone dot area or a line area (step S186). If the surrounding image pattern is a halftone dot area or a line area, the control unit 100 replaces the pixel value of the scanner line with the average value of the pixels surrounding that pixel (peripheral pixels) (step S186; Yes → step S188). For example, the control unit 100 sets 25-pixel areas (5 pixels vertically and 5 pixels horizontally) to the left and right of the pixel to be corrected (target pixel) so that they are adjacent to the target pixel, and the pixels included in these areas are designated as peripheral pixels. At this time, the control unit 100 obtains pixel values for each pixel from the peripheral pixels and sets the average of the obtained pixel values as the average value of the peripheral pixels.

On the other hand, if the surrounding image pattern is not a halftone dot area or a line area, the control unit 100 determines whether the surrounding image pattern is a character area (step S186; No → step S190). If the surrounding image pattern is a character area, the control unit 100 replaces the pixel value of the scanner line with the pixel value of the pixel adjacent to that pixel (adjacent pixel value) (step S190; Yes → step S192). For example, the control unit 100 sets the pixel value of the pixel adjacent to either the left or right of the pixel to be corrected as the adjacent pixel value.

If the surrounding image pattern is not a text area in step S190, the control unit 100 replaces the pixel values of the scanner line using a predetermined method (step S190; No → step S194). Areas where the surrounding image pattern is neither a halftone dot area nor a line area, and is not a text area (so-called "other" areas), are, for example, photographic image areas. If the surrounding image pattern is "other," the control unit 100 may replace the pixel values of the scanner line based on the average pixel values of the surrounding pixels, as in the case where the surrounding area is a halftone dot area or line area, or may replace them based on the pixel values of adjacent pixels, as in the case where the surrounding area is a text area. The method for replacing pixel values in step S194 may be predetermined or may be user-configurable. The control unit 100 may also replace the pixel values of the scanner line using a method other than replacing pixel values based on the average value or the pixel values of adjacent pixels.

The control unit 100 performs the processes from step S186 to step S194 for each pixel in the line selected in step S182. This allows the control unit 100 to switch the correction method (replacement method) for abnormal pixels included in the abnormal pixel region based on the region separation information surrounding the abnormal pixel region. The control unit 100 can also appropriately correct the abnormal pixel by replacing the pixel value of the abnormal pixel with a different pixel value depending on the correction method for the abnormal pixel. The control unit 100 may replace the pixel values of all pixels included in the line selected in step S182, or may replace only the pixel values of abnormal pixels included in the line (pixels for which the provisional scanner line determination result is "1R", "1G", or "1B").

Next, the control unit 100 determines whether all lines with scanner streaks have been selected (step S196). If all lines with scanner streaks have been selected, the control unit 100 ends the processing shown in FIG. 11 (step S196; Yes). On the other hand, if all lines with scanner streaks have not been selected, the control unit 100 returns to step S182 (step S196; No → step S182).

[1.3 Operation example]
Next, an example of the operation of this embodiment will be described. First, a scanner streak corrected by the image forming apparatus 10 of this embodiment will be described. Fig. 12 is a diagram showing a case where a white streak (a streak caused by loss of one component color) occurs.

P1 in Figure 12 indicates the document feed direction. Document A100 is transported in the direction of P1 by the document feeder. Document A100 reflects light P2 from light source A102. The reflected R component light R100, G component light G100, and B component light B100 are read by image sensor A104, such as a CCD. This allows the image forming device 10 to obtain an image of document A100.

Now, let's say there is a foreign object A108, such as a piece of paper or dust, between the original A100 and the glass A106 located below it. If the foreign object A108 is a small, white object, light P2 from the light source A102 is not absorbed by the original A100 and is reflected by the foreign object A108. If the foreign object A108 is small, light of one color (for example, light R100 of the R component) will always be reflected, and this light of one color will always be read by the image sensor A104. In this way, light of a specific color component will always be read by the image sensor A104, resulting in pixels in the input image that appear to be missing that specific color component.

Next, an example of the operation of the first scanner streak detection process will be described with reference to Figures 13 to 18. Figure 13(a) is a diagram showing a scanned image (input image) of a document, with the squares in the diagram representing pixels. Area E100 represents a halftone dot area. Furthermore, scanner streaks E102 represent scanner streaks. In this way, scanner streaks can appear, for example, on halftone dot areas.

Also, area E104 indicates the scanner line detection mask. Also, area E106 indicates the pixel of interest. To determine whether the pixel of interest is a scanner line, the image forming device 10 performs processing to obtain the density difference for each color component using the scanner line detection mask.

Figure 13(b) is an enlarged view of the scanner streak detection mask. Here, the pixel where X = 0 (pixel marked with *) is the pixel of interest. The pixels located from X = -3 to X = -1 are consecutive pixels in the reverse direction from the pixel of interest in the main scanning direction. The pixels located from X = 1 to X = 3 are consecutive pixels in the forward direction from the pixel of interest in the main scanning direction. In this way, the image forming device 10 sets the range of multiple pixels in the main scanning direction to which the pixel of interest belongs.

Figure 14 shows the results of a provisional scanner streak determination for a halftone dot area when no scanner streaks have occurred. Figure 14(a) is an enlarged view of a halftone dot area in the input image, with the squares in the figure representing pixels. Area E110 represents the scanner streak detection mask. Area E112 represents the pixel of interest. As shown in P110, rectangles with a white background represent white pixels. As shown in P112, rectangles with a non-white background represent non-white pixels (halftone dot pixels with some color).

Figure 14(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 14(b), there are no pixels with Flg=1 on either side of the pixel of interest. Therefore, the provisional scanner judgment result for the pixel of interest is "X".

All pixels in the input image are judged to determine whether they are candidate abnormal pixels, resulting in the judgment results shown in Figure 14(c). Note that for lines where scanner streaks do not occur, provisional scanner streak judgment results such as "Wh" or "X" continue in the sub-scanning direction, as shown in area E114.

Figure 15 shows the results of a provisional scanner streak determination for a halftone dot area when scanner streaks have occurred. Figure 15(a) is an enlarged view of a halftone dot area in the input image, with the squares in the figure representing pixels. Area E120 represents the scanner streak detection mask. Area E122 represents the pixel of interest. Note that cross-hatched pixels, such as P120 in Figure 15(a), represent halftone dot pixels with a color that is not approximately the same as the color of the surrounding halftone dots. In other words, within the halftone dot area of the input image, as shown in P120, halftone dot pixels with a color that is not approximately the same as the color of the surrounding halftone dots appear consecutively in the sub-scanning direction, and these pixels cause scanner streaks.

Figure 15(b) shows the Flg value for each pixel in the scanner line detection mask. Here, as shown in areas E124 and E126 in Figure 15(b), there are pixels where Flg = 1 and the specific color component is red (R) (pixels with an Flg value of "1R") in the pixels consecutive from the pixel of interest in the forward main scanning direction and the pixels consecutive from the pixel of interest in the reverse main scanning direction. Therefore, the provisional scanner line determination result for the pixel of interest is "1R".

The determination results shown in Figure 15(c) are obtained by determining whether all pixels in the input image are candidates for abnormal pixels. Note that, in the lines where scanner streaks occur, provisional scanner streak determination results continue in the sub-scanning direction for pixels that are candidates for abnormal pixels and have the same color components, such as "Wh" and "1R," with density differences exceeding Th1, as shown in area E128.

Figure 16 shows the results of a provisional scanner streak determination for a character region when no scanner streaks have occurred. Figure 16(a) is an enlarged view of a character region in the input image, with the squares in the figure representing pixels. Area E130 represents the scanner streak detection mask. Area E132 represents the pixel of interest. Also, as shown in P130, rectangles with a white background represent white pixels, and as shown in P132, rectangles with a black background represent black pixels.

Figure 16(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 16(b), there are no pixels with Flg=1 on either side of the pixel of interest. Therefore, the provisional scanner judgment result for the pixel of interest is "X".

All pixels in the input image are judged to determine whether they are candidate abnormal pixels, resulting in the judgment results shown in Figure 16(c). As shown in Figure 16(c), if no scanner streaks have occurred, the provisional scanner streaks judgment results such as "Wh" or "X" continue in the sub-scanning direction on every line.

Figure 17 shows the results of a provisional scanner line determination for a character region when scanner lines have occurred. Figure 17(a) is an enlarged view of the character region in the input image, with the squares in the figure representing pixels. Area E140 represents the scanner line detection mask. Area E142 represents the pixel of interest. Note that cross-hatched pixels, such as P140 in Figure 17(a), are pixels of a color other than black or white (e.g., red). In other words, within the character region of the input image, as shown in P140, pixels of a color different from the surrounding pixels appear consecutively in the sub-scanning direction, and these pixels cause scanner lines.

Figure 17(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 17(b), among the pixels consecutive from the pixel of interest in the forward main scanning direction and the pixels consecutive from the pixel of interest in the reverse main scanning direction, there are pixels where Flg = 1 and the specific color component is red (R) (pixels with an Flg value of "1R"). Therefore, the provisional scanner line determination result for the pixel of interest is "1R".

The determination results shown in Figure 17(c) are obtained by determining whether all pixels in the input image are candidates for abnormal pixels. Note that, in the lines where scanner streaks occur, provisional scanner streak determination results continue in the sub-scanning direction for pixels that are candidates for abnormal pixels and have the same color components, such as "Wh" and "1R," with density differences exceeding Th1, as shown in area E144.

Figure 18 shows the case where ruled lines of a table or the like extend in the sub-scanning direction, with the squares in the figure representing pixels. Figure 18(a) is an enlarged view of a portion of the input image where ruled line L150 extends in the sub-scanning direction. Area E150 in Figure 18(a) represents the scanner line detection mask. Area E152 represents the pixel of interest. Here, the pixel of interest is assumed to be a pixel that makes up the ruled line.

Figure 18(b) shows the Flg value for each pixel in the scanner line detection mask. As shown in Figure 18(b), the pixel of interest is not a white pixel, so the Flg value for the pixel of interest is not Wh. Furthermore, there are no pixels on either side of the pixel of interest where Flg = 1. Therefore, the provisional scanner line determination result for the pixel of interest is "X", and it is determined not to be a candidate for an abnormal pixel. As a result, ruled lines (e.g., colored lines) included in the input image are not determined to be lines where the color has been removed by scanner lines (so-called missing lines).

Next, an example of the operation of the scanner streak correction process will be described with reference to Figures 19 and 20. Figure 19 shows an example of the operation of correcting scanner streaks on halftone dots, where the line shown in area E128 in Figure 15(c) is corrected. Here, it is assumed that area E128 in Figure 15(c) is determined to be a line where scanner streaks occur.

Figure 19(a) shows the result of performing scanner line correction for text on an abnormal pixel (a pixel with a provisional scanner line determination result of "1R") included in a line where a scanner line has occurred. Scanner line correction for text is a correction that replaces the pixel value of the scanner line with the pixel value of the pixel adjacent to that pixel. Here, area E160 in Figure 19(a) indicates the area of the line where a scanner line has occurred (the abnormal pixel area). When scanner line correction for text is performed on an abnormal pixel on a halftone dot, the color of the corrected abnormal pixel matches the color of the surrounding pixels, but a visual gap occurs due to a shift in the periodicity of the halftone dots.

Figure 19(b) shows the result of performing scanner streak correction for halftone dots on an abnormal pixel included in a line where a scanner streak has occurred. Scanner streak correction for halftone dots is a correction that replaces the pixel value of the scanner streak with the average value of the pixels surrounding that pixel. Area E162 in Figure 19(b) indicates the area of the line where the scanner streak has occurred (the abnormal pixel area). When scanner streak correction for halftone dots is performed on an abnormal pixel on a halftone dot, the color of the corrected abnormal pixel matches the color of the surrounding colored pixels, reducing the appearance of a gap in color.

Figure 20 shows an example of the operation of correcting scanner streaks on characters, showing the case where the line shown in area E144 in Figure 17(c) is corrected. Here, it is assumed that area E144 in Figure 17(c) is determined to be a line where scanner streaks occur.

Figure 20(a) shows the result of performing scanner line correction for text on abnormal pixels included in a line where scanner lines have occurred. Area E170 in Figure 20(a) indicates the area of the line where scanner lines have occurred (abnormal pixel area). When scanner line correction for text is performed on abnormal pixels on text, the pixel value of the abnormal pixel becomes the same as the pixel value of the surrounding pixels (black or white), and sharpness is maintained.

Figure 20(b) shows the result of performing scanner streak correction for halftone dots on abnormal pixels included in a line where scanner streaks have occurred. Area E172 in Figure 20(b) indicates the area of the line where scanner streaks have occurred (abnormal pixel area). When scanner streak correction for halftone dots is performed on abnormal pixels on text, the pixel value of the abnormal pixel is replaced with a color that takes into account the pixel values (density) of the surrounding pixels, resulting in an insufficient correction.

In this way, the pixel values of the scanner lines are replaced according to the results of separating the area around the scanner lines. This allows, for example, to correct scanner lines in character areas by replacing them with the pixel values of adjacent pixels, thereby achieving correction that maintains sharpness. Also, to correct scanner lines in line or halftone dot areas, they are replaced with the average value of the pixels around the lines, thereby minimizing gaps with the surrounding area. In this way, scanner lines are properly corrected.

In the above explanation, it is assumed that the number of scanned images (input images) of a document input to the image forming device 10 is one. If multiple documents are input to the image forming device 10, the image forming device 10 performs the above-mentioned processing on all scanned images (input images) of the documents. As a result, the image forming device 10 stores region separation information 164, provisional scanner streak determination result information 166, and scanner streak determination result information 168 for the input image, and can correct scanner streaks based on this information.

In the above explanation, the input image is divided into line areas, halftone dot areas, character areas, and background areas, and the method of correcting the scanner streaks is switched based on whether the type of area surrounding the scanner streaks is a line/halftone dot area or a character area. However, even if the type of area surrounding the scanner streaks is an area other than the types of areas described above, the image forming device 10 may switch the method of correcting the scanner streaks depending on the type of area.

In the above explanation, it was explained that in step S184 of FIG. 11, the control unit 100 identifies the image pattern around the scanner line by referencing the region separation information 164. However, the control unit 100 may identify the image pattern around the scanner line without using the region separation information 164. For example, the control unit 100 may identify the pixel pattern by pattern matching. Furthermore, the control unit 100 may identify the pixel pattern based on the periodicity of the brightness of pixels within a pixel block around the scanner line (for example, a region of 20 pixels vertically and horizontally). Note that when correcting the scanner line without referencing the region separation information 164, the control unit 100 may omit the region separation process in step S102 of FIG. 7.

As such, the image forming apparatus of this embodiment makes it possible to detect scanner lines in the scanned image in real time while reading the document image. Furthermore, the image forming apparatus of this embodiment corrects scanner lines according to the image pattern (region separation result) around the scanner lines. Conventionally, scanner lines have been corrected by uniformly replacing the pixel values of the scanner lines with the pixel values of the pixels around the scanner lines. This has resulted in gaps due to shifts in the periodicity of the halftone dots, or in insufficient correction. However, the image forming apparatus of this embodiment can solve these problems by performing correction according to the image pattern of the scanner lines.

[2. Second embodiment]
Next, a second embodiment will be described. In the second embodiment, in addition to the processing in the first embodiment, processing for correcting the region separation result is executed. In this embodiment, FIG. 11 of the first embodiment is replaced with FIG. 21. Note that the same functional units and processing are assigned the same reference numerals, and descriptions thereof will be omitted.

The flow of scanner streak correction processing in this embodiment will be described with reference to Figure 21. If the image pattern surrounding the scanner streak is a halftone dot area or a line area (step S186; Yes), the control unit 100 corrects the area separation results corresponding to the pixels whose pixel values have been corrected to a halftone dot area or a line area after processing step S188 (step S200). For example, the control unit 100 changes the values stored in the area separation information 164 that correspond to the pixels whose pixel values have been corrected to values that correspond to the image pattern identified in step S184.

On the other hand, if the image pattern around the scanner line is a text area (step S190; Yes), after processing step S192, the control unit 100 corrects the region separation results corresponding to the pixels whose pixel values have been corrected to a text area (step S202). For example, the control unit 100 changes the value of the region separation information 164 corresponding to the pixels whose pixel values have been corrected to a value corresponding to a text area.

Furthermore, if the image pattern around the scanner line is neither a halftone dot area nor a line area, nor a character area, the control unit 100 performs the process of step S194 (step S190; No → step S194). Next, the control unit 100 corrects the area separation information for the pixel by changing the area separation result corresponding to the pixel whose pixel value has been corrected to a value corresponding to the image pattern identified in step S184 (step S204).

Here, the region separation process for the input image is performed before the scanner line correction process. Therefore, it is possible that the results of the region separation process for the scanner line pixels may not be appropriate. In response to this, the control unit 100 can correct the region separation information 164 by performing the above-mentioned process.

By correcting the region separation information 164, the control unit 100 can more appropriately perform image processing when using the region separation information 164. Examples of image processing include color correction, black generation UCR (Under Color Removal), filter processing, and screen processing. In addition, processing for halftone dot regions is performed on halftone dot regions, and processing to emphasize halftone dot text is performed on halftone dot text regions. By performing image processing using the corrected region separation information 164, the control unit 100 can reduce the effects of scanner streaks and obtain appropriate image processing results.

Next, an example of the operation of this embodiment will be described. Figure 22(a) is an enlarged view of a halftone dot scanned image. It is assumed that there are no scanner streaks in the scanned image. Figure 22(b) is a view showing region separation information for Figure 22(a). In Figure 22(b), regions shown with halftone dots indicate that the region is a halftone dot region. As shown in Figure 22(b), the region separation information indicates that the entire region is a halftone dot region.

Figure 23(a) is an enlarged view of a halftone dot scanned image. Assume that scanner streaks have appeared in the scanned image. In the input image, scanner streaks have appeared, as shown in area E200. Figure 23(b) is a view showing region separation information for Figure 23(a). In Figure 23(b), regions shown with halftone dots indicate that they are halftone dot regions, and regions shown with cross-hatching indicate that they are halftone color text regions. As shown in Figure 23(b), the region separation information includes area E206, which is a "halftone color text region" caused by scanner streaks, in addition to halftone dot region E202 and halftone dot region E204. In other words, there is a halftone color text region caused by scanner streaks on top of a halftone dot region.

Here, the scanner streak correction process corrects the pixel values of the scanner streak pixels, as shown in area E208 in Figure 23(c). At the same time, the area separation results in the sub-scanning direction where the scanner streak occurred are also corrected. Figure 23(d) shows the area separation information after the area separation results have been corrected. As shown in Figure 23(d), the area separation information indicates a halftone dot area as a whole.

Figure 24(a) is an enlarged view of a scanned image of text. Note that the scanned image does not have scanner streaks. Figure 24(b) is a view showing region separation information for Figure 24(a). In Figure 24(b), regions shown in black indicate that the region is a text region, and regions shown in white indicate that the region is a background region. As shown in Figure 24(b), the region separation information indicates that regions E220, E222, E224, and E226 are black text regions.

Figure 25(a) is an enlarged view of a scanned image of text. Assume that scanner streaks have appeared in the scanned image. In the input image, scanner streaks have appeared, as shown in area E230. Figure 25(b) is a view showing region separation information for Figure 25(a). In Figure 25(b), areas shown in black indicate that the area is a text area, areas shown in white indicate that it is a background area, and areas shown in halftone dots indicate that it is a color text area. As shown in Figure 25(b), the region separation information includes, for example, area E236, which is a color text area, in addition to text area E232 and text area E234. In other words, there is a color text area on top of a black text area due to the influence of scanner streaks.

Here, the scanner streak correction process corrects the pixel values of the pixels in the scanner streak, as shown in area E240 in Figure 25(c). At the same time, the area separation results in the sub-scanning direction where the scanner streak occurred are also corrected. Figure 25(d) is a diagram showing the area separation information after the area separation results have been corrected. The area separation information is corrected as shown in Figure 25(d). For example, area E236, which was a "color character area" in Figure 25(b), becomes a "black character area," as shown in area E242 in Figure 25(d). As a result, the area determination results for all pixels included in area E244 become "black character area."

In this way, the image forming apparatus of this embodiment corrects scanner streaks and also corrects the region separation results. This allows the image forming apparatus of this embodiment to appropriately perform image processing using the region separation results.

3. Third embodiment
Next, a third embodiment will be described. In addition to the processing in the first embodiment, the third embodiment calculates the amount of change in density between a pixel of interest and pixels other than the pixel of interest, and executes processing to detect pixels where the amount of change exceeds a predetermined threshold value for two components.

In this embodiment, if the determination in step S140 of the first scanner detection process shown in Figure 9 is No, the control unit 100 determines whether the density difference between the pixel value of the pixel at position X and the pixel value of the target pixel satisfies the condition that the density difference of two of the color components exceeds threshold value Th1 and the density difference of the remaining color component is less than threshold value Th2. For example, the control unit 100 determines whether any of the following conditions is satisfied:
(1) |R ₀ −R _X |>Th1 and |G ₀ −G _X |>Th1 and |B ₀ −B _X |<Th2
(2) |R ₀ −R _X |>Th1 and |G ₀ −G _X |<Th2 and |B ₀ −B _X |>Th1
(3) |R ₀ −R _X |<Th2 and |G ₀ −G _X |>Th1 and |B ₀ −B _X |>Th1

For example, when X=−3, the control unit 100 determines whether any of the following formulas is satisfied.
(1) |R ₀ -R _-3 |>Th1 and |G ₀ -G _-3 |>Th1 and |B ₀ -B _-3 |<Th2
(2) |R ₀ -R _-3 |>Th1 and |G ₀ -G _-3 |<Th2 and |B ₀ -B _-3 |>Th1
(3) |R ₀ -R _-3 |<Th2 and |G ₀ -G _-3 |>Th1 and |B ₀ -B _-3 |>Th1

If any of the above formulas is satisfied, i.e., if the difference between the values of two color components exceeds Th1 and the difference between the values of the remaining color components is less than Th2, the control unit 100 sets the Flg value for the pixel at X=-3 to 1. On the other hand, if none of the above formulas is satisfied, the control unit 100 sets the Flg value for the pixel at X=-3 to 0.

When Flg = 1, the control unit 100 also stores information about color components that exceed the threshold value Th1. For example, if the color components that exceed Th1 are red and green, the control unit 100 sets the Flg value to "1RG." Similarly, if the color components that exceed Th1 are red and blue, the control unit 100 sets the Flg value to "1RB," and if the color components that exceed Th1 are green and blue, the control unit 100 sets the Flg value to "1GB."

In addition, if there are pixels on both sides of the pixel of interest with Flg values of "1R", "1G", "1B", "1RG", "1RB", or "1GB", the control unit 100 sets the provisional scanner line determination result for the pixel of interest to 1 and also stores information about the color component set as the Flg value.

In addition, in the second scanner line detection process, the control unit 100 also performs processing on the assumption that the provisional scanner line determination result for a pixel of the provisional scanner line includes any of "1R," "1G," "1B," "1RG," "1RB," or "1GB." For example, in step S154, if the provisional scanner line determination result value for a pixel included in the target line includes two or more of "1R," "1G," "1B," "1RG," "1RB," or "1GB," the control unit 100 determines that the provisional scanner line determination results for multiple color components are included.

In this way, with the image forming apparatus of this embodiment, even if the dust or dirt on the glass or mirror is relatively large and causes color loss (reflections of light) in two components, resulting in scanner streaks, it is possible to correct this.

4. Modifications
The present invention is not limited to the above-described embodiments and various modifications are possible. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the present invention are also within the technical scope of the present invention. For example, while the above description has been given of an image processing device according to the present disclosure implemented as an image forming device, the present disclosure may also be applied to an image reading device such as a scanner. Furthermore, an image processing device according to the present disclosure may be incorporated into a server device to provide an image correction service. In this case, the server device acquires an image from another device, performs a first scanner line detection process, a second scanner line detection process, and a scanner line correction process on the acquired image, and then transmits the corrected image to the other device.

Furthermore, although the above-mentioned embodiments are described separately in some parts for convenience of explanation, they may of course be combined to the extent technically possible. For example, by combining the second and third embodiments, an image processing device can be realized that detects pixels where the change in density between a pixel of interest and pixels other than the pixel of interest exceeds a predetermined threshold for only two components as candidates for abnormal pixels, and corrects the region separation results for these candidate abnormal pixels.

In the above explanation, the method for detecting candidate abnormal pixels uses the density difference between the pixel of interest and its surrounding pixels. However, other methods are also possible as long as they detect changes between the pixel of interest and its surrounding pixels. For example, detection may be based on changes in hue between the pixel of interest and pixels other than the pixel of interest, or on changes in luminance between the pixel of interest and pixels other than the pixel of interest, or the determination of whether the pixel of interest is a candidate abnormal pixel may be based on similarities or differences between the results of region separation between the pixel of interest and pixels other than the pixel of interest.

In addition, the programs that run on each device in the embodiments are programs that control the CPU and other devices (programs that cause a computer to function) so as to realize the functions of the above-mentioned embodiments. Information handled by these devices is temporarily stored in temporary storage devices (e.g., RAM) during processing, and is then stored in various storage devices such as ROMs (Read Only Memory) and HDDs, from which it is read, modified, and written by the CPU as needed.

Here, the recording medium for storing the program may be any of semiconductor media (e.g., ROM, non-volatile memory cards, etc.), optical recording media/magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray (registered trademark) Disc), etc.), and magnetic recording media (e.g., magnetic tape, flexible disks, etc.). Furthermore, not only are the functions of the above-described embodiments realized by executing the loaded program, but the functions of the present invention may also be realized by processing in cooperation with an operating system or other application programs, etc., based on the program's instructions.

Furthermore, when distributing the program on the market, it can be stored on a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is of course also included in the present invention.

10 Image forming apparatus 100 Control unit 102 Image processing unit 104 Region separation processing unit 120 Image input unit 130 Image forming unit 140 Display unit 150 Operation unit 160 Storage unit 162 Parameter information storage area 164 Region separation information 166 Provisional scanner line determination result information 168 Scanner line determination result information 190 Communication unit

Claims (10)

An acquisition unit that acquires an image and a control unit are provided,
The control unit
Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit;
If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region;
correcting the abnormal pixels based on the detection result of the abnormal pixel region ;
If the pixels detected as candidates for the abnormal pixel are present intermittently in a row of pixels whose longitudinal direction is the sub-scanning direction, the pixels detected as candidates for the abnormal pixel are determined to be continuous in the sub-scanning direction, and the row of pixels is detected as an abnormal pixel region.
1. An image processing device comprising: 2. The image processing device according to claim 1, wherein the control unit determines that the pixels detected as candidates for abnormal pixels are continuous in the sub-scanning direction when the pixels detected as candidates for abnormal pixels are arranged side by side with a white pixel in between in a row of pixels whose longitudinal direction is the sub-scanning direction. An acquisition unit that acquires an image and a control unit are provided,
The control unit
Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit;
If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region;
correcting the abnormal pixels based on the detection result of the abnormal pixel region;
For each pixel other than the pixel of interest included in the plurality of pixels, a density difference between the pixel other than the pixel of interest and the pixel of interest is obtained for each color component;
The color component whose density difference exceeds a threshold is determined as a specific color component of a pixel other than the target pixel,
an image processing device, wherein, when the specific color component of a pixel among the plurality of pixels at a position forward in the main scanning direction relative to the target pixel is the same as the specific color component of a pixel among the plurality of pixels at a position reverse in the main scanning direction relative to the target pixel, the target pixel is detected as a candidate for the abnormal pixel. The control unit
if the specific color component of a pixel among the plurality of pixels at a position in the forward direction of the main scanning direction relative to the target pixel is the same as the specific color component of a pixel among the plurality of pixels at a position in the reverse direction of the main scanning direction relative to the target pixel, the specific color component is determined to be the specific color component corresponding to the pixel detected as the candidate for the abnormal pixel,
4. The image processing device according to claim 3, wherein, when the specific color components corresponding to the pixels detected as candidates for the abnormal pixel included in a pixel row in the sub-scanning direction are not the same, each pixel included in the pixel row in the sub-scanning direction is excluded from the candidates for the abnormal pixel. An acquisition unit that acquires an image and a control unit are provided,
The control unit
Detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs, from among the image acquired by the acquisition unit;
If the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels is detected as an abnormal pixel region;
correcting the abnormal pixels based on the detection result of the abnormal pixel region;
A method for replacing pixels included in the abnormal pixel region is switched according to region separation information around the abnormal pixel region.
1. An image processing device comprising: The control unit
If the periphery of the abnormal pixel region is a halftone dot or line region, the pixels of the abnormal pixel region are replaced with pixels based on an average value of pixel values of the pixels surrounding the abnormal pixel region;
The image processing apparatus according to claim 5 , wherein, when the periphery of the abnormal pixel region is a character region, the pixels of the abnormal pixel region are replaced with pixels based on pixel values of pixels adjacent to the abnormal pixel region. The control unit
replacing information corresponding to the replaced pixel in the region separation information with information on a region surrounding the pixel;
The image processing device according to claim 6 , wherein the image processing is performed based on the area separation information. 8. The image processing apparatus according to claim 1, wherein the acquisition unit acquires an image from a document conveyed by a document feeder. acquiring an image;
detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes among a plurality of pixels in the main scanning direction to which the pixel of interest belongs in the image;
detecting, when the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels as an abnormal pixel region;
correcting the abnormal pixels based on the detection result of the abnormal pixel region;
a step of detecting, when pixels detected as candidates for abnormal pixels are present intermittently in a pixel row having a sub-scanning direction as its longitudinal direction, determining that the pixels detected as candidates for abnormal pixels are continuous in the sub-scanning direction and detecting the pixel row as an abnormal pixel region;
An image processing method comprising: On the computer,
The ability to acquire images;
a function of detecting a pixel of interest as a candidate for an abnormal pixel based on pixel changes in a plurality of pixels in the main scanning direction to which the pixel of interest belongs in the image;
a function of detecting, when the pixels detected as the candidate abnormal pixels have continuity in the sub-scanning direction, a region in the sub-scanning direction including the pixels detected as the candidate abnormal pixels as an abnormal pixel region;
a function of correcting the abnormal pixel based on the detection result of the abnormal pixel region;
a function of detecting, when pixels detected as candidates for abnormal pixels exist intermittently in a pixel row having a sub-scanning direction as its longitudinal direction, the pixels detected as candidates for abnormal pixels as being continuous in the sub-scanning direction, and detecting the pixel row as an abnormal pixel region;
A program characterized by realizing the above.