JP3467195B2 - Character area extraction method and apparatus, and recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a frame image in which characters such as telop characters in a television broadcast image are displayed in a plurality of frames constituting a color moving image such as a television broadcast image. The present invention relates to a character area extraction technique for extracting a character portion as a pixel connected area. Specifically, when extracting a character area as a binary image from a video containing characters, the binary image of the image using the maximum / minimum method is extracted. The present invention relates to a character region extraction technique that has an advantage that pixels that form a character are less likely to be lost when the digitization technique is used.

[0002]

2. Description of the Related Art A character region extraction technique for extracting a character portion as a pixel connected region from a frame image in which characters are displayed in a plurality of frames forming a color moving image or a still color image in which characters are displayed. about,
Much research has been conducted from the past. As shown in FIG. 13, when the input character display color image is an image in a video in the NTSC signal format, the color of each pixel is blurred along each horizontal line, and the vertical stroke component with a narrow width is also in the character portion. The color of may deteriorate. There is a need for a method for stably extracting a character area even if there is color deterioration in the same character.

To solve this problem, a character area extraction method has been proposed in the past, which focuses on the characteristics of the luminance distribution on the line (reference [1]: Kuwano, Arai, Kurakake, Sugimura: "Binary value for each line". Method for Extracting Character Areas in Video Using Digitalization, "Shugaku Akidai, D-12-42, pp.264, (1996-0
9)).

This method focuses on the feature that the brightness distribution on the horizontal line has a high contrast between the telop character and the background even if color blurring occurs, and the binary distribution in each horizontal line. Process. In the binarization method, a range in which a brightness value is locally higher in a line than a peripheral part by a preset value and a range in which the brightness value is lower by a preset value are extracted as character regions. The binarization method within this line is performed for each horizontal line and each vertical line, and the binarization results in each direction are compared, and both areas have the same position, size, and shape. Is the final character area extraction result.

[0005]

However, in the above-mentioned conventional method, as shown in FIG. 14, the character range to be extracted in each line is determined in advance from the pixels having the extreme values in the luminance distribution on the line. Since the brightness value is changed by the value, if the difference in brightness between the deteriorated part and the non-deteriorated part is larger than a predetermined value in the horizontal stroke component where color deterioration occurs only at the edges, the color deteriorated part is marked. It cannot be included as a range and extracted. Further, since there are various degrees of color deterioration, depending on the value of the threshold value of the brightness difference determined in advance, the color deterioration portion may be included in the extraction. Therefore, FIG.
As shown in (1), in the vicinity of the end of the horizontal stroke component, there is a problem that the contour shape of the area becomes jagged or the stroke is interrupted in the final character area extraction result.

In the horizontal line binarization process, the horizontal boundary between the character and the background can be detected, but the vertical luminance distribution is not taken into consideration. Therefore, as shown in FIG. Since the boundary between the character and the background is not detected, the area pixel of the background portion may be extracted while being connected in the vertical direction of the character portion. Similarly, in the binarization processing of the vertical line, the boundary between the vertical character and the background is guaranteed, but the boundary between the horizontal character and the background is not detected. Therefore, as shown in FIG. , Area pixels of the background portion may be extracted while being connected in the horizontal direction of the character portion. Therefore, even if the binarization result for each horizontal line and the binarization result for each vertical line are compared, the area of the character part is not always
The positions, sizes, and shapes are not always the same, and there is a problem that they cancel each other out and the extraction of the character region fails.

Therefore, the present invention provides a character area extracting method and apparatus that smoothes the shape of the character outline portion and suppresses the failure of extracting the character area even when the brightness and the color are deteriorated in the same character. This is an issue.

[0008]

In order to solve the above-mentioned problems, a character area extracting method according to the present invention comprises a first step of inputting and storing a color image in which characters are displayed,
In the color image input in the first step, a plurality of pixels having similar luminance values and adjacent to each other in the image space are first
Of the first connected region image having similar luminance values obtained by the second step and the second connected region image formed as the connected region of In the third step of forming a plurality of pixels adjacent to each other as the second connected area and the second connected area image having similar brightness and saturation values obtained in the third step, A fourth step of forming a plurality of pixels having similar hue values in the area and adjacent to each other in the pixel space as a third connection area;
In the character area extraction method, the method includes the fifth step of leaving only the third connected area pixels obtained by the step of # 3 in the image and accumulating as a character area image.
The first step of inputting and storing a color image
The same telop text is included in color moving images such as broadcast images.
Characters such as letters are displayed consecutively in time.
Image is input and memorized, or characters are displayed
To store and store a single still color image
Steps 1-1 and a plurality of the same characters are displayed
If a color image is input, the
The average value of the pixel values of the corresponding pixels at the position is the value of each pixel
The average value color image that has as the input color image and
Realized by using the first and second stages of creating and outputting
However, the brightness values in the color image are similar, and they are adjacent in the image space.
Forming a plurality of pixels as a first connection region
The two steps of luminance within each horizontal line in the color image are
The 2-1th stage that performs binarization and obtains the result as a binary image
And binarization of luminance within each vertical line in the color image
And the second step of obtaining the result as a binary image,
2 obtained by the step 2-1 and the step 2-2
Compare connected components in two binary images,
Connected components whose position difference is within the preset allowable range
The character is judged to be the character area, and the area and shape are left in the binary image.
Shape, position difference exceeds the preset allowable range.
Boundary components are judged to be background noise parts and removed from the binary image
And the second to third steps to be performed.

[0009]

[0010]

Alternatively, in the above character area extraction method, the 2-1 step of binarizing the luminance in each horizontal line in the color image and obtaining the result as a binary image is performed in the color image. In the luminance value distribution within a horizontal line ,
Find all pixels that have a maximum value of luminance, and find each maximum value image
With the pixel as the starting point, operate horizontally on the horizontal line pixel by pixel.
The start point on the left and right sides of the maximum pixel of the start point.
The brightness value is lower than the brightness value of
Pixels of the horizontal line
In the step 2-1-1 of extracting a plurality of pixels existing between the two pixels on the IN as a connected pixel area, and the left and right of the connected pixel area obtained in the step 2-1-1. The pixels with the maximum absolute value of the horizontal brightness gradient are found on the left and right from a plurality of pixels that exist near the pixels on both ends of the pixel and are determined by a predetermined method. A step 2-1-2 of extracting as a region,
Luminance-value-distribution smell in horizontal lines with in the color image
And find all pixels that have a minimum brightness value,
Left and right pixel by pixel on the horizontal line starting from the value pixel
To open the left and right sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
The first pixel is detected, and the detected two pixels on the left and right and the water
Multiple images that exist between these two pixels on a flat line
A first 2-1-3 step of extracting the element as a connected pixel regions, were selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by step said 2-1-3 A step 2-1-4 in which the pixel having the maximum absolute value of the horizontal luminance gradient is obtained from the plurality of pixels on each of the left and right sides, and the range having the pixels at both ends is extracted as the low-luminance character region; The high-intensity character area on each horizontal line extracted in the step 2-1-2 is collected to create a high-intensity character area image as the whole image, and in the step 2-1-4. Collecting the low-intensity character areas on each of the extracted horizontal lines to create a low-intensity character area image for the entire image.
1-5, a 2-1-6 step of detecting pixels at an edge portion in the color image, a high-intensity character area image obtained in the 2-1-5 step, and a low From the plurality of pixels forming the character connected component in the luminance character area image, the second
And the step 2-1-7 of removing pixels corresponding to the edge portion in the vertical direction in the color image obtained in step 1-6.

Alternatively, in the above character area extraction method, the second to second steps of binarizing the luminance in each vertical line in the color image and obtaining the result as a binary image are performed. In the luminance value distribution in a certain vertical line ,
Find all pixels that have a maximum value of luminance, and find each maximum value image
The element as a starting point, steering vertically on the vertical line pixel by pixel
Start point on both upper and lower sides of the maximum pixel of the start point
The brightness value is lower than the brightness value of
Detecting a pixel, the detected upper and lower two pixels and the vertical La
In the step 2-2-1 of extracting a plurality of pixels existing between the two pixels on the IN as a connected pixel area, and the left and right sides of the connected pixel area obtained in the step 2-2-1. The pixels with the maximum absolute value of the vertical brightness gradient are found on the left and right from a plurality of pixels that exist near the pixels on both ends of the pixel, and the range with these pixels at both ends is defined as high-brightness characters. 2-2-2 stage of extracting as a region,
Luminance-value-distribution smell of the vertical line with in the color image
And find all pixels that have a minimum brightness value,
The value pixel as a starting point, lower the upper the vertical line pixel by pixel
To open both the upper and lower sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
Detecting a first pixel, a plurality of picture that exists between those two pixels at the detected upper and lower two pixels and the vertical <br/> straight line on
A first 2-2-3 step of extracting the element as a connected pixel regions, were selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by step said 2-2-3 A step 2-2-4 in which a pixel having the maximum absolute value of the vertical luminance gradient is obtained from the plurality of pixels on each of the left and right sides, and a range having the pixels at both ends is extracted as a low-luminance character region; The high-intensity character area on each vertical line extracted in the step 2-2-2 is collected to create a high-intensity character area image as the entire image, and is also extracted in the step 2-2-4. The low-intensity character area on each vertical line is collected to create a low-intensity character area image for the entire image. 2-2
-5, a 2-2-6 step of detecting a pixel at an edge portion in the color image, a high-intensity character area image obtained in the 2-2-6 step, and a low intensity From the plurality of pixels forming the character connected component in the character area image,
-2-7-7 of removing pixels corresponding to the edge portion in the vertical direction in the color image obtained in step -6,
It is characterized by using it.

Alternatively, in the above-mentioned character area extracting method, the values obtained by the steps 2-1 and 2-2 can be obtained.
Compare connected components in two binary images,
Connected components whose position difference is within the preset allowable range
The character is judged to be the character area, and the area and shape are left in the binary image.
Shape, position difference exceeds the preset allowable range.
Boundary components are judged to be background noise parts and removed from the binary image
In the second to third steps, the two obtained binary images are arranged.
And align them so that the two corresponding pixels are both
If it belongs to a connected component, the corresponding pixel is regarded as a temporary character pixel.
Memorize and connect either one of the two corresponding pixels.
If the pixel does not belong to a component, the corresponding pixel is set as a non-tentative character area.
Stored, and includes a connected component composed of the temporary character pixel
Step 2-3-1 to create an image by defining it as an AND image
The two floor images and the obtained two binary images are aligned and superposed.
And connect either one of the two corresponding pixels
If it belongs to a component, store the corresponding pixel as a temporary character area
And both corresponding two pixels do not belong to the connected component
In this case, the corresponding pixel is stored as a non-temporary character area and
An image including a connected component composed of pixels is defined as an OR image.
2-3-2 stage to be created in the sense of
AND image obtained by the step 2 and the step 2-3-2
The OR images obtained by
Among the connected components in the AND image, the connected component in the OR image
If the number of pixels that overlap the
And the area of the connected component in the OR image with overlap is
If it is within the preset range, the connection formation in the AND image
Minutes are judged as character connected components and left in the AND image.
However, the other connected components serve as the background noise region of the AN.
2-3-3 step of removing from the D image, and the 2-3 step
Only the character connected component obtained by the step of -3 remains
Third, storing pixels in the AND image as character area pixels
-2-4 stages and are realized by using
It Alternatively, in the above character area extraction method, the second
The connected components in the two binary images obtained in the step -1 and the step 2-2 are compared, and the connected components whose difference in area, shape, and position are within the preset allowable range are compared. It is determined that the character region is left in the binary image, and connected components whose differences in area, shape, and position exceed a preset allowable range are determined to be background noise portions and are removed from the binary image.
In step 3, the obtained two binary images are overlapped with their positions aligned, and when the corresponding two pixels both belong to the connected component, the corresponding pixel is stored as a temporary character pixel, and the corresponding two pixels are stored. If either one of the two pixels does not belong to the connected component, the corresponding pixel is stored as a non-temporary character area,
An image including a connected component composed of the temporary character pixels is AN
The step 2-3-1, which is defined and created as a D image, and the obtained two binary images are aligned and overlapped, and one of the corresponding two pixels is used as a connected component. If it belongs, the corresponding pixel is stored as a temporary character area, and if neither of the two corresponding pixels belong to the connected component, the corresponding pixel is stored as a non-temporary character area and a connected component composed of temporary character pixels is stored. The image obtained by defining the image including the image as the OR image is created in the step 2-3-2, the AND image obtained in the step 2-3-1, and the step obtained in the step 2-3-2. The OR images are overlapped with their positions aligned, and in the connected components in the AND image, the number of pixels overlapping with the connected components in the OR image is equal to or greater than a preset threshold value, and the OR images have an overlap. If the area of the connected components is within the preset range, The second to third steps of judging the connected component in the AND image as a character connected component and leaving it in the AND image, and removing the other connected components from the AND image as a background noise region; -3-Connected pixels in each line obtained by the binarization for each horizontal line obtained in the step 2-1 in the AND image in which only the character connected components obtained in the step 3-3 remain Among them, a connected pixel having a connected pixel number smaller than a preset value is set as a character area, and a pixel in the AND image existing at the same position as the pixel is stored as a character area pixel. Is realized by using.

Further, a program for causing a computer to execute the steps in the character area extracting method described above is recorded on a computer-readable recording medium.

Also, in order to solve the above-mentioned problems, the character area extracting device according to the present invention includes a color image input storing means for inputting and storing a color image in which characters are displayed, and the color image input storing means. Brightness value similar areas in the color image input by the means, the brightness value similar area forming means forming a plurality of adjacent pixels in the image space as a connection area, and the brightness value similar area forming means And a saturation value similar region forming means for forming a plurality of adjacent pixels in the image space as a connection region, the saturation value being similar to the connection region images having similar brightness values. Luminance value obtained by the saturation value similar region forming means,
Within each region, for connected region images with similar saturation values,
Hue value similar, forming a plurality of adjacent pixels in the pixel space as a connected area, a hue value similar area forming unit, and leaving only the connected area pixels obtained by the hue value similar area forming unit in the image, The execution order of the character area image accumulation means for accumulating as a character area image, the color image input storage means, the luminance value similar area forming means, the saturation value similar area forming means, and the hue value similar area forming means is controlled. In the character area extraction device including processing control means,
Color image to be stored by inputting the displayed color image
The image input storage means is a color moving image such as a television broadcast image.
, The same characters such as telop characters are displayed.
Input and store multiple time-sequential color images
Or a static color with the text displayed
Color moving image / still image input record for inputting and storing images
Memories and multiple colors in which the same characters are displayed
When an image is input, the position between the color images
The average value of the pixel values of corresponding pixels is set as the value of each pixel.
Color image input by creating an average value color image
And a plurality of frame averaging means for outputting
Image / still image input storage means and plural frame averaging means
Processing control means for controlling the execution order of
-Plurality of images that have similar brightness values and are adjacent in the image space
Luminance value similar region shape that forms pixels of each other as connected region
The synthesizing means has two luminance values within each horizontal line in the color image.
Horizontal line unit that performs binarization and obtains the result as a binary image
The binarization means and the brightness within each vertical line in the color image
A vertical line that binarizes and obtains the result as a binary image.
A unit binarizing unit, the horizontal line unit binarizing unit, and the hanging unit.
Two binary images obtained by the straight line unit binarizing means
Compare the connected components inside and see the difference in area, shape, and position.
Connected components that are within the preset allowable range to character areas
The difference in area, shape, and position.
Connected components that exceed the preset allowable range
A binarization result that is judged to be a background noise part and is removed from the binary image
Integrating means, the horizontal line unit binarizing means, the vertical line
Execution order of the binarization unit and the binarization result integration unit
And a processing control means for controlling the order.
To do.

[0016]

[0017]

Alternatively, in the above character area extracting device, the horizontal line unit binarizing means for binarizing the luminance in each horizontal line in the color image and obtaining the result as a binary image is Luminance value distribution in a horizontal line with
, All pixels with the maximum brightness value
Each local maximum pixel is used as a starting point, and no pixel is added on the horizontal line.
The left and right sides of the maximum pixel at the starting point
Luminance value by the preset threshold value than the luminance value of each start point
The first pixel is low, the two pixels on the left and right are detected
And the complex existing between the two pixels on the horizontal line.
A maximum luminance region detecting unit that extracts a number of pixels as a continuous pixel region, and a plurality of pixels selected by a predetermined method that exist near the left and right end pixels of the continuous pixel region obtained by the maximum luminance region detecting unit. There is a high-intensity character area extracting means for extracting a pixel having the maximum absolute value of the horizontal brightness gradient from the inside on each of the left and right sides, and extracting a range having the pixels at both ends as a high-intensity character area, and in the color image. In the luminance value distribution in the horizontal line, all pixels that have a minimum luminance value
The horizontal line is obtained by using each obtained minimum pixel as a starting point.
Operate the top one pixel at a time to the left and right to the left of the minimum pixel at the start point.
The threshold values set on the right and left sides are more than the brightness value of the starting point.
The first pixel with high brightness value is detected, and the detected left pixel
Between the right two pixels and the two pixels on the horizontal line
Minimum luminance region detecting means for extracting a plurality of pixels existing in the above as a connected pixel region, and a plurality of pixels selected by a predetermined method existing near the left and right end pixels of the connected pixel region obtained by the minimum luminance region detecting means. Low-luminance character region extraction means for obtaining a pixel having the maximum absolute value of the horizontal luminance gradient from the left and right sides, and extracting a range having the pixels at both ends as a low-luminance character region, and the high-luminance The high-intensity character areas on each horizontal line extracted by the character area extracting means are collected to create a high-intensity character area image as the whole image, and each horizontal line extracted by the low-intensity character area extracting means A binarization result integrating means for each horizontal line that collects the above low-brightness character areas to create a low-brightness character area image for the entire image, the brightness maximum area detecting means, the high-brightness character area extracting means, Kagayaki Processing control means for controlling the execution order of the minimum area detection means, the low-brightness character area extraction means, and the horizontal line-by-horizontal binarization result integration means; and edge detection means for detecting pixels at the edge portion in the color image. , A high-intensity character area image obtained by the horizontal line binarization result integrating means, and a color image obtained by the edge detecting means from a plurality of pixels forming a character connected component in the low-intensity character area image Edge portion removing means for removing pixels corresponding to the edge portion in the vertical direction.

Alternatively, in the above character area extracting device, the vertical line unit binarizing means for binarizing the brightness in each vertical line in the color image and obtaining the result as a binary image is Luminance value distribution in vertical line with
, All pixels with the maximum brightness value
Each local maximum pixel is set as the starting point, and no pixel is added on the vertical line.
Up and down, and move it up and down on both sides of the maximum pixel at the starting point.
Luminance value by the preset threshold value than the luminance value of each start point
The first pixel is low, the upper and lower two pixels detected
Double that to exist between those two pixels on the vertical line
A maximum luminance region detecting unit that extracts a number of pixels as a continuous pixel region, and a plurality of pixels selected by a predetermined method that exist near the left and right end pixels of the continuous pixel region obtained by the maximum luminance region detecting unit. A pixel having the maximum absolute value of the vertical brightness gradient from the inside is obtained on each of the left and right sides, and a high-intensity character area extracting means for extracting a range having the pixel at both ends as a high-intensity character area, and in the color image In the luminance value distribution in the vertical line, all pixels that have a minimum luminance value
The vertical line is obtained by using each obtained minimum pixel as a starting point.
Move up and down one pixel at a time, and
The threshold values that are set in advance on both lower sides than the brightness value of the starting point
The first pixel with a high brightness value is detected and
Between these two pixels under two pixels and on the vertical line
Minimum luminance region detecting means for extracting a plurality of pixels existing in the above as a connected pixel region, and a plurality of pixels selected by a predetermined method existing near the left and right end pixels of the connected pixel region obtained by the minimum luminance region detecting means. A pixel having the maximum absolute value of the brightness gradient in the vertical direction from the left and right pixels, and a low-brightness character region extracting means for extracting a range having the pixels at both ends as a low-brightness character region; The high-intensity character areas on each vertical line extracted by the character area extracting means are collected to create a high-intensity character area image as the whole image, and each vertical line extracted by the low-intensity character area extracting means. A binarization result integrating means for each vertical line that collects the above-described low-intensity character areas to create a low-intensity character area image as an entire image, the maximum brightness area detecting means, the high brightness character area extracting means, The Processing control means for controlling the execution sequence of the minimum degree area detection means, the low-brightness character area extraction means, and the vertical line-by-vertical binarization result integration means; and edge detection means for detecting pixels in the edge portion in the color image And a color image obtained by the edge extracting means from a plurality of pixels forming a character connected component in the high-intensity character area image and the low-intensity character area image obtained by the vertical line-by-vertical binarization result integrating means And an edge portion removing means for removing pixels corresponding to the edge portion in the vertical direction.

Alternatively, in the above character area extraction device, horizontal line unit binarization means and vertical line unit binarization
The connected components in the two binary images obtained by
Compared with each other, the difference in area, shape, and position is set within the allowable range.
The connected components within the box are judged to be character areas, and the binary image
Allowed to remain in the image, with preset differences in area, shape, and position
Connected components that exceed the range are judged to be background noise.
The binarization result integrating means for removing from the binary image is
The two binary images that have been
If the two pixels that belong to the
The pixel is stored as a temporary character pixel, and
If either of them does not belong to the connected component, the corresponding
Pixels are stored as non-temporary character areas and composed of temporary character pixels
Created by defining an image containing connected components as an AND image
AND image creating means and the two binary images obtained
Of the two corresponding pixels
If either one belongs to the connected component, the corresponding pixel is
It is stored as a temporary character area, and two corresponding pixels are
If the corresponding pixel does not belong to the connected component,
Stored as, and includes the connected component composed of the dummy character pixels.
OR image creating means for creating and defining an image as an OR image
And an AND image obtained by the AND image creating means
The positions of the OR images obtained by the OR image creating means are aligned.
Of the connected components in the AND image,
The number of pixels that overlap with the connected component in the OR image is preset
In the OR image which is equal to or greater than the threshold value and has an overlap
If the area of the connected component is within the preset range, the AN
The connected component in the D image is determined as the character connected component, and the A
It is left in the ND image and the other connected components are background noise regions.
Character connected component judgment hand to be removed from the AND image as a region
And the character concatenation obtained by the character connected component determination means
Pixels in the AND image where only the components remain are character area pixels
And a character pixel registration means for storing as
Character area extraction device. Alternatively, in the above character area extraction device, the connected components in the two binary images obtained by the horizontal line unit binarization unit and the vertical line unit binarization unit are compared to determine the area, shape, and position. The connected components whose difference is within the preset allowable range are judged to be character regions and are left in the binary image, and the connected components whose area, shape, and position differences exceed the preset allowable range are background noise. If the binarization result integrating means for judging that the binary image is a part and removing it from the binary image, the obtained two binary images are superposed with their positions aligned and both corresponding two pixels belong to the connected component. , The corresponding pixel is stored as a temporary character pixel, and if either one of the two corresponding pixels does not belong to the connected component, the corresponding pixel is stored as a non-temporary character area, and the connected pixel is composed of the temporary character pixels. A containing the component The AND image creating means defined to be created as a D image and the obtained two binary images are overlapped with their positions aligned, and if either one of the corresponding two pixels belongs to the connected component, The corresponding pixel is stored as a temporary character area and the corresponding 2
When neither one pixel belongs to the connected component, the corresponding pixel is stored as a non-temporary character area, and an image including the connected component composed of the temporary character pixels is defined as an OR image and created. , The AND image obtained by the AND image creating means and the O image obtained by the OR image creating means
The R images are overlapped with their positions aligned, and in the connected components in the AND image, the number of pixels overlapping with the connected components in the OR image is equal to or greater than a preset threshold value, and the OR images have an overlap. If the area of the connected component therein is within a preset range, the connected component in the AND image is determined as a character connected component and left in the AND image, and the other connected components are used as the background noise area in the AND. In the AND image in which only the character connected components obtained by the character connected component determination means and the character connected component determination means to be removed from the image remain,
Of the connected pixels in each line obtained by the binarization for each horizontal line obtained by the horizontal line unit binarization means, the connected pixels whose number of connected pixels is less than a preset value are set as the character area, and Character pixel registration means for storing a pixel in the AND image existing at the same position as the pixel as a character area pixel.

As described above, the conventional method has a problem that the extraction of the color deteriorated portion at the end of the horizontal stroke is unstable. Therefore, in the present invention, a range that locally stands out from the brightness distribution on the line is obtained by using a predetermined threshold value of the brightness value, and then the range of the brightness gradient values at both ends of the obtained range is further calculated. By setting the maximum pixel to the end of the true character, the character color deterioration part can be stably included in the character area and extracted, so that the character outline shape is smooth as in the original image and the stroke is interrupted. Can be eliminated (see FIG. 10).

Further, in the conventional method, the boundary between the character and the background can be guaranteed only in the reference direction in the line unit for each direction, so that there is a problem in that the extraction of the character region finally fails. In the present invention, after binarizing line by line in each direction, edge pixels in the original image are removed from the connected component, so that after the binarization in each direction, all boundary parts between the character and the background are detected. However, the character part can be extracted as a connected component completely separated from the background. As a result, the binarized results for each line in each direction are finally overlapped, compared, and even in the final integrated result, the character part has the same position, size, and shape in each result, It is possible to suppress the failure to extract the character area (FIGS. 11 and 12).
reference).

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing the flow of processing along with the configuration of an apparatus according to an embodiment of the present invention.

In FIG. 1, reference numeral 1 denotes a color image input / storage unit which inputs color image data in which a character to be processed is displayed and stores it in a memory.

Reference numeral 2 denotes a brightness similar area forming section, which forms a plurality of pixels having similar brightness values in the color image input by the color image input storage section 1 and adjacent to each other in the image space as a connection area.

Reference numeral 3 denotes a saturation value similar area forming unit, which has similar saturation values in each area in the connected area pixels having similar brightness values obtained by the brightness value similar area forming unit 2 and is in the image space. A plurality of pixels adjacent to each other are formed as a connection region.

Reference numeral 4 denotes a hue value similar region forming unit, which has similar hue values in each region of connected region pixels having similar luminance values and saturation values obtained by the saturation value similar region forming unit 3, A plurality of adjacent pixels in the image space are formed as a connected region.

A character area image storage unit 5 stores the connected area image obtained by the hue value similar area forming unit 4 as a character area extraction result image.

A processing control unit 6 controls the execution order of each of the above units 1 to 5.

FIG. 2 shows a color image input storage unit 1 in FIG.
3 is a block diagram showing an example of the flow of processing together with an example of the configuration of FIG.

In FIG. 2, reference numeral 7 denotes a color image input / storing unit, which is a plurality of time-sequential color images in which the same telop character or the like is displayed in a color moving image such as a television broadcast image. Or a single static color image containing characters is input and stored.

A multi-frame averaging unit 8 has an average value color image having an average value of pixel values of corresponding pixels at positions among a plurality of color images displaying the same character as a value of each pixel. To create.

Generally, in a color moving image such as a television broadcast image, the telop character display is displayed at the same position on the screen over a plurality of frames. On the other hand, there are many cases where motion occurs in general objects other than character display and the background part.

Therefore, the pixel values of corresponding pixels between a plurality of temporally consecutive color images in which the same telop character is displayed in the color moving image do not change in the character portion over a plurality of frames. Since there is a motion of the object in the background portion, the value changes over a plurality of frames.

Therefore, when an averaged color image having the average value of the corresponding pixels of the plurality of frames as the value of each pixel is created, the value remains almost the same even after the averaging because the value does not change in the character portion over the plurality of frames. Although it does not change, in the background part, various values are averaged, so that the object contour part in the image becomes unclear and blurry.

This has the effect of suppressing the extraction of the area of the general object from the background portion in the binarization processing of the image in the subsequent stage.

A processing control unit 9 controls the execution order of each unit 7-8.

FIG. 3 shows the luminance value similar region forming section 2 in FIG.
3 is a block diagram showing an example of the flow of processing together with an example of the configuration of FIG.

In FIG. 3, reference numeral 10 denotes a horizontal line unit binarizing unit which binarizes the luminance in each horizontal line in the image.

Reference numeral 11 denotes a vertical line unit binarizing unit which binarizes the luminance in each vertical line in the image.

Reference numeral 12 is a binarization result integration unit, which compares the two binary images obtained by the horizontal line unit binarization unit 10 and the vertical line unit binarization unit 11, and both results are the same. Connected components existing at the positions and having the same area value are determined to be character regions, and other connected components are determined to be background noise portions and are removed from the image.

Reference numeral 13 denotes a processing control unit, which is the above 10 to 12
Control the execution order of each part of.

FIG. 4 is a block diagram showing an example of the configuration of the horizontal line unit binarization unit 10 in FIG. 3 and an example of the processing flow.

In FIG. 4, reference numeral 14 denotes a horizontal maximum luminance area detection unit, which examines the luminance distribution in a certain horizontal line in the image and locally increases the luminance value in the horizontal line by a preset value. The connected pixel area is detected.

Reference numeral 15 denotes a high-intensity character area extraction unit, which is located on a horizontal line in the image and is among a plurality of pixels near the left and right ends of the connected pixel area obtained by the horizontal maximum-intensity area detection unit 14. From the above, the pixel having the maximum absolute value of the horizontal brightness gradient is obtained on each of the left and right sides, and the range having the pixels at both ends is extracted as the high brightness character area.

Reference numeral 16 denotes a horizontal minimum-brightness area detection unit, which checks the brightness distribution in a certain horizontal line in the image and determines a connected pixel area in which the brightness value is locally lower by a preset value. To detect.

Reference numeral 17 denotes a low-intensity character area extraction unit, which is located on a horizontal line in the image, among a plurality of pixels near the left and right ends of the connected pixel area obtained by the horizontal minimum-intensity area detection unit 16. From this, the pixel having the maximum absolute value of the horizontal brightness gradient is obtained on each of the left and right sides, and the range having the pixels at both ends is extracted as the low-intensity character area.

A horizontal line binarization result integration unit 18 integrates the character regions on each horizontal line obtained by the high-intensity character region extraction unit 15 and the low-intensity character region extraction unit 17 to obtain the entire image. Create a character area image of.

A processing control unit 19 controls the execution order in each of the units 14-18.

Reference numeral 20 denotes an edge detection unit which detects pixels in the edge portion in the image. For edge detection, for example, Robi
An existing method such as using an operator of nson is used.

Reference numeral 21 denotes an edge removal section, which is a plurality of pixels forming the character connected components in the high-intensity character area image and the low-intensity character area extracted image obtained by the horizontal line binarization result integrating section 18. The pixels corresponding to the vertical edge portions in the average value image obtained by the edge detection unit 18 are removed.

FIG. 5 is a block diagram showing an example of the configuration of the vertical line unit binarization unit 11 in FIG. 3 and an example of the processing flow.

In FIG. 5, reference numeral 22 denotes a vertical maximum luminance area detection unit for examining the luminance distribution in a certain vertical line in the image, and locally increasing the luminance value in the vertical line by a preset value. The connected pixel area is detected.

Reference numeral 23 denotes a high-intensity character area extracting unit, which is located on a vertical line in the image and is among a plurality of pixels near the left and right ends of the connected pixel area obtained by the vertical-line maximum brightness area detecting unit 22. A pixel having the maximum absolute value of the brightness gradient in the vertical direction is obtained from each of the left and right, and a range having the pixels at both ends is extracted as a high brightness character area.

Reference numeral 24 denotes a vertical minimum luminance area detection unit, which checks the luminance distribution in a certain vertical line in the image, and determines a connected pixel area in which the luminance value is locally lower by a preset value. To detect.

Reference numeral 25 is a low-intensity character area extraction unit, which is located on a vertical line in the image, and among a plurality of pixels near the left and right ends of the connected pixel area obtained by the vertical minimum brightness area detection unit 24. From the above, the pixel having the maximum absolute value of the brightness gradient in the vertical direction is obtained on each of the left and right sides, and the range having the pixels at both ends is extracted as the low brightness character area.

Reference numeral 26 denotes a vertical line-by-vertical binarization result integration unit, which integrates the character regions on each vertical line obtained by the high-intensity character region extraction unit 23 and the low-intensity character region extraction unit 25 to obtain the entire image. Create a character area image of.

A processing control unit 27 controls the execution order in each of the units 22 to 26.

Reference numeral 28 denotes an edge detection unit which detects pixels in the edge portion of the image. For edge detection, for example, Robi
An existing method such as using an operator of nson is used.

Reference numeral 29 denotes an edge portion removing unit, which is a plurality of pixels forming a character connected component in the high-intensity character region image and the low-intensity character region extracted image obtained by the vertical line-specific binarization result integrating unit 26. Pixels corresponding to the vertical edge portions in the average value image obtained by the edge detection unit 28 are removed.

FIG. 6 is a block diagram showing an example of the configuration of the binarization result integration unit 12 in FIG. 3 and an example of the processing flow.

In FIG. 6, reference numeral 30 denotes an AND image creating section, which superimposes two input binary images with their positions aligned, and when two corresponding pixels both belong to a connected component, the corresponding pixel. Is stored as a temporary character pixel, and if any one of the two corresponding pixels does not belong to the connected component, the corresponding pixel is stored as a non-temporary character area and an image including a connected component composed of the temporary character pixel is stored. (Here, this is defined as an AND image) is created.

Reference numeral 31 is an OR image creating section, which superimposes two input binary images with their positions aligned and superimposes them.
If either one of the two pixels belongs to the connected component, the corresponding pixel is stored as a temporary character area and the corresponding 2
If neither of the two pixels belongs to a connected component, the corresponding pixel is stored as a non-temporary character area, and an image including a connected component composed of temporary character pixels (herein, this is defined as an OR image) is created. To do.

Reference numeral 32 is a character connected component judgment unit, which is AND
The AND image obtained by the image creating unit 30 and the OR image obtained by the OR image creating unit 31 are aligned and overlapped, and among the connected components in the AND image, the pixels that overlap with the connected component in the OR image. Is greater than or equal to a preset threshold value and the area of the connected component in the OR image having overlap is within the preset range, the connected component in the AND image is determined as the character connected component.

Reference numeral 33 denotes a character pixel registration unit, which is included in each line obtained by the horizontal line unit binarization unit 10 in the AND image in which only the character connected components obtained by the character connected component determination unit 32 remain. Among the connected pixels, a connected pixel in which the number of connected pixels is smaller than a preset value is set as a character area, and a pixel in the AND image existing at the same position as the pixel is stored as a character area pixel.

FIG. 7 is a flow chart for explaining an embodiment of the processing of the horizontal maximum brightness area detection unit 14 and the high brightness character area extraction unit 15 in FIG. In addition,
The vertical maximum brightness area detection unit 22 in FIG.
The processing of the degree character area extracting unit 23 is performed in the same manner. However
However, read horizontal as vertical, left as upper, and right as lower.

In FIG. 7, step (30) is a process of reading the luminance image into the memory.

Step (31) is a process of determining whether or not there is an unprocessed horizontal line in the luminance similar character area extraction process. If there is an unprocessed horizontal line, the process proceeds to step (32), If there is no unprocessed horizontal line, the process ends.

In step (32), the luminance distribution is checked for each horizontal line in the image obtained in step (31),
This is a process of obtaining all pixels having a maximum value.

The step (33) is a step of judging whether or not there is an unprocessed local maximum pixel of the high-luminance character region extracting process among the local maximum pixels obtained in the step (32). If there is a local maximum value pixel of, the process proceeds to step (34), and if there is no unprocessed local maximum value pixel, the process proceeds to step (31).

In step (34), the maximum value pixel obtained in step (33) is used as a starting point, and one pixel is scanned on the horizontal line in the leftward and rightward directions. A process of detecting a first pixel having a brightness value lower than a maximum value by a preset threshold value and extracting the detected two pixels and a pixel existing between the two pixels on the horizontal line as a connected pixel region. Is.

Step (35) is a process of obtaining the absolute value of the brightness gradient value in a plurality of pixels in a certain range at the left and right ends of the connected pixel area obtained in step (34).

The horizontal coordinate values of the left and right end points of the connected pixel area on a certain horizontal line obtained in step (34) are respectively defined as x and y (0 <x <y, the coordinate value is the leftmost end of the line). Is set to 0, and thereafter, is incremented by 1 every time one pixel is shifted to the right). The constant range in x and y is
X-r, x-r + 1, ..., X, ..., X + r-
1, x + r, and y-r, y-r + 1, ..., Y, ...,
y + r-1, y + r. Note that r is a preset value.

The luminance values of the pixels in the fixed range on both ends are set to L
(X-r), L (x-r + 1), ..., L (x), ..., L
(X + r-1), L (x + r), and R (x-r),
R (x-r + 1), ..., R (x), ..., R (x + r-
1) and R (x + r), the absolute value of the brightness gradient value at each coordinate in the fixed range at both ends is abs {L (x−r) −L (x−r−1)} for the left end. ，
..., abs {L (x) -L (x-1)}, ..., abs
{L (x + r) -L (x + r-1)}, for the right end, abs {R (x-r) -R (x-r + 1)}, ...,
abs {R (x) -R (x + 1)}, ..., abs {R
(X + r) -R (x + r + 1)}. However, for each pixel on the left end, only one absolute value of the brightness difference with the pixel on the left is calculated, and for each pixel on the right end, the absolute value of the brightness difference with the pixel on the right is calculated. The value is the absolute value of the brightness gradient value in each pixel.

In step (36), the maximum value is obtained for each of the left and right of the absolute values of the brightness gradient values obtained in step (35). If both are larger than the preset value, the process proceeds to step (37). If either one is smaller than the preset value, the process proceeds to step (33).

The maximum absolute value of the brightness gradient values of the pixels in the fixed range at the left end of the pixel connection area obtained in step (35) is max [abs {L (x-r) -L (x-r-
1)}, ..., abs {L (x) -L (x-1)}, ...,
abs {L (x + r) -L (x + r-1)}],
The maximum absolute value of the brightness gradient values of the pixels in the certain range on the right end is max [abs {R (x-r) -R (x-r +
1)}, ..., Abs {R (x) -R (x + 1)}, ...,
abs {R (x + r) -R (x + r + 1)}].

Generally, there is a large difference between the brightness of a character on the character display screen and the brightness of the background in contact with the character.
There is a sharp brightness gradient at the boundary between the two. On the other hand, a sharp brightness gradient does not always exist at the boundary of general objects other than the characters displayed on the screen. Therefore, when the absolute values of the brightness gradient values at both the left and right ends of the connected pixel area obtained in step (35) are both larger than a preset value, the connected pixel area is set as a character area, and either one of them is also set in advance. When the value is smaller than the value, the background noise area is removed, so that the extraction of the background noise area can be suppressed more effectively than before, and the character area can be extracted with high accuracy.

The step (37) includes pixels at both ends of which the absolute value of the brightness gradient value obtained in the step (36) is larger than a preset value, and the pixels existing between the pixels are defined as the high brightness character area. This is the process of confirming and returning to step (33).

FIG. 8 is a flow chart for explaining an embodiment of the processing of the horizontal minimum brightness value area detection unit 16 and the low brightness character area extraction unit 17 in FIG. In addition,
The vertical minimum luminance value area detection unit 24 in FIG.
The processing of the degree character area extracting unit 25 is performed in the same manner. However
However, read horizontal as vertical, left as upper, and right as lower.

In FIG. 8, step (40) is a process of reading the luminance image into the memory.

Step (41) is a process of determining whether or not there is an unprocessed horizontal line in the luminance similar character area extraction process. If there is an unprocessed horizontal line, the process proceeds to step (42). If there is no unprocessed horizontal line, the process ends.

The step (42) examines the luminance distribution on each horizontal line in the image obtained in the step (41),
This is a process of obtaining all pixels having a minimum value.

Step (43) is a process of determining whether or not there is an unprocessed minimum value pixel of the low-luminance character region extraction processing among the minimum value pixels obtained in step (42). If there is a local minimum value pixel, the process proceeds to step (44). If there is no unprocessed local minimum value pixel, the process proceeds to step (41).

In step (44), the minimum value pixel obtained in step (43) is used as a starting point, and the horizontal line is scanned one pixel at a time in the leftward and rightward directions. also left <br/> side, detects the luminance value is higher the first pixel by the threshold value set in advance from the minimum value, respectively on the right side, between the two pixels on the detected two pixels and the horizontal line This is a process of extracting existing pixels as a connected pixel area.

Step (45) is a process of obtaining the absolute value of the brightness gradient value in a plurality of pixels in a certain range at the left and right ends of the connected pixel area obtained in step (44).

In step (46), the maximum value is obtained for each of the left and right of the absolute values of the brightness gradient values obtained in step (45). If both are larger than the preset value, the process proceeds to step (47). If either one is smaller than the preset value, the process proceeds to step (43).

The step (47) includes pixels at both ends of which the absolute value of the brightness gradient value obtained in the step (46) is larger than a preset value, and the pixels existing between the pixels are defined as the low brightness character area. This is the process of confirming and returning to step (43).

FIG. 9 is a block diagram of the character connected component judging unit 32 shown in FIG.
5 is a flowchart for explaining an embodiment of the processing of FIG.

In FIG. 9, the step (50) is AND
This is the process of reading the image and the OR image into the memory.

Step (51) is a process of aligning the positions of the AND image and the OR image.

Step (52) is a process of initializing the variable n for the connected component number in the AND image to 1.

In step (53), the variable n is compared with the total number N of connected components in the AND image. If the value of n is N or less, the process proceeds to step (54). If the value of n is larger than N, This is the process of ending the process.

In step (54), the number Q of connected component pixels in the OR image which overlaps with the nth connected component in the AND image.
_This is a process of calculating _or (n) and the number of pixels Q _itand (n) having the _overlap in the n-th connected component in the AND image.

Step (55) consists of Q _and (n) and AN.
A value (P _and (n) × th) obtained by multiplying the area value P _and (n) of the n-th connected component in the D image by a preset value th1.
1) and comparing Q _or (n) with the area value P _or (n) of the connected component including the overlapping pixel in the OR image multiplied by a preset value th2 (P _or (n) × th2 ),
Q _and (n) is P _and (n) × th1 or more, and Q
_{If or} (n) is P _or (n) × th2 or more, the process proceeds to step (56), and if not, the process proceeds to step (57).

Step (56) is a process of determining the nth connected component in the AND image as a character area. In general, a character in a video is an area in which both the AND image and the OR image have almost the same area value, whereas a general object other than the character is necessarily obtained as a region in which the AND image and the OR image both have substantially the same area. Step (56) in step (55) because it is not always
By satisfying the condition for moving to, it becomes possible to effectively remove the area of the general object other than the character and extract the character area with high accuracy.

Step (57) is a process of incrementing the variable n by 1 and returning to step (53).

FIG. 10 is a diagram for explaining a character area range on a line determined by the present invention. In the present invention, a range that locally stands out from the brightness distribution on the line is obtained by using a predetermined brightness value threshold value, and then the maximum of the brightness gradient values at both ends of the obtained range is determined. By setting the pixel to be the end of the true character, the character color deterioration portion can be stably included in the character area and extracted. This makes it possible to make the contour shape of the character smooth as in the original image, and to eliminate the stroke discontinuity generated in the conventional method.

FIG. 11 is a diagram showing an example of the binarization result according to the present invention. FIG. 12 is a diagram for explaining the details. In the conventional method, as shown in FIG. 16, in the line unit for each direction, the boundary detection between the character and the background can be guaranteed only in the reference direction, so that there is a problem that the extraction of the character region finally fails. . In the present invention, after binarizing line by line in each direction, edge pixels in the original image are removed from the connected component, so that after the binarization in each direction, all boundary parts between the character and the background are detected. However, the character part can be extracted as a connected component completely separated from the background. As a result, the binarized results for each line in each direction are finally overlapped, compared, and even in the final integrated result, the character part has the same position, size, and shape in each result, It is possible to suppress the failure to extract the character area.

It should be noted that a part or all of the means shown in FIGS. 1 to 6 can be made to function by using a computer, or the steps of the processing shown in FIGS. Needless to say, it is possible to execute a program for causing a computer to function as the means or a program for executing the processing steps of the computer by a computer-readable recording medium such as an FD.
(Floppy disk), MO, ROM, memory card, CD, DVD, removable disk, etc. can be recorded, provided, and distributed.

[0101]

As described above, according to the present invention,
When binarizing line by line in an image, find the pixel with the largest brightness gradient at both ends of the part where the brightness distribution on the line has locally distinct values, and use this as the end of the character range. Thus, it is possible to obtain the effect of stably extracting the character region including the deteriorated portion at the end of the horizontal stroke. In addition, since the edge portion is removed after binarizing line by line for each direction, the character and the background can be reliably separated, and an effect of suppressing a failure in extracting the character region can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a character area extracting device according to an embodiment of the present invention and a processing flow.

FIG. 2 is a block diagram showing an example of a configuration of a color image input storage section in FIG. 1 and an example of a processing flow in the embodiment.

FIG. 3 is a block diagram showing an example of a configuration of a brightness value similar region forming unit in FIG. 2 and an example of a processing flow in the embodiment.

FIG. 4 is a block diagram showing an example of a configuration of a horizontal line unit binarizing unit in the luminance value similar region forming unit in FIG. 3 and an example of a processing flow.

5 is a block diagram showing an example of a configuration of a vertical line unit binarizing unit in the luminance value similar region forming unit in FIG. 3 and an example of a processing flow.

FIG. 6 is a block diagram showing an example of a configuration of a binarization result integrating section of the luminance value similar area forming section in FIG. 3 and an example of a processing flow.

FIG. 7 is a flowchart showing an example of processing contents of a horizontal maximum brightness value area detection unit and a high brightness character area extraction unit in the horizontal line unit binarization unit in FIG.

8 is a flowchart showing an example of processing contents of a horizontal minimum brightness value area detection unit and a low brightness character area extraction unit in the horizontal line unit binarization unit in FIG.

9 is a flowchart showing an example of processing contents of a character connected component determination unit in the binarization result integration unit in FIG.

FIG. 10 is a diagram showing an example of a character area range on a line determined according to the present invention.

FIG. 11 is a diagram showing an example of a binarization result according to the present invention.

FIG. 12 is a diagram illustrating details of an example of the binarization result of FIG. 11.

FIG. 13 is a diagram showing an example of color deterioration of characters in an NTSC format video.

14A and 14B are an example 1 of the binarization result by the conventional method.

15 (a) and 15 (b) are an example 2 of the binarization result by the conventional method.

FIG. 16 is an example 3 of the binarization result by the conventional method.

[Explanation of symbols]

1 ... Color image input storage unit 2 ... Luminance similar region forming unit 3 ... Saturation value similar region forming unit 4 ... Hue value similar region forming unit 5: Character area image storage unit 6 ... Processing control unit 7 ... Color image input storage unit 8 ... Multiple frame averaging unit 9 ... Processing control unit 10 ... Horizontal line unit binarization unit 11 ... Vertical line unit binarization unit 12 ... Binarization result integration unit 13 ... Processing control unit 14 ... Horizontal brightness maximum area detection unit 15 ... High-intensity character area extraction unit 16 ... Brightness minimum area detection unit on horizontal line 17 ... Low-intensity character area extraction unit 18 ... Binarization result integration unit for each horizontal line 19 ... Processing control unit 20 ... Edge detection unit 21 ... Edge removal section 22 ... Maximum brightness detection area on vertical line 23 ... High-intensity character area extraction unit 24 ... Minimum luminance area detection unit on vertical line 25 ... Low-intensity character area extraction unit 26 ... Binary result integration unit for each vertical line 27 ... Processing control unit 28 ... Edge detection unit 29 ... Edge removal section

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshiaki Sugimura 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Nihon Telegraph and Telephone Corporation (56) Reference JP 4-74283 (JP, A) Matsuo Kenichi, Michio Umeda, Extraction of Character Strings from Scene Image by Shading and Color Information, IEICE Technical Report, Japan, The Institute of Electronics, Information and Communication Engineers, December 22, 1993, Vol. 92, No. 434 (PRU92-121), P.P. 25-32 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T ^7/ 00-7/60 300 Z H04N 5/278 H04N 9/74 JISST file (JOIS)

Claims (11)

(57) [Claims] 1. A color image in which characters are displayed is input.
And the luminance value in the color image input by the first step
Of pixels that are similar to each other and are adjacent to each other in the image space.
And a first series having similar brightness values obtained by the second step.
In each region, the saturation value is similar to the connected region image, and the image
A plurality of pixels that are adjacent to each other in space are used as a second connection region.
And the luminance value and the saturation value obtained by the third step are similar to each other.
The hue value is similar in each region for the second connected region image.
Similarly, a plurality of adjacent pixels in the pixel space are connected to each other by a third connecting region.
Only the fourth step of forming a region and the third connected region pixel obtained by the fourth step
Fifth stage of leaving in the image and accumulating as a character area image
In the character area extraction method having, a color image in which characters are displayed is input and stored.
The first step is the same terrorism in color moving images such as TV broadcast images.
Characters such as a pop character are displayed consecutively in time.
Enter and store a number of color images or text
Input and memorize a single still color image where is displayed.
Steps 1-1 and multiple color images displaying the same characters are input
, The image corresponding to the position among the plurality of color images is displayed.
Average value that has the average value of the pixel values of the prime as the value of each pixel
Create a color image as an input color image and output it.
And a 1-2 step of that, realized by using, similar in luminance value of the color image, a second step of forming a plurality of adjacent pixels in the image space as a first coupling region, The luminance is binarized in each horizontal line in the color image, and the result is obtained as a binary image in step 2-1 and the luminance is binarized in each vertical line in the color image. , 2-2 stage for obtaining the result as a binary image, 2 obtained by the 2-1 stage and the 2-2 stage
Compare connected components in two binary images,
Connected components whose position difference is within a preset allowable range are determined to be character regions, and are left in the binary image, and connected components whose area, shape, position difference exceed the preset allowable range are And a second step of judging the background noise portion to be removed from the binary image, and implementing the same. 2. The character region extracting method according to claim 1 , wherein the luminance is binarized in each horizontal line in the color image,
A 2-1 step of obtaining the result as binary image, luminance-value-distribution smell of horizontal in line with in the color image
And find all the pixels that have the maximum value of luminance, and find each maximum
Left and right pixel by pixel on the horizontal line starting from the value pixel
To open both the left and right sides of the maximum pixel at the start point.
The brightness value is lower than the brightness value of the starting point by a preset threshold value
The first pixel is detected, and the detected two pixels on the left and right and the water
Multiple images that exist between these two pixels on a flat line
A step 2-1-1 of extracting the element as a connected pixel area and a selection method existing near the left and right end pixels of the connected pixel area obtained by the step 2-1-1 are selected by a predetermined method. A step 2-1-2 in which a pixel having a maximum absolute value of a horizontal luminance gradient is obtained from a plurality of pixels on each of the left and right sides, and a range having the pixels at both ends is extracted as a high-luminance character region; luminance-value-distribution smell in horizontal lines with in the color image
And find all pixels that have a minimum brightness value,
Left and right pixel by pixel on the horizontal line starting from the value pixel
To open the left and right sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
The first pixel is detected, and the detected two pixels on the left and right and the water
Multiple images that exist between these two pixels on a flat line
A first 2-1-3 step of extracting the element as a connected pixel regions, were selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by step said 2-1-3 A step 2-1-4 in which the pixel having the maximum absolute value of the horizontal luminance gradient is obtained from the plurality of pixels on each of the left and right sides, and the range having the pixels at both ends is extracted as the low-luminance character region; The high-intensity character area on each horizontal line extracted in the step 2-1-2 is collected to create a high-intensity character area image as the whole image, and in the step 2-1-4. Collecting the extracted low-intensity character areas on each horizontal line to create a low-intensity character area image for the entire image 2-1-5
And the step 2-1 for detecting the pixels at the edge portion in the color image.
6, the high-intensity character area image obtained in the 2-1-5th step,
And a second pixel for removing a pixel corresponding to a vertical edge portion in the color image obtained in the step 2-1-6 from a plurality of pixels forming the character connected component in the low-luminance character area image. 1-7, and a character area extraction method characterized by realizing. 3. The character area extracting method according to claim 1 , wherein the luminance is binarized in each vertical line in the color image,
The 2-2 stage of obtaining a binary image results, the luminance value distribution odor vertical in line with in the color image
And find all the pixels that have the maximum value of luminance, and find each maximum
The value pixel as a starting point, lower the upper the vertical line pixel by pixel
To open both the upper and lower sides of the maximum pixel at the start point.
The brightness value is lower than the brightness value of the starting point by a preset threshold value
Detecting a first pixel, a plurality of picture that exists between those two pixels at the detected upper and lower two pixels and the vertical <br/> straight line on
The second step of extracting the element as the connected pixel area and the selection in the predetermined method existing near the left and right end pixels of the connected pixel area obtained in the second step A second 2-2-2 step in which a pixel having the maximum absolute value of the vertical luminance gradient is obtained from the plurality of pixels on each of the left and right sides, and a range having the pixels at both ends is extracted as a high-luminance character region; luminance-value-distribution smell of the vertical line with in the color image
And find all pixels that have a minimum brightness value,
The value pixel as a starting point, lower the upper the vertical line pixel by pixel
To open both the upper and lower sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
Detecting a first pixel, a plurality of picture that exists between those two pixels at the detected upper and lower two pixels and the vertical <br/> straight line on
A first 2-2-3 step of extracting the element as a connected pixel regions, were selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by step said 2-2-3 A step 2-2-4 in which a pixel having the maximum absolute value of the vertical luminance gradient is obtained from the plurality of pixels on each of the left and right sides, and a range having the pixels at both ends is extracted as a low-luminance character region; The high-intensity character area on each vertical line extracted in the step 2-2-2 is collected to create a high-intensity character area image as the entire image, and is also extracted in the step 2-2-4. The low brightness character areas on each vertical line are collected to create a low brightness character area image for the entire image, and the step 2-2-5, and the pixels of the edge portion in the color image are detected. 2-2-
6, the high-intensity character area image obtained in the 2-2-6 step,
And a second pixel for removing a pixel corresponding to an edge portion in the vertical direction in the color image obtained in the step 2-2-6 from the plurality of pixels forming the character connected component in the low-luminance character area image. 2-7, and a character area extraction method, which is characterized by: 4. The character area extracting method according to claim 1.
hand, The two sheets obtained by the steps 2-1 and 2-2
Area, shape and position are compared by comparing the connected components in the binary image.
Difference between connected components that are within the preset allowable range
It is judged as a character area and left in the binary image, and the area, shape and position
Connected component whose position difference exceeds the preset allowable range
The two are judged to be background noise parts and are removed from the binary image.
2-3 steps, The obtained two binary images are aligned and superposed,
If the two corresponding pixels both belong to the connected component,
The corresponding pixel is stored as a temporary character pixel, and the corresponding two
If either one of the pixels does not belong to the connected component,
The corresponding pixel is stored as a non-temporary character area, and the temporary character pixel is stored.
An image containing a connected component composed of is defined as an AND image
And the stage 2-3-1, which is created, The obtained two binary images are aligned and superposed,
One of the two corresponding pixels will be connected
If it belongs, the corresponding pixel is stored as a temporary character area and
If the two corresponding pixels do not belong to the connected component,
The corresponding pixel is stored as a non-temporary character area,
OR the images containing connected components Defined as an image
The stage 2-3-2 to create, The AND image obtained by the step 2-3-1 and the second
Align the positions of the OR images obtained in step 3-2
And the OR among the connected components in the AND image.
The number of pixels that overlap with the connected component in the image is preset
Concatenation in the OR image that is equal to or greater than the threshold value and has an overlap
If the area of the component is within the preset range, the AND image
The connected component in the image is judged as the character connected component and the AND
Leave it in the image, and the other connected components are
And a second step of removing from the AND image, Only the character connected component obtained by the step 2-3-3
The remaining pixels in the AND image are described as character area pixels.
Remember the 3-2-4 stage, A character area extraction method characterized by being realized by using. 5. The character region extraction method according to claim 1 , wherein the connected components in the two binary images obtained in the step 2-1 and the step 2-2 are compared to determine an area, Connected components in which the difference in shape and position is within the preset allowable range are judged to be character regions and are left in the binary image, and the difference in area, shape, and position exceeds the preset allowable range. The second to third steps of judging that they are background noise parts and removing them from the binary image, aligning the obtained two binary images with each other,
If the two corresponding pixels both belong to the connected component,
If the corresponding pixel is stored as a temporary character pixel and either one of the two corresponding pixels does not belong to the connected component,
A step 2-3-1 in which the corresponding pixel is stored as a non-temporary character region and an image including a connected component composed of the temporary character pixel is defined as an AND image; Align the two binary images of
When either one of the two corresponding pixels belongs to the connected component, the corresponding pixel is stored as a temporary character area, and when both of the two corresponding pixels do not belong to the connected component,
A step 2-3-2, in which the corresponding pixel is stored as a non-temporary character region, and an image including a connected component made up of temporary character pixels is defined as an OR image; AND image obtained by the step of
-3-2: The OR images obtained by the step of 3-2 are aligned and overlapped, and the OR image is extracted from the connected components in the AND image.
If the number of pixels overlapping the connected component in the image is equal to or greater than a preset threshold and the area of the connected component in the OR image having the overlap is within a preset range, the connected component in the AND image Is determined as a character connected component and the AND
Remaining in the image, the other connected components are removed as background noise regions from the AND image in the step 2-3-3, and only the character connected component obtained in the step 2-3-3 remains. In the AND image, among the connected pixels in each line obtained by binarization for each horizontal line obtained in the 2-1st step, the connected pixels whose number of connected pixels is less than a preset value are selected. And a step 3-2-4 in which a pixel in the AND image existing at the same position as the pixel is stored as a character area pixel, is used as a character area, and the character area extracting method is realized. 6. Inputting a color image in which characters are displayed
A color image input storage means for and storing the color image input by the color image input storing means
Of multiple images that have similar brightness values in the
Luminance value similar region forming hand that forms elements as connected regions
And the brightness value obtained by the brightness value similar region forming means are similar.
Saturation value is similar in each area to the connected area image
Then, a plurality of adjacent pixels in the image space are connected areas.
And a saturation value obtained by the saturation value similar region forming means.
For connected area images with similar values, the hue is
Connects multiple pixels that have similar values and are adjacent in pixel space.
Hue value similar region forming means to be formed as a region, and connected region pixels obtained by the hue value similar region forming means
Characters that are stored in the image as character area images
Area image storage means, the color image input storage means, the brightness value similar area forming means
Stage, the saturation value similar region forming means, and the hue value similar region
Processing control means for controlling the execution order of the forming means,
In the character area extraction device, the color image in which characters are displayed is input and stored.
The color image input and storage means can be used for the same terrorism in color moving images such as TV broadcast images.
Characters such as a pop character are displayed consecutively in time.
Enter and store a number of color images or text
Input and memorize a single still color image where is displayed.
Inputting a color moving image / still image input storage means and a plurality of color images in which the same characters are displayed
, The image corresponding to the position among the plurality of color images is displayed.
Average value that has the average value of the pixel values of the prime as the value of each pixel
Create color image and output as input color image
Multiple frame averaging means, color moving image / still image input storage means, and multiple frames
A brightness control unit for controlling the execution order of the averaging unit, which has similar brightness values in the color image and which forms a plurality of adjacent pixels in the image space as a connected region. Is a horizontal line unit binarizing means for binarizing the brightness in each horizontal line in the color image and obtaining the result as a binary image, and a binary value of the brightness in each vertical line in the color image. Vertical line unit binarizing means for performing binarization and obtaining the result as a binary image, and concatenation in the two binary images obtained by the horizontal line unit binarizing means and the vertical line unit binarizing means. The components are compared with each other, and the connected components whose differences in area, shape, and position are within the preset allowable range are determined to be character regions and left in the binary image, and the differences in area, shape, and position are preset. Connected components that exceed the allowable range do not A binarization result integrating unit that determines that the image is a blank portion and removes it from the binary image, the horizontal line unit binarizing unit, the vertical line unit binarizing unit, and the binarizing result integrating unit And a processing control means for controlling the character area extraction device. 7. The character area extracting device according to claim 6 , wherein the luminance is binarized in each horizontal line in the color image,
Result horizontal line unit binarizing means for obtaining a binary image, luminance-value-distribution smell of horizontal in line with in the color image
Te, obtains all lifting one pixel the maximum value of the luminance, the calculated maximum
Left and right pixel by pixel on the horizontal line starting from the value pixel
To open both the left and right sides of the maximum pixel at the start point.
The brightness value is lower than the brightness value of the starting point by a preset threshold value
The first pixel is detected, and the detected two pixels on the left and right and the water
Multiple images that exist between these two pixels on a flat line
A luminance maximum area detecting means for extracting the oxygen as a connected pixel regions, from the plurality of pixels selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by the luminance maxima region detection unit A high-brightness character area extracting means for obtaining a pixel having the maximum absolute value of the horizontal brightness gradient on each of the left and right sides, and extracting a range having the pixels at both ends as a high-brightness character area, and a horizontal line in the color image. luminance-value-distribution smell of the inner
And find all pixels that have a minimum brightness value,
Left and right pixel by pixel on the horizontal line starting from the value pixel
To open the left and right sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
The first pixel is detected, and the detected two pixels on the left and right and the water
Multiple images that exist between these two pixels on a flat line
A luminance minimum area detecting means for extracting the oxygen as a connected pixel regions, from the plurality of pixels selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by the luminance minimum area detecting means A low-brightness character area extracting unit that obtains a pixel having the maximum absolute value of the horizontal brightness gradient on each of the left and right sides and extracts a range having the pixels at both ends as a low-brightness character area, and the high-brightness character area extracting unit. The high-intensity character areas on each horizontal line extracted are collected to create a high-intensity character area image as the entire image, and the low-intensity character areas on each horizontal line extracted by the low-intensity character area extracting means are collected. A binarization result integrating means for each horizontal line that collects areas to create a low-brightness character area image for the entire image; a maximum-brightness area detecting means; and a high-brightness character area extracting means,
Processing control means for controlling the execution order of the minimum brightness area detection means, the low brightness character area extraction means, and the horizontal line-by-horizontal binarization result integration means, and edge detection for detecting pixels in the edge portion in the color image Means, and a color obtained by the edge detecting means from a plurality of pixels forming a character connected component in the high-intensity character area image and the low-intensity character area image obtained by the horizontal line-based binarization result integrating means. A character area extracting device, comprising: an edge portion removing unit that removes a pixel corresponding to an edge portion in a vertical direction in an image. 8. The character area extracting device according to claim 6 , wherein the luminance is binarized in each vertical line in the color image,
Results vertical line unit binarizing means for obtaining a binary image, luminance-value-distribution odor vertical in line with in the color image
And find all the pixels that have the maximum value of luminance, and find each maximum
Starting from the value pixel and moving up and down one pixel on the vertical line
To open both the upper and lower sides of the maximum pixel at the start point.
The brightness value is lower than the brightness value of the starting point by a preset threshold value
Detecting a first pixel, a plurality of picture that exists between those two pixels at the detected upper and lower two pixels and the vertical <br/> straight line on
A luminance maximum area detecting means for extracting the oxygen as a connected pixel regions, from the plurality of pixels selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by the luminance maxima region detection unit A pixel having the maximum absolute value of the brightness gradient in the vertical direction is obtained on each of the left and right sides, and a high-intensity character region extracting means for extracting a range having the pixels at both ends as a high-intensity character region, and a vertical line in the color image. luminance-value-distribution smell of the inner
And find all pixels that have a minimum brightness value,
Starting from the value pixel and moving up and down one pixel on the vertical line
To open both the upper and lower sides of the minimum pixel at the start point.
The brightness value is higher than the brightness value at the start point by a preset threshold value
Detecting a first pixel, a plurality of picture that exists between those two pixels at the detected upper and lower two pixels and the vertical <br/> straight line on
A luminance minimum area detecting means for extracting the oxygen as a connected pixel regions, from the plurality of pixels selected in a predetermined manner to present in the vicinity of both ends pixels of the left and right connected pixel regions obtained by the luminance minimum area detecting means A low-brightness character area extracting unit that obtains a pixel having the maximum absolute value of the brightness gradient in the vertical direction on each of the left and right sides, and extracts a range having the pixels at both ends as a low-brightness character area; The high-intensity character areas on each vertical line that have been extracted are collected to create a high-intensity character area image for the entire image, and the low-intensity character areas on each vertical line extracted by the low-intensity character area extracting means are collected. The vertical line-by-vertical line binarization result integrating means that collects areas to create a low-intensity character area image for the entire image; the maximum-intensity area detecting means; and the high-intensity character area extracting means,
Processing control means for controlling the execution order of the minimum brightness area detection means, the low brightness character area extraction means, and the vertical line-by-vertical binarization result integration means, and edge detection for detecting pixels in the edge portion in the color image Means, and a color obtained by the edge extracting means from a plurality of pixels constituting a character connected component in the high-intensity character area image and the low-intensity character area image obtained by the vertical line-by-vertical binarization result integrating means. A character area extracting device, comprising: an edge portion removing unit that removes a pixel corresponding to an edge portion in a vertical direction in an image. 9. The character area extracting device according to claim 6.
hand, Horizontal line unit binarization means and vertical line unit binarization means
Connected components in two binary images obtained by
However, the difference in area, shape, and position is within the preset allowable range.
It is determined that the connected components are character areas, and
Remaining, area, shape, position difference is preset tolerable range
The connected components that exceed the
The binarization result integrating means for removing from the value image is The obtained two binary images are aligned and superposed,
If the two corresponding pixels both belong to the connected component,
The corresponding pixel is stored as a temporary character pixel, and the corresponding two
If either one of the pixels does not belong to the connected component,
The corresponding pixel is stored as a non-temporary character area,
An image containing connected components composed of
AND image creation means created by The obtained two binary images are aligned and superposed,
One of the two corresponding pixels will be connected
If it belongs, the corresponding pixel is stored as a temporary character area and
If the two corresponding pixels do not belong to the connected component,
The corresponding pixel is stored as a non-temporary character area,
An image containing connected components composed of
OR image creating means to create, The AND image obtained by the AND image creating means and the O
Align the positions of the OR images obtained by the R image creating means
And the OR among the connected components in the AND image.
The number of pixels that overlap with the connected component in the image is preset
Above threshold , And concatenation in the OR image with overlap
If the area of the component is within the preset range, the AND image
The connected component in the image is determined as the character connected component, and the AND
Leave it in the image, and the other connected components are
Character connected component determining means for removing from the AND image
When, It is a character connected component obtained by the character connected component determination means.
Pixels in the AND image with injuries remain as character area pixels
Character pixel registration means to store, A character area extraction device comprising: 10. The character region extracting apparatus according to claim 6 , wherein the connected components in the two binary images obtained by the horizontal line unit binarizing unit and the vertical line unit binarizing unit are compared with each other, The connected components whose difference in area, shape, and position are within the preset allowable range are determined to be character regions and are left in the binary image, and the difference in area, shape, and position exceeds the preset allowable range. The binarization result integrating means that determines that the connected components are background noise parts and removes them from the binary image, superimposes the obtained two binary images at the same position,
If the two corresponding pixels both belong to the connected component,
If the corresponding pixel is stored as a temporary character pixel and either one of the two corresponding pixels does not belong to the connected component,
An AND image creating unit that stores the corresponding pixel as a non-temporary character region and defines an image including a connected component composed of the temporary character pixel as an AND image, and the obtained two binary images. Align and stack
When either one of the two corresponding pixels belongs to the connected component, the corresponding pixel is stored as a temporary character area, and when both of the two corresponding pixels do not belong to the connected component,
An OR image creating unit that creates the image by storing the corresponding pixel as a non-temporary character region and defining an image including a connected component composed of the temporary character pixel as an OR image, and an AND image obtained by the AND image creating unit And the O
The OR images obtained by the R image creating means are aligned with each other and overlapped, and the OR image is extracted from the connected components in the AND image.
If the number of pixels overlapping the connected component in the image is equal to or greater than a preset threshold and the area of the connected component in the OR image having the overlap is within a preset range, the connected component in the AND image Is determined as the character connected component, and the AND
In the AND image in which only the character connected components remaining in the image and the other connected components removed from the AND image as background noise areas are removed, and the character connected component obtained by the character connected component determination means, Of the connected pixels in each line obtained by the binarization for each horizontal line obtained by the horizontal line unit binarization means, the connected pixels whose number of connected pixels is less than a preset value are set as the character area, and A character area extracting device comprising: a character pixel registration unit that stores a pixel in the AND image existing at the same position as the pixel as a character area pixel. 11. wherein a program for executing steps of the computer in the character region extracting process according to any one of claims 1 to 5, the computer is recorded in a recording medium readable It shall be the Symbol <br/> recording media.