JP2960366B2 - Target extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for extracting only a target from an image with a high probability by using a plurality of images having different wavelength bands. More specifically, the present invention relates to a method of extracting an artificial object such as a vehicle from a natural object as a background.Since a difference in spectral characteristics is used instead of a difference in shape, a minute shape that cannot be recognized is used. It is an excellent method for extracting a target (artifact). In addition, since a target candidate is selected using a plurality of images having different wavelength bands, and the target and the clutter are discriminated with respect to the selected pixel, a high-speed and high-resolution target extraction is required. It is an effective method for the field.

[0002]

2. Description of the Related Art Target extraction techniques for vehicles and the like mainly include 3 to
It is performed using a monochrome image of a single wavelength in the mid-infrared region, which is a wavelength band of 5 μm or 4 to 5 μm, and in the far-infrared region, which is a wavelength band of 8 to 13 μm. The mainstream of this technique is a spatial filter that emphasizes only the target, a threshold value determination method for performing binarization processing, and the like. However, especially in the case of a ground target, the background has a complicated shape, so that it includes clutter having the same size and the same luminance as the target, and there has been a problem that not only the target but also the clutter is extracted by the conventional method.

In the field of remote sensing, LA
NDSAT (Earth observation satellite launched by NASA in the United States)
Research has been carried out to discriminate plants from soil, etc. by performing ratio arithmetic processing and the like on the data of LANDSAT.
MSS (Multispectral Scanne)
Abbreviation of r. It has been reported that an iron oxide band was extracted from a ratio calculation image using data from optical sensors mounted on LANDSAT Nos. 1 to 5 in Gold field, Nevada, USA (Remote sensing practical series for resource exploration separate terminology) Dictionary, published on March 31, 1989,
Edited and published by the Resource Observation and Analysis Center). But,
An effective means of extracting a target having almost the same color as the background has not been clarified yet.

In general, extraction performed in the field of remote sensing targets a wide area on the ground, so that only a point target from an image area having a wide and complex texture such as extraction of a minute target under a complex background is extracted. There has been no effective method for extracting a high probability.

Also, image analysis and target extraction using images in a plurality of wavelength bands has a problem in that an enormous processing time is required.

[0006]

When a target is extracted under a complicated background, there is a problem that not only the target but also the background clutter is extracted by the conventional processing using only a single wavelength monochrome image. .

[0007] Therefore, there are cases where the number of target candidates to be extracted increases, and there is a problem that the load on a person to search for a true target from the large number of extracted target candidates increases.

Further, there is no effective means for extracting a target painted in the same color as the background with high probability due to camouflage.

[0009] There is no effective means for extracting only one pixel level point target with high probability from a complicated image having many textures.

Heretofore, there has been a problem that the enormous processing time required for target extraction must be reduced.

An object of the present invention is to provide a wavelength band for suppressing clutter in a complex background using images captured in a plurality of wavelength bands and extracting only a target with a high probability.
The problem of the prior art is solved by providing a signal processing method for utilizing the wavelength band.

Another object of the present invention is to provide a wavelength band for extracting only a target painted in the same color as the background due to camouflage, and a signal processing method for utilizing the wavelength band.

It is another object of the present invention to provide a high-speed and accurate target extraction signal processing method.

[0014]

In order to achieve the above object, a target extraction method of the present invention performs a ratio operation between two or more image data and calculates a luminance histogram of the ratio operation image. By using a 1.6 μm band image and a 2.1 μm band image, it is possible to discriminate a target from a background.

Further, according to the present invention, a 1.6 μm band image and a 2.3 μm band image are used in a method of performing a ratio operation process between two or more image data and calculating a luminance histogram of the ratio operation image. This makes it possible to discriminate between the goal and the background.

According to the present invention, a target and a clutter are discriminated after extracting a portion having the same color as a target from a visible image (hereinafter, referred to as a visible image) which is an image in a visible region of 0.4 to 0.7 μm. For this purpose, a ratio calculation process using a 1.6 μm band image and a 2.1 μm band image or a ratio calculation process using a 1.6 μm band image and a 2.3 μm band image is combined.

Further, the present invention relates to a method for producing a visible image of 0.7 to 1
Near-infrared image, which is an image in the near-infrared region of μm
Target images using near-infrared images),
Ratio calculation using 1.6 μm band image and 2.1 μm band image or 1.6 for discriminating target and clutter
This is a combination of a μm band image and a ratio calculation process using a 2.3 μm band image.

According to the present invention, 3 to 5 μm or 4 to 5 μm
After extracting a target candidate using a mid-infrared image (hereinafter, referred to as a mid-infrared image) which is an image in the m-infrared region, a 1.6 μm band image and a 2.1 μm band are used to distinguish a target from clutter. Calculation processing using the image of 1.6 or 1.6
This is a combination of a μm band image and a ratio calculation process using a 2.3 μm band image.

Further, the present invention extracts a target candidate using a far-infrared image (hereinafter, referred to as a far-infrared image) which is an image in a far-infrared region of 8 to 13 μm, and discriminates the target from clutter. This is a combination of a ratio calculation process using a 1.6 μm band image and a 2.1 μm band image or a ratio calculation process using a 1.6 μm band image and a 2.3 μm band image.

According to the present invention, a part having the same color as a target is extracted from a visible image, and a target candidate is narrowed down using a mid-infrared image.
This is a combination of a ratio calculation process using a 6 μm band image and a 2.1 μm band image or a ratio calculation process using a 1.6 μm band image and a 2.3 μm band image.

Further, the present invention extracts a portion having the same color as the target from the visible image, further narrows down the target candidates using the far-infrared image, and then discriminates the clutter from the target in the 1.6 μm band. This is a combination of a ratio calculation process using an image and a 2.1 μm band image or a ratio calculation process using a 1.6 μm band image and a 2.3 μm band image.

According to the present invention, a target candidate is extracted using a visible image and a near-infrared image, and after narrowing down the target candidates using a mid-infrared image, an image in the 1.6 μm band is used to distinguish the target from clutter. This is a combination of a ratio calculation process using an image in the 2.1 μm band and a ratio calculation process using an image in the 1.6 μm band and an image in the 2.3 μm band.

Further, the present invention extracts a target candidate using a visible image and a near-infrared image, further narrows down the target candidate using a far-infrared image, and then discriminates the target from clutter in a 1.6 μm band. Calculation processing using the image of the 2.1 μm band and the image of the 1.6 μm band and the image of the 2.3 μm band.
This is a combination of ratio calculation processing using m-band images.

According to the present invention, after a target candidate is extracted using a mid-infrared image and a far-infrared image, a ratio calculation process is performed using a 1.6 μm band image and a 2.1 μm band image to discriminate a target from clutter. Alternatively, a 1.6 μm band image and 2.3
This is a combination of ratio calculation processing using an image in the μm band.

Further, the present invention extracts a portion having the same color as a target from a visible image, further extracts a target candidate using a mid-infrared image and a far-infrared image, and then discriminates the target from clutter. Ratio arithmetic processing using a 6 μm band image and a 2.1 μm band image, or a 1.6 μm band image and 2.
This is a combination of ratio calculation processing using an image in the 3 μm band.

According to the present invention, a target candidate is extracted by using a visible image and a near-infrared image, and a target candidate is extracted by using a mid-infrared image and a far-infrared image. This is a combination of a ratio calculation process using a 6 μm band image and a 2.1 μm band image or a ratio calculation process using a 1.6 μm band image and a 2.3 μm band image.

According to the present invention, the invisible region of 1.6 μm
After acquiring the image of the band and the image of 2.1 μm, by taking the ratio between the images, it is easy to detect only the relative difference, and it is hard to be affected by the weather condition and the shape of the material, and the like. A luminance histogram is calculated for the ratio calculation image, and a binarization process is performed using the difference between the luminance distribution of the target and the luminance distribution of the background to discriminate the target from the background.

Further, the present invention provides the following:
After acquiring the image of the 6 μm band and the image of the 2.3 μm band, and taking the ratio between the images, it is easy to detect only the relative difference so as to make it less affected by the weather conditions and the shape of the substance. Then, a histogram of the luminance is calculated for the ratio operation image, and a binarization process is performed using the difference between the luminance distribution of the target and the luminance distribution of the background to discriminate the target from the background.

According to the present invention, the hue, luminance,
Pixels of the same color as the target are extracted using the saturation information, and discrimination of the target or the clutter of the background is made from the extracted pixels 1.6.
The ratio extraction processing using the image in the μm band and the image in the 2.1 μm band, or the ratio operation processing using the image in the 1.6 μm band and the image in the 2.3 μm band is combined to improve the target extraction ability. At this time, after the target candidate is selected using the visible image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction processing can be increased.

Further, the present invention provides a method for controlling the size of 0.4
Blue region (hereinafter referred to as B) in a wavelength band of μm to 0.49 μm, Green region (hereinafter referred to as G) in a wavelength band of 0.49 μm to 0.57 μm, and 0.5.
Red region (hereinafter, referred to as a wavelength band of 7 μm to 0.7 μm)
Called R. ) And the ratio calculation image of the near-infrared image to extract pixels having the same special color as the target, and to determine whether the target or the background is clutter from the extracted pixels.
The ratio extraction process using the m-band image or the ratio calculation process using the 1.6 μm band image and the 2.3 μm band image is combined to improve the target extraction ability. At this time, after selecting a target candidate using the visible image and the near-infrared image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction processing can be increased.

According to the present invention, a pixel having the same special color as a target is extracted from a mid-infrared image, and a 1.6 μm band image and a 2.1 μm band image are discriminated from the extracted pixels to determine whether the target or background is clutter. Ratio operation using the image of
By combining the ratio calculation processing using the image in the 6 μm band and the image in the 2.3 μm band, the target extraction ability is improved. At this time, after the target candidate is selected using the mid-infrared image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased.

Further, the present invention extracts pixels having the same special color as the target from the far-infrared image, and distinguishes between the 1.6 μm band image and the 2.1 μm band from the extracted pixels to determine whether the target or the background is clutter. The ratio extraction processing using the band image or the ratio calculation processing using the 1.6 μm band image and the 2.3 μm band image is improved to improve the target extraction ability. At this time, since the target candidate is selected using the far-infrared image, discrimination between the target pixel and the clutter is performed only on the selected pixel, so that the speed of the target extraction processing can be increased.

According to the present invention, the hue, luminance,
A pixel having the same color as the target is extracted using the saturation information, and a pixel having the same special color as the target is extracted from the mid-infrared image. Ratio calculation processing using a 1.6 μm band image and a 2.1 μm band image, or a 1.6 μm band image and 2.3 μm
The target extraction ability is improved by combining the ratio calculation processing using the band image. At this time, after selecting a target candidate using the visible image and the mid-infrared image, discrimination as to whether the target is a target or a clutter is performed only on the selected pixel, so that the speed of the target extraction processing can be increased.

The present invention also provides a hue of a visible image,
Extract the pixels of the same color as the target using the information of luminance and saturation,
Further, a pixel having the same special color as the target is extracted from the far-infrared image, and a ratio using the 1.6 μm band image and the 2.1 μm band image is discriminated from the extracted pixels as to whether the target or the background is clutter. Operation processing, or 1.6 μm band image and 2.
The target extraction ability is improved by combining the ratio calculation processing using the image in the 3 μm band. At this time, after the target candidate is selected using the visible image and the far-infrared image, discrimination between the target pixel and the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased.

According to the present invention, a pixel having the same special color as the target is extracted from a ratio operation image of R, G, B and the near infrared image in the visible image, and a pixel having the same special color as the target is further extracted from the middle infrared image. Is extracted, and ratio calculation processing using a 1.6 μm band image and a 2.1 μm band image is performed to discriminate a target or a background clutter from the extracted pixels, or a 1.6 μm band image and 2.3 μm The target extraction ability is improved by combining the ratio calculation processing using the band image. At this time, after selecting the target candidate using the visible image, the near-infrared image, and the mid-infrared image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased. it can.

Further, the present invention provides a method for controlling R,
A pixel having the same special color as the target is extracted from the ratio calculation image of G, B and the near-infrared image, and a pixel having the same special color as the target is extracted from the far-infrared image. 1.6μm band image and 2.
Ratio arithmetic processing using an image in the 1 μm band, or 1.6 μm
By combining the ratio calculation processing using the m-band image and the 2.3 μm-band image, the target extraction ability is improved.
At this time, after selecting the target candidate using the visible image, the near-infrared image, and the far-infrared image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased. it can.

According to the present invention, a pixel having the same special color as the target in the mid-infrared image and a pixel having the same special color as the target in the far-infrared image are extracted, and the target or background clutter is extracted from the extracted pixels. 1.6μm band image and 2.1μ discrimination
The ratio extraction process using the m-band image or the ratio calculation process using the 1.6 μm band image and the 2.3 μm band image is combined to improve the target extraction ability. At this time, since the target candidate is selected using the mid-infrared image and the far-infrared image, discrimination of the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased.

Further, the present invention provides a method for adjusting the hue of a visible image,
Extract the pixels of the same color as the target using the information of luminance and saturation,
Further, a pixel having the same special color as the target in the mid-infrared image and a pixel having the same special color as the target in the far-infrared image are extracted, and one of the extracted pixels is discriminated as a target or a background clutter. Ratio calculation processing using an image of a 1.6 μm band and an image of a 2.1 μm band, or an image of a 1.6 μm band and an image of 2.3 μm
The target extraction ability is improved by combining the ratio calculation processing using the m-band image. At this time, after selecting the target candidate using the visible image, the mid-infrared image, and the far-infrared image, discrimination as to the target or the clutter is performed only on the selected pixel, so that the speed of the target extraction process can be increased. it can.

According to the present invention, a pixel having the same special color as the target is extracted from a ratio calculation image of R, G, B and the near infrared image in the visible image, and has the same special color as the target in the middle infrared image. Pixels and pixels having the same special color as the target in the far-infrared image are extracted, and a 1.6 μm band image and a 2.1 μm band image are used to discriminate between the target pixel and the background clutter from the extracted pixels. By combining the ratio calculation processing or the ratio calculation processing using the image in the 1.6 μm band and the image in the 2.3 μm band, the target extraction capability is improved. At this time, after selecting target candidates using the visible image, near-infrared image, mid-infrared image, and far-infrared image, the target extraction process is accelerated because discrimination between the selected pixel and target or clutter is performed. Can be achieved.

[0040]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a 1.2-2.5 μm band image in which a vehicle with OD color painting, a sunny plant, and a shaded plant are visible on the same screen. It shows a histogram of a leaf in a certain sunshine and shade. The horizontal axis indicates luminance and the vertical axis indicates frequency. Reference numeral 1 denotes a histogram of OD color paint used in a vehicle or the like, 2 denotes a leaf in the shade, and 3 denotes a histogram of a leaf in the sun. From now on 1.
By using the 2-2.5 μm band image, it is possible to discriminate between the OD color paint used in a vehicle or the like and the sun leaf, but the shade leaf is extracted together with the target.

FIG. 2 shows 1.6 μm in a state where a vehicle painted with OD color, a plant in the sun, and a plant in the shade are visible on the same screen.
In the ratio calculation image obtained by taking the ratio of the image of the band to the image of the 2.1 μm band, the histogram of the OD color painting of the target vehicle and the like and the background of the sun and the leaves in the shade are shown. The horizontal axis indicates the normalized luminance, and the vertical axis indicates the frequency. 4 is a histogram of the OD color painting used in the vehicle or the like in the ratio calculation image, 5 is a histogram of a leaf in the shade in the ratio calculation image, and 6 is a histogram of a leaf in the sun in the ratio calculation image. I have. Taking the luminance ratio between a plurality of images having different wavelength bands has the effect of reducing the difference in how the sun hits due to the shape and eliminating the influence of the intensity of sunlight. By calculating the ratio calculation image using the wavelengths of the 1.6 μm band and the 2.1 μm band, variations in the luminance distribution having leaf characteristics are suppressed, and the OD used in vehicles and the like is reduced.
Color painting and leaves can be distinguished.

FIG. 3 shows 1.6 μm in a state where the vehicle of the OD color painting, the plants in the sun, and the plants in the shade are visible on the same screen.
5 is a flowchart illustrating a process for extracting an OD color vehicle using a band image and a 2.1 μm band image. 1.6μ
After inputting the 1.6 μm band image and the 2.1 μm band image in the m band image input step a and the 2.1 μm band image input step b, the 1.6 μm band image and 2. Align each pixel of the 1μm band image,
In the ratio calculation processing step d, the alignment is completed 1.6 μm
A ratio calculation image is calculated by taking the ratio between each pixel of the m band and the 2.1 μm band image, and a histogram calculation step e
Calculates the histogram of the ratio calculation image of the 1.6 μm band and the 2.1 μm band, and in the target extraction processing step f, determines the threshold value of the binarization processing of the vehicle and the plant from the histogram to determine the threshold value of the vehicle and the plant. Perform discrimination.

FIG. 4 is a diagram for explaining the image matching process. When an operation is performed between n images from the image G _1a 4 to the image G _na 6, each pixel constituting the image in each image is the same. Need to look at the object. Target point Aa1,
Assuming the target point Ba2 and the target point Ca3, it is necessary to match the coordinates of the target point in each image. For this reason, in the image matching process, the coordinates of all the pixels are converted so that the coordinates of each target point in the images G _1a 4 to G _na 6 match, and the results are respectively calculated as the calculation image G _1a 7 and the calculation image It is assumed that G _2a 8 and the calculation image G _na 9. Since the images here are two kinds of images of the 1.6 μm band and the 2.1 μm band, the coordinates of each pixel are matched between the two images, and the calculation image of the 1.6 μm band and the calculation image of the 2.1 μm band Create a business image. The subsequent calculations are performed between the calculation images created by the image matching process.

Embodiment 2 Fig. 5 shows a vehicle with OD color painting.
OD color painting and background of target vehicle, etc., in a ratio calculation image obtained by taking a ratio of 1.6 μm band and 2.3 μm band image in a state where the sunlit plants and the shaded plants are visible on the same screen. 7 shows a histogram of the sun and the leaves in the shade. The horizontal axis indicates the normalized luminance, and the vertical axis indicates the frequency. 7 is an OD used in a vehicle or the like in the ratio calculation image
8 is a histogram of shaded leaves in the ratio calculation image, and 9 is a histogram of sunlit leaves in the ratio calculation image. Taking the luminance ratio between a plurality of images having different wavelength bands has the effect of reducing the difference in how the sun hits due to the shape and eliminating the influence of the intensity of sunlight. This combination is 1.6 μm and 1.6 μm as in the case of the ratio calculation image of the 2.1 μm band.
By calculating the ratio calculation image using the wavelength of the band and the wavelength of the 2.3 μm band, it is possible to suppress the variation in the luminance distribution having the characteristics of the leaves, and to discriminate the leaves from the OD color coating used in vehicles and the like.

FIG. 6 shows 1.6 μm in a state in which the vehicle of the OD color painting, the plant in the sun, and the plant in the shade are visible on the same screen.
5 is a flowchart illustrating a process for extracting an OD color vehicle using a band image and a 2.3 μm band image. 1.6μ
After inputting the 1.6 μm band image and the 2.3 μm band image in the m-band image input step a and the 2.3 μm band image input step g, the 1.6 μm band image and the 2. μm band image are input in the image matching processing step c. Align each pixel of 3μm band image,
In the ratio calculation processing step d, the alignment is completed 1.6 μm
A ratio calculation image is calculated by taking the ratio between each pixel of the m band image and the 2.3 μm band image, and a histogram calculation step e is performed.
Calculates the histogram of the ratio operation image of the 1.6 μm band and the 2.3 μm band, and in step f of the target extraction processing, determines the threshold value of the binarization processing of the vehicle and the plant from the histogram to determine the threshold of the vehicle and the plant. Perform discrimination. In the image matching processing step c, the target image in FIG.
For two types of images, the m band and the 2.3 μm band, the coordinates of each pixel are matched between the two images, and a 1.6 μm calculation image and a 2.3 μm calculation image are created. The subsequent calculations are performed between the calculation images created by the image matching process.

Embodiment 3 Fig. 7 shows a vehicle with OD color painting.
To extract an OD-colored vehicle using a visible image, a 1.6 μm band image, and a 2.1 μm band image in an outdoor situation where a sunny plant, a shade plant, and soil are visible in the same space. It shows a flowchart. In the input step h of the visible image, the input step a of the 1.6 μm band image, and the input step b of the 2.1 μm band image, the visible image is changed to 1.6.
After inputting the μm band image and the 2.1 μm band image, the visible image, the 1.6 μm band image and the 2.1 μm band image are aligned in the image matching processing step c, and the visible image data conversion step i is performed. The hue, luminance, and saturation of each pixel are calculated using the R, G, and B images of the input visible image.
A pixel having a D color hue is extracted. Since the extracted pixels include not only the target vehicle but also clutters such as plants, the ratio calculation processing step d is performed on the extracted pixels.
Calculates the normalized ratio operation image by taking the ratio between each pixel of the 1.6 μm band image and the 2.1 μm band image, and calculates the histogram of the 1.6 μm band and 2.1 μm band ratio operation image in the histogram calculation step e. Is calculated, and in the target extraction processing step f, the threshold value of the binarization processing of the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. After extracting the color region considered to be the target from the visible image, the vehicle and the plant are discriminated using the images of the 1.6 μm band and the 2.1 μm band. First, extract the pixels that are likely to be targets in the image,
After that, since the vehicle and the plant are discriminated only for that pixel, the number of pixels to be processed is reduced and the target extraction processing can be performed at high speed. In the image matching processing step c, the target image in FIG.
For the three types of images in the 6 μm band and the 2.1 μm band, the coordinates of each pixel are matched between the three images, and the visible calculation image and 1.
A calculation image in the 6 μm band and a calculation image in the 2.1 μm band are created. The subsequent calculations are performed between the calculation images created by the image matching process. In the flowchart of FIG. 7, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

In the visible image data conversion step i in FIG. 7, the visible image is decomposed into an R image, a G image, and a B image, and the hue, saturation, and brightness are calculated using the luminance of each pixel. Assuming that the luminance of each of the R pixel, G pixel, and B pixel is R, G, and B, the hue H, the saturation C, and the lightness Y are obtained as in Equations 1 to 3, respectively. FIG. 8 shows the relationship between hue, saturation, and lightness. The vertical axis of the cylindrical coordinate system is lightness Yb1, the horizontal axis is saturation Cb2,
The rotation direction in the same plane as the horizontal axis around the intersection of the vertical axis and the horizontal axis indicates the hue Hb3. FIG. 9 shows the relationship between the take-out angle and the color of only the hue Hb3 in FIG.

[0048]

(Equation 1)

[0049]

(Equation 2)

[0050]

(Equation 3)

FIG. 10 shows an example of the extraction step j of the OD color area in FIG. The OD color area extraction step j utilizes the fact that each of the visible pixels has hue information from the visible image data conversion step i,
By extracting the pixels having the hue in the vicinity of the green region C ₁ that may contain OD color, and an object thereof is to narrow down the region to implement the target extraction process.

Embodiment 4 FIG. FIG. 11 shows a visible image, a near-infrared image, a 1.6 μm band image, and a 2.1 μm band in an outdoor situation where an OD color painted vehicle, a sunny plant, a shaded plant, soil, and the like are visible in the same space. OD using image
5 is a flowchart illustrating a process for extracting a vehicle having a color. 1. A visible image, a near-infrared image, a 1.6 μm band image and 2. a visible image input step h, a near infrared image input step k, a 1.6 μm band image input step a, and a 2.1 μm band image input step b. After the input of the 1 μm band image, the visible image, the near-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned in the image alignment processing step c, and the alignment is performed in the ratio calculation processing step l. From the visible image and the near-infrared image composed of R, G, and B, the ratio of the near-infrared image to the B image, the ratio of the R image to the B image, and the ratio of the G image to the B image are calculated. After the normalization, the calculation result is plotted in a target candidate extraction space in a target candidate extraction processing step m, and OD
A threshold for extracting a target candidate which is a color painting region is determined, and the target candidate is extracted. The ratio calculation processing step d calculates the normalized ratio calculation image by taking the ratio between each pixel of the 1.6 μm band image and the 2.1 μm band image in the ratio calculation processing step d, and 1.6 μm in the histogram calculation step e. The histogram of the ratio calculation image of the band and the 2.1 μm band is calculated, and in the target extraction processing f, the threshold value of the binarization processing of the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. After extracting a target candidate using information of a near-infrared image in addition to a visible image, discriminating between a vehicle and a plant using images in the 1.6 μm band and the 2.1 μm band, thereby increasing the target extraction capability. Since it is possible to extract pixels with high possibility of the target from the visible image and the near-infrared image first, and then discriminate between vehicles and plants only on those pixels,
The number of pixels to be processed is reduced, and the speed of the target extraction processing can be increased. Note that the image matching processing step c
In FIG. 4, since the target image is a near-infrared image, a visible image, and four types of images of 1.6 μm band and 2.1 μm band, the coordinates of each pixel are matched between the four images, An image for calculation, a visible calculation image, a 1.6 μm band calculation image, and a 2.1 μm band calculation image are created. The subsequent calculations are performed between the calculation images created by the image matching process. Further, in the flowchart of FIG.
It is also possible to use a 2.3 μm band image instead of a μm band image.

FIG. 12 shows an example of the target candidate extraction processing step m in FIG. The target candidate extraction processing step m includes the standard value of the ratio of the G image and the B image calculated in the ratio calculation processing step l, the standard value of the ratio of the R image and the B image, and the near infrared image and the B image. Target candidate extraction space c where the standard values of the ratios are X-axis, Y-axis, and Z-axis, respectively.
5, and the calculation results of each pixel are plotted in this space. The plant area c2, the soil area c3, and the OD color painting area c4
The purpose of the present invention is to narrow down the area in which the target extraction processing is performed by extracting pixels in the OD color painting area c4.

Embodiment 5 FIG. FIG. 13 shows a mid-infrared image and 1.6 in an outdoor situation where vehicles with OD color painting, plants in the sun, plants in the shade, soil, and the like are visible in the same space.
5 is a flowchart illustrating a process for extracting an OD color vehicle using a μm band image and a 2.1 μm band image. After inputting the middle infrared image, the 1.6 μm band image, and the 2.1 μm band image in the input step n of the middle infrared image, the input step a of the 1.6 μm band image, and the input step b of the 2.1 μm band image, In the image alignment processing step c, the mid-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned, and in the target enhancement processing step o, the aligned mid-infrared image is subjected to a spatial filter or the like. A target emphasis process is performed, a histogram of the mid-infrared image subjected to the emphasis process is calculated in a histogram calculation step e, and a target candidate is extracted in a binarization process step p. Since the extracted pixels include not only the target vehicle but also clutter such as plants, the ratio of the extracted pixels to the 1.6 μm band and 2.1 μm band images in the ratio calculation processing step d. A ratio operation image normalized by taking the ratio between the pixels is calculated, a histogram of ratio operation images of 1.6 μm band and 2.1 μm band is calculated in a histogram calculation step e, and the histogram is calculated in a target extraction processing step f. , The threshold value of the binarization process between the vehicle and the plant is determined to discriminate the vehicle from the plant. After extracting target candidates using the information of the mid-infrared image, 1.6 μm
In order to carry out discrimination between a vehicle and a plant using images in the m band and the 2.1 μm band, it is possible to increase the target extraction capability, and first extract pixels having a high possibility of the target from the mid-infrared image, and then Since discrimination between the vehicle and the plant is performed only on the pixel, the number of pixels to be processed is reduced, and the target extraction process can be performed at high speed. In the image matching processing step c, since the target image in FIG. 4 is a mid-infrared image and three types of images in the 1.6 μm band and the 2.1 μm band, the coordinates of each pixel are matched between the three images. Mid-infrared calculation image and 1.6 μm band calculation image and 2.1 μm
A band calculation image is created. The subsequent calculations are performed between the calculation images created by the image matching process. Also,
In the flowchart of FIG. 13, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

FIG. 14 shows the principle of a median difference filter which is an example of the target emphasis processing step o in FIG. The target emphasis processing step o is processing for emphasizing a target such as a vehicle which is assumed to have high luminance in the mid-infrared image. For example, median difference filter processing, which is one of spatial filters, is performed on the mid-infrared image, This is a process of emphasizing the pixels with the highest luminance. The median difference filter is a process of taking a difference between an image obtained by applying a median filter d1 to an input image and an original image,
The median filter d1 is a process of replacing each pixel of the image with a median value of a local area centered on the pixel. For example, in the case of a 3 × 3 local area, this is an operation of arranging nine pixel values in ascending order and searching for the fifth value from the smallest.

Embodiment 6 FIG. FIG. 15 shows a far-infrared image and 1.6 in an outdoor situation where vehicles with OD color painting, plants in the sun, plants in the shade, soil, and the like are visible in the same space.
5 is a flowchart illustrating a process for extracting an OD color vehicle using a μm band image and a 2.1 μm band image. After inputting the far infrared image, the 1.6 μm image, and the 2.1 μm image in the input step q of the far infrared image, the input step a of the 1.6 μm image, and the input step b of the 2.1 μm image, In the image matching processing step c, the far infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned, and in the target enhancement processing step o, the aligned far infrared image is subjected to a spatial filter or the like. A target emphasis process is performed, a histogram of the far-infrared image subjected to the emphasis process is calculated in a histogram calculation step e, and a target candidate is extracted in a binarization process step p. Since the extracted pixels include not only the target vehicle but also clutter such as plants, the ratio of the extracted pixels to the 1.6 μm band and 2.1 μm band images in the ratio calculation processing step d. A ratio operation image normalized by taking the ratio between the pixels is calculated, a histogram of ratio operation images of 1.6 μm band and 2.1 μm band is calculated in a histogram calculation step e, and the histogram is calculated in a target extraction processing step f. , The threshold value of the binarization process between the vehicle and the plant is determined to discriminate the vehicle from the plant. After extracting a target candidate using the information of the far-infrared image, 1.6 μm
In order to carry out discrimination between the vehicle and the plant using the images of the m band and the 2.1 μm band, it is possible to increase the target extraction capability, and to first extract pixels having a high possibility of the target from the far infrared image, Since discrimination between the vehicle and the plant is performed only on the pixel, the number of pixels to be processed is reduced, and the target extraction process can be performed at high speed. In the image matching processing step c, since the target image in FIG. 4 is a far-infrared image and three types of images in the 1.6 μm band and the 2.1 μm band, the coordinates of each pixel are matched between the three images. Calculation image of far infrared rays, calculation image of 1.6 μm band, and 2.1 μm
A band calculation image is created. The subsequent calculations are performed between the calculation images created by the image matching process. In the target enhancement processing step o, the input image in FIG. 14 may be converted to a far-infrared image, and the processing content is the same as that described in the fifth embodiment. In the flowchart of FIG. 15, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

Embodiment 7 FIG. FIG. 16 shows a visible image, a mid-infrared image, a 1.6 μm band image, and a 2.1 μm band in an outdoor situation in which an OD color painted vehicle, a sunny plant, a shaded plant, soil, and the like are visible in the same space. OD using image
5 is a flowchart illustrating a process for extracting a vehicle having a color. 1. Input step h of visible image, input step n of mid-infrared image, input step a of 1.6 μm band image, input step b of 2.1 μm band image, visible image, mid-infrared image, 1.6 μm band image and 2. After inputting the 1 μm band image, the visible image, the mid-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned in the image matching processing step c, and the visible image data conversion step i is performed.
The hue, luminance, and saturation of each pixel are calculated using each of the R, G, and B images of the visible image input in step, and the pixel having the OD hue characteristic of the vehicle in the OD color area extraction step j Is extracted. Since the extracted pixels include not only the target vehicle but also clutters such as plants, target candidates are narrowed down next using the mid-infrared image. In the target emphasis processing step o, the mid-infrared image that has been aligned is subjected to target emphasis processing using a spatial filter or the like, and in the histogram calculation step e, a histogram of the emphasized mid-infrared image is calculated and binarized. At step p, target candidates are extracted. After the target candidate is extracted using the visible image and the mid-infrared image, the ratio between each pixel of the 1.6 μm band image and the 2.1 μm band image is calculated for the extracted pixels in a ratio calculation processing step d. A normalized ratio operation image is calculated, a histogram of the 1.6 μm band and a 2.1 μm band ratio operation image is calculated in a histogram calculation step e, and a binary of a vehicle and a plant is obtained from the histogram in a target extraction processing step f. The vehicle is distinguished from the plant by determining the threshold value of the conversion process. After extracting target candidates using the information of the mid-infrared image in addition to the visible image, discriminating between the vehicle and the plant using the images in the 1.6 μm band and the 2.1 μm band, thereby increasing the target extraction capability. In addition to extracting pixels that are likely to be targets from the visible image and the mid-infrared image, and then performing vehicle and plant discrimination only on those pixels, the number of pixels to be processed is reduced and target extraction is performed. Processing can be performed at high speed. Note that the image matching processing step c is performed in FIG.
In (4), since the target image is a visible image, a mid-infrared image, and four types of images in the 1.6 μm band and the 2.1 μm band, the coordinates of each pixel are matched between the four images, and the visible calculation image and the mid-infrared light are used. , A 1.6 μm band calculation image, and a 2.1 μm band calculation image. The subsequent calculations are performed between the calculation images created by the image matching process. The data conversion step i of the visible image and the extraction step j of the OD color area are the same as those described in FIG. 8, FIG. 9, FIG. 10, and Embodiment 3, and the target enhancement processing step o uses the input image of FIG. A mid-infrared image may be used, and the processing content is the same as the content described in the fifth embodiment. In the flowchart of FIG. 16, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

Embodiment 8 FIG. FIG. 17 shows a visible image, a far-infrared image, a 1.6 μm band image, and a 2.1 μm band in an outdoor situation in which an OD color painted vehicle, a sunny plant, a shaded plant or soil are visible in the same space. OD using image
5 is a flowchart illustrating a process for extracting a vehicle having a color. 1. The input step h of the visible image, the input step q of the far-infrared image, the input step a of the 1.6 μm band image, and the input step b of the 2.1 μm band image, the visible image, the far infrared image, the 1.6 μm band image, and 2. After inputting the 1 μm band image, the visible image, the far infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned in the image matching processing step c, and the visible image data conversion step i is performed.
The hue, luminance, and saturation of each pixel are calculated using each of the R, G, and B images of the visible image input in step, and the pixel having the OD hue characteristic of the vehicle in the OD color area extraction step j Is extracted. Since the extracted pixels include not only the target vehicle but also clutter such as plants, target candidates are narrowed down next using a far-infrared image. In the target enhancement processing step o, a target enhancement processing is performed on the aligned far-infrared image using a spatial filter or the like, and in the histogram calculation step e, a histogram of the enhanced far-infrared image is calculated and binarized. At step p, target candidates are extracted. After the target candidate is extracted using the visible image and the far-infrared image, the ratio between each pixel of the 1.6 μm band image and the 2.1 μm band image is calculated for the extracted pixel in a ratio calculation processing step d. A normalized ratio operation image is calculated, a histogram of the 1.6 μm band and a 2.1 μm band ratio operation image is calculated in a histogram calculation step e, and a binary of a vehicle and a plant is obtained from the histogram in a target extraction processing step f. The vehicle is distinguished from the plant by determining the threshold value of the conversion process. After extracting target candidates using the information of the far-infrared image in addition to the visible image, discrimination between the vehicle and the plant is performed using the images of the 1.6 μm band and the 2.1 μm band. In addition to extracting pixels with a high possibility of target from the visible image and far-infrared image, and then discriminating vehicles and plants only from those pixels, the number of pixels to be processed is reduced and target extraction Processing can be performed at high speed. Note that the image matching processing step c is performed in FIG.
In (4), since the target image is a visible image, a far-infrared image, and four types of images of 1.6 μm band and 2.1 μm band, the coordinates of each pixel are matched between the four images, and the visible calculation image and the far infrared ray , A 1.6 μm band calculation image, and a 2.1 μm band calculation image. The subsequent calculations are performed between the calculation images created by the image matching process. The data conversion step i of the visible image and the extraction step j of the OD color area are the same as those described in FIG. 8, FIG. 9, FIG. 10, and Embodiment 3, and the target enhancement processing step o uses the input image of FIG. What is necessary is just to make it into a far-infrared image, and the processing content is the same as the content explained in Embodiment 5. In the flowchart of FIG. 17, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

Embodiment 9 FIG. 18 shows a visible image, a near-infrared image, a mid-infrared image, and a 1.6 μm band image in an outdoor situation where an OD color painted vehicle, a sunny plant, a shaded plant, soil, and the like are visible in the same space. 5 is a flowchart for extracting an OD color vehicle using a 2.1 μm band image. Input step h of visible image, input step k of near-infrared image, input step n of mid-infrared image, input step a of 1.6 μm band image and 2.1 μm
After inputting a visible image, a near-infrared image, a mid-infrared image, a 1.6 μm band image, and a 2.1 μm band image in an m-band image input step b, a visible image and a near-infrared image are input in an image matching processing step c. The images of the mid-infrared image, the 1.6 μm band image and the 2.1 μm band image are aligned, and in the ratio calculation processing step l, the positions of the aligned R, G, and B visible images and the near-infrared image are compared. After standardizing the ratio of the near-infrared image to the B image, the ratio of the R image to the B image, and the ratio of the G image to the B image, the calculation result is extracted in the target candidate extraction processing step m. Plotting is performed in a space, a threshold for extracting a target candidate which is an OD color painting area in the space is determined, and the target candidate is extracted. Since the extracted pixels may include clutter such as plants in addition to the target vehicle, target candidates are then narrowed down using the mid-infrared image. Target enhancement processing step o
Then, a target enhancement process is performed on the aligned mid-infrared image using a spatial filter or the like, a histogram of the emphasized mid-infrared image is calculated in a histogram calculation step e, and the target is processed in a binarization processing step p. Extract candidates. After extracting a target candidate using the information of the visible image, the near-infrared image, and the mid-infrared image, the ratio calculation processing step d calculates the ratio between each pixel of the 1.6 μm band and the 2.1 μm band image with respect to the extracted pixels. , And a normalized ratio operation image is calculated, and in the histogram calculation step e, the 1.6 μm band and 2.
A histogram of the ratio calculation image in the 1 μm band is calculated, and in a target extraction processing step f, a threshold value for binarizing the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. Pixels with a high probability of being targeted are first extracted from the visible image, near-infrared image, and mid-infrared image, and then the vehicle and plants are discriminated only for those pixels. Can be made faster. Note that the image matching processing step c is performed between five images because the target image in FIG. 4 is a visible image, a near-infrared image, a mid-infrared image, and five images of 1.6 μm band and 2.1 μm band. The coordinates of the pixels are matched, and a visible calculation image, a near-infrared calculation image, a mid-infrared calculation image, a 1.6 μm band calculation image, and a 2.1 μm band calculation image are created. The subsequent calculations are performed between the calculation images created by the image matching process. The ratio calculation processing step 1 and the target candidate extraction processing step m are the same as those described in FIG. 12 and the fourth embodiment, and the target enhancement processing step o may be performed by converting the input image in FIG.
The processing content is the same as the content described in the fifth embodiment. In addition, in the flowchart of FIG.
It is also possible to use a 2.3 μm band image instead of the m band image.

Embodiment 10 FIG. FIG. 19 shows a visible image, a near-infrared image, a far-infrared image, and a 1.6 μm band image in an outdoor situation where an OD color painted vehicle, a sunny plant, a shaded plant, soil, and the like are visible in the same space. 5 is a flowchart for extracting an OD color vehicle using a 2.1 μm band image. Input step h of visible image, input step k of near-infrared image, input step q of far-infrared image, input step a of 1.6 μm band image and 2.1 μm
After inputting a visible image, a near-infrared image, a far-infrared image, a 1.6 μm band image, and a 2.1 μm band image in the m-band image input step b, the visible image and the near-infrared image are input in the image matching processing step c. The far-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned, and in the ratio calculation processing step l, the aligned R, G, and B visible image and the near-infrared image are used. After standardizing the ratio of the near-infrared image to the B image, the ratio of the R image to the B image, and the ratio of the G image to the B image, the calculation result is extracted in the target candidate extraction processing step m. Plotting is performed in a space, a threshold for extracting a target candidate which is an OD color painting area in the space is determined, and the target candidate is extracted. Since the extracted pixels may include clutter such as plants in addition to the target vehicle, the target candidates are narrowed down next using the far-infrared image. Target enhancement processing step o
Then, a target enhancement process is performed on the aligned far-infrared image using a spatial filter or the like, a histogram of the enhanced far-infrared image is calculated in a histogram calculation step e, and the target is processed in a binarization process step p. Extract candidates. After extracting a target candidate using the information of the visible image, the near-infrared image, and the far-infrared image, the ratio calculation processing step d calculates the ratio between each pixel of the 1.6 μm band and the 2.1 μm band image with respect to the extracted pixels. , And a normalized ratio operation image is calculated, and in the histogram calculation step e, the 1.6 μm band and 2.
A histogram of the ratio calculation image in the 1 μm band is calculated, and in a target extraction processing step f, a threshold value for binarizing the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. Pixels with a high probability of being targeted are first extracted from the visible image, near-infrared image, and far-infrared image, and then only those pixels are used to discriminate between vehicles and plants. Can be made faster. Note that the image matching processing step c is performed between five images because the target image in FIG. 4 is a visible image, a near-infrared image, a far-infrared image, and five types of 1.6 μm and 2.1 μm images. The coordinates of the pixels are matched, and a visible calculation image, a near infrared calculation image, a far infrared calculation image, a 1.6 μm calculation image, and a 2.1 μm calculation image are created. The subsequent calculations are performed between the calculation images created by the image matching process. The ratio calculation processing step 1 and the target candidate extraction processing step m are the same as those described in FIG. 12 and the fourth embodiment, and the target enhancement processing step o may be performed by converting the input image in FIG.
The processing content is the same as the content described in the fifth embodiment. Further, in the flowchart of FIG.
It is also possible to use a 2.3 μm band image instead of the m band image.

Embodiment 11 FIG. FIG. 20 shows a mid-infrared image, a far-infrared image, a 1.6-μm band image, and a 2.1-μm image in an outdoor situation where vehicles with OD color painting, plants in the sun, plants in the shade, soil, and the like are visible in the same space. 5 is a flowchart illustrating a process for extracting an OD color vehicle using a band image. The input step n for the mid-infrared image, the input step q for the far-infrared image, the input step a for the 1.6 μm band image, and the input step b for the 2.1 μm band image include the mid-infrared image, the far-infrared image, and the 1.6 μm band image. After inputting the 2.1 μm band image, in the image matching processing step c, the middle infrared image, the far infrared image, the 1.6 μm band image and the 2.1 μm image are obtained.
The image of the m-band image is aligned, and in the ratio calculation processing step r, the ratio between the aligned mid-infrared image and the far-infrared image is standardized, and the calculation result is used in the target candidate extraction processing step s. To extract target candidates. Since the extracted pixels may include clutter such as plants in addition to the target vehicle, the 1.6 μm band and the 2.1 μm band in the ratio calculation processing step d are applied to the extracted pixels.
A ratio calculation image is calculated by taking a ratio between each pixel of the band image, and a histogram of the 1.6 μm band and 2.1 μm band ratio calculation images is calculated in a histogram calculation step e, and a target extraction processing step is performed. In f, the threshold value of the binarization process between the vehicle and the plant is determined from the histogram to discriminate the vehicle from the plant. After extracting the target candidates using the information of the mid-infrared image and the far-infrared image, the target extraction capability should be increased to discriminate the vehicle and the plant using the images of the 1.6 μm band and the 2.1 μm band. In addition to extracting pixels that are likely to be targets from the mid-infrared image and far-infrared image and then discriminating between vehicles and plants only for those pixels, the number of pixels to be processed is reduced and target extraction is performed. Processing can be performed at high speed. It should be noted that the image matching processing step c is performed when the target image in FIG. 4 is a mid-infrared image, a far-infrared image, a 1.6 μm band and a 2.1 μm band.
For each type of image, the coordinates of each pixel are matched between the four images, and the calculation image of the middle infrared ray, the calculation image of the far infrared ray, the calculation image of the 1.6 μm band, and the calculation image of the 2.1 μm band are obtained. Perform creation. The subsequent calculations are performed between the calculation images created by the image matching process. Also, in the flowchart of FIG. 20, 2.3 images are used instead of the 2.1 μm band images.
It is also possible to use an image in the μm band.

In the target candidate extraction processing step s shown in FIG. 20, for example, a process for targeting a portion having high luminance in both the middle infrared image and the far infrared image is performed. Since the ratio between the mid-infrared image and the far-infrared image is standardized by taking the ratio of the mid-infrared image and the far-infrared image in the ratio calculation processing step r, a pixel having a ratio calculation result close to 1 is a pixel having a high brightness in both the mid-infrared image and the far-infrared image, that is, Becomes Therefore, in the target candidate extraction processing step s, the threshold value of the binarization processing is determined from the ratio calculation image that is the result of the ratio calculation processing step r, and the target candidates are narrowed down.

Embodiment 12 FIG. FIG. 21 shows a visible image, a mid-infrared image, a far-infrared image, a 1.6 μm band image in an outdoor situation where an OD-colored painted vehicle, a sunny plant, a shaded plant or soil, etc. can be seen in the same space. 5 is a flowchart for extracting an OD color vehicle using a 2.1 μm band image. Input step h of visible image, input step n of mid-infrared image, input step q of far-infrared image, input step a of 1.6 μm band image and 2.1 μm
After inputting a visible image, a mid-infrared image, a far-infrared image, a 1.6 μm band image, and a 2.1 μm band image in an m-band image input step b, the visible image and the mid-infrared image are input in an image matching processing step c. The far-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned, and the R, G, and R of the visible image input in the visible image data conversion step i.
The hue, luminance, and saturation of each pixel are calculated using each image of B, and in the OD color area extraction step j, pixels having the OD hue characteristic of the vehicle are extracted. Since the extracted pixels include not only the target vehicle but also clutter such as plants, target candidates are narrowed down next using the mid-infrared image and the far-infrared image. In the ratio calculation processing step r, the ratio between the mid-infrared image and the far-infrared image is taken and standardized, and in the target candidate extraction processing step s, target candidates are extracted using the calculation results. A target candidate is extracted using the visible image, target extraction is performed on the extracted pixels using the mid-infrared image and the far-infrared image, and then the extracted pixels are subjected to a ratio calculation processing step d at 1 A ratio calculation image is calculated by taking the ratio between each pixel of the images in the .6 μm band and the image in the 2.1 μm band.
The histogram of the ratio calculation image of the 6 μm band and the 2.1 μm band is calculated, and in the target extraction processing step f, the threshold value of the binarization processing of the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. . After extracting target candidates using the information of the visible image, the mid-infrared image, and the far-infrared image, 1.6
Since the discrimination between the vehicle and the plant is performed using the image of the μm band and the image of the 2.1 μm band, the target extraction capability can be increased, and the possibility of the target from the visible image, the mid-infrared image, and the far-infrared image is high. Since pixels are first extracted and then only the pixels are subjected to discrimination between the vehicle and the plant, the number of pixels to be processed is reduced, and the target extraction processing can be performed at high speed. Note that the image matching processing step c is performed between five images because the target image in FIG. 4 is a visible image, a mid-infrared image, a far-infrared image, and five types of images of 1.6 μm band and 2.1 μm band. The coordinates of the pixels are matched, and a visible calculation image, a mid-infrared calculation image, a far-infrared calculation image, a 1.6 μm calculation image, and a 2.1 μm calculation image are created. The subsequent calculations are performed between the calculation images created by the image matching process. The visible image data conversion step i and the OD color area extraction step j are shown in FIG.
9 and 10 and the contents described in the third embodiment, and the ratio calculation processing step r and the target candidate extraction processing step s are the same as the contents described in the eleventh embodiment.
In the flowchart of FIG. 21, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

Embodiment 13 FIG. FIG. 22 shows a visible image, a near-infrared image, a middle-infrared image, a far-infrared image, and 1. an outdoor situation in which an OD color painted vehicle, a sunny plant, a shaded plant, soil, and the like are visible in the same space. 9 is a flowchart illustrating a process for extracting an OD color vehicle using a 6 μm band image and a 2.1 μm band image. Input step h of visible image, input step k of near-infrared image, input step n of mid-infrared image, input step q of far-infrared image
In the input step a of the 1.6 μm band image and the input step b of the 2.1 μm band image, the visible image, the near infrared image, the middle infrared image, the far infrared image, the 1.6 μm band image, and 2.1 μm
After the input of the band image, in the image matching processing step c, the positions of the visible image, the near-infrared image, the mid-infrared image, the far-infrared image, the 1.6 μm band image, and the 2.1 μm band image are aligned. In the arithmetic processing step l, the ratio between the near-infrared image and the B image, the ratio between the R image and the B image, and the G image from the visible image and the near-infrared image composed of the aligned R, G, and B After calculating and normalizing the ratio of the images of B, the calculation result is plotted in a target candidate extraction space in a target candidate extraction processing step m, and a threshold value for extracting a target candidate which is an OD color painting area in the space is extracted. Is determined, and target candidates are extracted. Since the extracted pixels may include clutter such as plants in addition to the target vehicle, the target candidates are then narrowed down using the mid-infrared image and the far-infrared image. In the ratio calculation processing step r, the ratio between the mid-infrared image and the far-infrared image is taken and standardized, and in the target candidate extraction processing step s, target candidates are extracted using the calculation results. Target candidates are extracted using the visible image and near-infrared image, target extraction is performed on the extracted pixels using the mid-infrared image and far-infrared image, and then ratio calculation is performed on the extracted pixels. 1.6 μm in processing step d
A ratio calculation image is calculated by taking the ratio between each pixel of the image of the band and the 2.1 μm band, and a histogram of the ratio calculation image of the 1.6 μm band and the 2.1 μm band is calculated in a histogram calculation step e. In the target extraction processing step f, the threshold value of the binarization processing of the vehicle and the plant is determined from the histogram to discriminate the vehicle and the plant. After extracting target candidates using the information of the visible image, the near-infrared image, the mid-infrared image, and the far-infrared image, the vehicle and the plant are discriminated using the 1.6 μm band and the 2.1 μm band images. The target extraction capability can be increased, and pixels with a high possibility of target are first extracted from the visible image, near-infrared image, mid-infrared image, and far-infrared image, and then only those pixels are used to discriminate between vehicles and plants Therefore, the number of pixels to be processed is reduced, and the speed of the target extraction processing can be increased. Note that the image matching processing step c is performed in FIG. 4 because the target images are visible, near-infrared, middle-infrared, far-infrared, 1.6 μm, and 2.1 μm images. The coordinates of each pixel are matched between the images, and the visible calculation image, near-infrared calculation image, mid-infrared calculation image, far-infrared calculation image, 1.6 μm band calculation image, and 2.1 μm A band calculation image is created. The subsequent calculations are performed between the calculation images created by the image matching process. The ratio calculation processing step 1 and the target candidate extraction processing step m are the same as the contents described in FIG. 12 and the fourth embodiment, and the ratio calculation processing step r and the target candidate extraction processing step s are the contents described in the eleventh embodiment. Is the same as In the flowchart of FIG. 22, it is also possible to use a 2.3 μm band image instead of a 2.1 μm band image.

Embodiment 14 FIG. FIG. 23 is a system configuration diagram for executing target extraction. The target extraction processing device A2 executes the various target extraction flowcharts shown in FIGS. 3, 6, and 7, 11, 13, and 15 to 22. FIG. The image input device A1 displays a visible image, a near infrared image, a middle infrared image, a far infrared image, and 1.6.
This is a device for acquiring an image in the μm band, an image in the 2.1 μm band, and an image in the 2.3 μm band. The image combining device A3 combines the input image output from the target extraction processing device A2 with the target extraction result. The image display device A4 is a device that displays an output image of the image synthesis device A3. The target extraction processing device A2 includes an image input device interface unit A5
, Image synthesizing device interface unit A6, image matching operation unit A7, target candidate extraction operation unit A8, and target extraction operation unit A
9, an image memory A10, an image memory A11, an image memory A12, and an image memory A13.

Various analog image signals acquired by the image input device A1 are converted into A by the image input device interface unit A5.
A / D conversion is performed and stored in the image memory A10. The image matching calculation unit A7 performs positioning of various images using the digital image read from the image memory A10, and stores the result in the image memory A11. The arithmetic unit reads the contents of the image memory A11 in which the alignment of each image is completed, and reads the contents of FIGS. 7, 11, 13, and 15 to 22.
, A ratio conversion processing step 1, a target candidate extraction processing step m, a target enhancement processing step o, a histogram calculation step e, a binarization processing step p, and a ratio calculation. The coordinate data on the image from which the target candidate is extracted by executing according to the target extraction flowchart using the processing step r and the target candidate extraction processing step s is recorded in the image memory A12,
The target extraction calculation unit A9 retrieves the coordinates of the target candidate from the image memory A12 and obtains 1.6 μm band and 2.1 μm band or 2.3 μm from the image memory A11 whose alignment has been completed.
The digital image data of the μm band is read, and the coordinate data of the target candidate is read out of the digital image data, and FIG.
11, 13, and 15 to 22, the ratio calculation processing step d, the histogram calculation step e, and the target extraction processing step f are executed to obtain the coordinates of the target position on the image, and the result is recorded in the image memory AB. The image synthesizing device interface unit A6 reads the digital image data for synthesizing the target extraction result from the image memory A11 in which the alignment is completed, and performs D / A conversion to convert the analog image data for synthesizing the target extraction result into the image synthesizing device. A3, the coordinate data of the target position is read from the image memory A13 and sent to the image synthesizing device A3. The image synthesizing device A3 uses the sent analog image data and the coordinate data of the target position to set the target in the analog image data. The displayed symbols are combined and sent to the image display device A4, and the target sent by the image display device A4. Symbol is the analog image data combined into image display.

[0067]

As is clear from the above description, according to the present invention, since the images of the 1.6 μm band and the 2.1 μm band are used for discriminating the target and the background, the same shape and the same size are used. Under the situation where there is a target and a clutter having the same luminance in the respective images of the 1.6 μm band and the 2.1 μm band, the same luminance is obtained in the images of both the 1.6 μm band and the 2.1 μm band. Since it is unlikely that the target and the clutter exist, it is possible to extract only the target with a high probability.

Further, discrimination between the object and background of the present invention is as follows.
Since the images of the 6 μm band and the 2.3 μm band are used, they have the same shape and the same size, and the 1.6 μm band and the 2.1 μm band.
Under the situation where the target and the clutter having the same luminance exist in each image of the m band, the possibility that the target and the clutter having the same luminance exist in both the 1.6 μm band image and the 2.3 μm band image may exist. Is low, so that only the target can be extracted with a high probability.

The present invention extracts a target in consideration of the characteristics of a visible image in addition to 1.6 μm band and 2.1 μm band images or 1.6 μm band and 2.3 μm band images. It has the effect of extracting only

Further, the present invention distinguishes between a target and a background by using 1.6 μm band and 2.1 μm band images or 1.6 μm band and 2.3 μm band images for target candidates selected from visible images. Is performed, the number of pixels to be processed is reduced, and the target extraction processing can be speeded up.

According to the present invention, a target is extracted by taking into account the characteristics of a visible image and a near-infrared image in addition to 1.6 μm band and 2.1 μm band images or 1.6 μm band and 2.3 μm band images. There is an effect that only the target can be extracted with a high probability.

The present invention is also applicable to a 1.6 μm band and a 2.1 μm band for target candidates selected from a visible image and a near-infrared image.
Since the discrimination between the target and the background is performed using the image in the μm band or the images in the 1.6 μm band and the image in the 2.3 μm band, the number of pixels to be processed is reduced and the target extraction processing can be performed at high speed.

According to the present invention, in addition to the 1.6 μm band and the 2.1 μm band image or the 1.6 μm band and the 2.3 μm band image, the target is extracted in consideration of the characteristics of the mid-infrared image. There is an effect that only the target can be extracted.

Further, according to the present invention, for a target candidate selected from a mid-infrared image, images of 1.6 μm band and 2.1 μm band or images of 1.6 μm band and 2.3 μm band are used for the target and background. Since discrimination is performed, there is an effect that the number of pixels to be processed is reduced and the target extraction processing can be performed at high speed.

The present invention extracts a target in consideration of the features of far-infrared images in addition to 1.6 μm and 2.1 μm band images or 1.6 μm and 2.3 μm band images. There is an effect that only the target can be extracted.

Further, according to the present invention, for a target candidate selected from a far-infrared image, an image of 1.6 μm band and 2.1 μm band or an image of 1.6 μm band and 2.3 μm band is used to define a target and a background. Since discrimination is performed, there is an effect that the number of pixels to be processed is reduced and the target extraction processing can be performed at high speed.

The present invention extracts a target by taking into account the features of a visible image and a mid-infrared image in addition to 1.6 μm band and 2.1 μm band images or 1.6 μm band and 2.3 μm band images. There is an effect that only the target can be extracted with a high probability.

The present invention uses a 1.6 μm band and a 2.1 μm band image or a 1.6 μm and a 2.3 μm band image for a target and background for a target candidate selected from a visible image and a mid-infrared image. Is performed, the number of pixels to be processed is reduced, and the target extraction processing can be speeded up.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the m-band image or the 1.6 μm and 2.3 μm band images, the target is extracted by taking into account the features of the visible image and the far-infrared image, so that there is an effect that only the target can be extracted with a high probability.

The present invention uses a 1.6 μm band and a 2.1 μm band image or a 1.6 μm and a 2.3 μm band image for a target and a background for a target candidate selected from a visible image and a far-infrared image. Is performed, the number of pixels to be processed is reduced, and the target extraction processing can be speeded up.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the m-band image or the 1.6 μm and 2.3 μm band images, the target is extracted taking into account the features of the visible image, the near-infrared image, and the mid-infrared image, so that only the target can be extracted with a high probability. There is.

According to the present invention, a 1.6 μm band and a 2.1 μm band image or a 1.6 μm band and a 2.3 μm band are selected for target candidates selected from a visible image, a near-infrared image, and a mid-infrared image.
Since discrimination between the target and the background is performed using the m-band image, the number of pixels to be processed is reduced, and there is an effect that the target extraction processing can be sped up.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the band image or the 1.6 μm band and 2.3 μm band images, the target is extracted taking into account the characteristics of the visible image, the near-infrared image, and the far-infrared image. is there.

According to the present invention, a 1.6 μm band and a 2.1 μm band image or a 1.6 μm band and a 2.3 μm band are selected for a target candidate selected from a visible image, a near infrared image and a far infrared image.
Since discrimination between the target and the background is performed using the m-band image, the number of pixels to be processed is reduced, and there is an effect that the target extraction processing can be sped up.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the m-band image or the 1.6 μm band and the 2.3 μm band image, the target is extracted in consideration of the features of the mid-infrared image and the far-infrared image, so that there is an effect that only the target can be extracted with high probability.

According to the present invention, a 1.6 μm band and a 2.1 μm band are selected for target candidates selected from a mid-infrared image and a far-infrared image.
Since the target and the background are discriminated using the m-band image or the 1.6 μm band and the 2.3 μm band image, the number of pixels to be processed is reduced and the target extraction process can be performed at high speed.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the m-band image or the 1.6 μm and 2.3 μm band images, the target is extracted taking into account the features of the visible image, the mid-infrared image, and the far-infrared image. There is.

According to the present invention, a 1.6 μm band and a 2.1 μm band image or a 1.6 μm band and a 2.3 μm band are selected for target candidates selected from a visible image, a middle infrared image and a far infrared image.
Since discrimination between the target and the background is performed using the m-band image, the number of pixels to be processed is reduced, and there is an effect that the target extraction processing can be sped up.

The present invention is applicable to the 1.6 μm band and the 2.1 μm band.
In addition to the m-band image or 1.6 μm and 2.3 μm band images, the target is extracted by taking into account the characteristics of the visible image, near-infrared image, mid-infrared image, and far-infrared image. There is an effect that can be extracted.

According to the present invention, a 1.6 μm band and a 2.1 μm band image or a 1.6 μm band image are selected for target candidates selected from a visible image, a near infrared image, a middle infrared image, and a far infrared image.
Since the discrimination between the target and the background is performed using the image in the μm band and the image in the 2.3 μm band, the number of pixels to be processed is reduced and the target extraction processing can be performed at high speed.

Further, the present invention combines a visible image, a near-infrared image, a mid-infrared image, a far-infrared image, and images of 1.6 μm, 2.1 μm, and 2.3 μm bands having different wavelength bands to obtain a material. Since the target extraction processing using the difference in spectral characteristics is performed in pixel units, there is an effect that a minute target can be extracted under a complicated background that does not depend on the size of the target.

[Brief description of the drawings]

FIG. 1 shows an OD color paint used in a vehicle in a 1.2 to 2.5 μm band image, and a brightness histogram of shade and sunny leaves.

FIG. 2 shows an OD color paint used in a vehicle and a luminance histogram of shade and sunny leaves used in a ratio calculation image of a 1.6 μm band and a 2.1 μm band.

FIG. 3 is a flowchart for discriminating a target from a background using images in the 1.6 μm and 2.1 μm bands.

FIG. 4 is a diagram illustrating the content of an image matching process.

FIG. 5 shows luminance histograms of OD color painting, shade, and sunny leaves used in a vehicle in a ratio calculation image of a 1.6 μm band and a 2.3 μm band.

FIG. 6 is a flowchart for discriminating a target from a background using images in the 1.6 μm band and the 2.3 μm band.

FIG. 7: 1.6 μm after extracting target candidates from the visible image
It is a flowchart figure for discriminating a target and a background using the image of an m band and a 2.1 micrometer band.

FIG. 8 is a diagram illustrating a relationship among hue, saturation, and brightness.

FIG. 9 is a diagram showing the relationship between color and angle in hue.

FIG. 10 is a diagram illustrating an example of a region including an OD color in a hue.

FIG. 11 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image and a near-infrared image.

FIG. 12 is a diagram illustrating an example of a target candidate extraction space for extracting a target candidate for OD coating used for a vehicle or the like using a visible image and a near-infrared image.

FIG. 13 After extracting target candidates from the mid-infrared image,
It is a flowchart figure for discriminating a target and a background using the image of a 1.6-micrometer band and a 2.1-micrometer band.

FIG. 14 is a diagram showing an outline of a median difference filter process which is an example of a target enhancement process for a mid-infrared image or a far-infrared image.

[FIG. 15] After extracting target candidates from the far-infrared image,
It is a flowchart figure for discriminating a target and a background using the image of a 1.6-micrometer band and a 2.1-micrometer band.

FIG. 16 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image and a mid-infrared image.

FIG. 17 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image and a far-infrared image.

FIG. 18 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image, a near-infrared image, and a mid-infrared image.

FIG. 19 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image, a near-infrared image, and a far-infrared image.

FIG. 20 is a flowchart for discriminating a target from a background using images in the 1.6 μm band and the 2.1 μm band after extracting target candidates from the middle infrared image and the far infrared image.

FIG. 21 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from the visible image, the mid-infrared image, and the far-infrared image.

FIG. 22 is a flowchart for discriminating a target from a background by using 1.6 μm band and 2.1 μm band images after extracting target candidates from a visible image, a near infrared image, a middle infrared image, and a far infrared image. It is.

FIG. 23 is a diagram showing a system configuration diagram for executing target extraction.

[Description of Signs] a 1.6 μm band image input step, b 2.1 μm
A band image input step, c image matching processing step,
d ratio calculation processing step, e histogram calculation step, f target extraction processing step, g 2.3 μm band image input step, h visible image input step, i
Visible image data conversion step, j OD color area extraction step, k near-infrared image input step, l ratio calculation processing step, m target candidate extraction processing step, n
Mid-infrared image input step, o target enhancement processing step, p binarization processing step, q far-infrared image input step, r ratio calculation processing step, s target candidate extraction processing step, 11.2 to 2.5 μm band Histogram of OD color paint used in vehicle in image, 21.2-
Histogram of shaded leaves in 2.5 μm band image, 3
Histogram of leaf in the sun in 1.2-2.5 μm band image, 4 Histogram of OD color paint used for vehicle in ratio operation image of 1.6 μm band and 2.1 μm band, 5
Histogram of shaded leaves in the 1.6 μm band and 2.1 μm band ratio operation image, 6 Histogram of sunny leaves in the 1.6 μm band and 2.1 μm band ratio operation image, 7
Histogram of OD color painting used in vehicles with ratio calculation images of 1.6 μm band and 2.3 μm band, histogram of shaded leaves in ratio calculation images of 81.6 μm band and 2.3 μm band, 91 The histogram of the leaf in the sun in the ratio calculation image of the 2.6 μm band and the 2.3 μm band, a1 target points A and a
2 target point B, a3 target point C, a4 image G _1, a5
Image G ₂ , a6 Image G _n , a7 Calculation image G ₁ ,
a8 Calculation image G ₂ , a9 Calculation image G _n , b1
Brightness Y, b2 Saturation C, b3 Hue H, c1 OD color candidate area, c2 plant area, c3 soil area, c4 OD color painting area, c5 target candidate extraction space, d1 median filter, A1 image input device, A2 target extraction Processing device, A3 image synthesis device, A4 image display device, A5
Image input device interface unit, A6 image synthesis device interface unit, A7 image matching operation unit, A8 target candidate extraction operation unit, A9 target extraction operation unit, A10 image memory, A11 image memory, A12 image memory, A
13 Image memory.

────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Masahiro Ito 2-3-2 Marunouchi, Chiyoda-ku, Tokyo Inside Mitsubishi Electric Corporation (72) Inventor Satoshi Wakabayashi 2-3-2 Marunouchi, Chiyoda-ku, Tokyo Mitsubishi Electric (56) References JP-A-61-76970 (JP, A) Bulletin of Faculty of Engineering, Shinshu University, 67 (1990) Masaaki Kondo, Yoshiharu Asano, Tsutomu Shima, p. 33-52 Journal of the Japanese Association of Agricultural Engineers, 57 [12] (1989) Kazuya Miyama, p. 1175-1181 Abstracts of the 43rd Annual Meeting of the Japan Society of Civil Engineers Part 4, (1989) Sei Hoshi, Koji Nishimura, Yashiko Ikebe, p. 445-446 (58) Fields investigated (Int.Cl. ⁶ , DB name) G01J 1/02 G01V 9/04 G01W 1/02 G01M 11/00 G01S 3/781 G06F 15/00 G06T 1/00 F41G 1 / 32

Claims (13)

(57) [Claims] 1. In a ratio calculation process for discriminating a target which is an artificial object from a background which is a natural object by utilizing a difference in reflection characteristics of a substance, a ratio between an image in a 1.6 μm band and an image in a 2.1 μm band is obtained. A target extraction method characterized by the following. 2. In a ratio calculation process for discriminating between a target which is an artificial object and a background which is a natural object by utilizing a difference in reflection characteristics of a substance, a ratio between 1.6 μm band and 2.3 μm band images is obtained. A target extraction method characterized by the following. 3. A material having a reflection characteristic of 0.4 to 0.2.
3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target by utilizing a difference in reflection characteristics in a visible region of a wavelength band of 7 μm. 4. A material having a reflection characteristic of 0.4 to 0.0.
3. The method according to claim 1, wherein the method is combined with a method of extracting a target by utilizing a difference in reflection characteristics between a visible region having a wavelength band of 7 μm and a near infrared region having a wavelength band of 0.7 to 1 μm. Described target extraction method. 5. An infrared ray radiated from an object, wherein
3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target using a difference in radiation amount from a mid-infrared region in a wavelength band of 4 [mu] m or 4 to 5 [mu] m. 6. An infrared ray radiated from an object, and
3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target by using a difference in radiation amount from a far-infrared region having a wavelength band of 3 μm. 7. A material having a reflection characteristic of 0.4 to 0.4.
A method of extracting a target by using the difference in reflection characteristics in the visible region, which is a 7 μm wavelength band, and the amount of radiation from the mid-infrared region, which is a 3 to 5 μm or 4 to 5 μm wavelength band, of infrared rays radiated from an object 3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target using a difference between the two. 8. Among the reflection characteristics of the substance, 0.4 to 0.1.
A method of extracting a target by using a difference in reflection characteristics in the visible region, which is a 7 μm wavelength band, and a difference in the amount of radiation from a far-infrared region, which is a wavelength band of 8 to 13 μm, among infrared rays emitted from an object. 3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method for extracting a target. 9. A material having a reflection characteristic of 0.4-0.
A method of extracting a target by using a difference between a visible region having a wavelength band of 7 μm and a near-infrared region having a wavelength band of 0.7 to 1 μm, and a method of extracting 3 to 5 μm or 4 to 5 μm of infrared radiation emitted from an object. 3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target using a difference in radiation amount from a mid-infrared region that is a wavelength band. 10. A material having a reflection property of 0.4 to 0.4
Visible region of 0.7 μm wavelength band and 0.7 to 1 μm
The method of extracting the target by using the difference in the reflection characteristics in the near infrared region, which is the wavelength band of the above, and the difference in the amount of radiation from the far infrared region, which is the wavelength band of 8 to 13 μm, of the infrared radiation radiated from the object 3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method for extracting a target. 11. Three to three infrared rays emitted from an object
A method for extracting a target by utilizing a difference in radiation amount from a mid-infrared region having a wavelength band of 5 μm or 4 to 5 μm and a far-infrared region having a wavelength band of 8 to 13 μm. The target extraction method according to claim 1 or 2. 12. The material having a reflection characteristic of 0.4 to
A method of extracting a target by utilizing the difference in reflection characteristics in the visible region, which is a wavelength band of 0.7 μm, and a mid-infrared region, which is a wavelength band of 3 to 5 μm or 4 to 5 μm among infrared rays emitted from an object, and 8 3. The method according to claim 1, wherein the method is combined with a method of extracting a target using a difference in radiation amount in a far infrared region which is a wavelength band of about 13 [mu] m.
Described target extraction method. 13. The material having a reflection characteristic of 0.4 to 0.4.
Visible region of 0.7 μm wavelength band and 0.7 to 1 μm
The method of extracting a target by using the difference in reflection characteristics in the near infrared region, which is the wavelength band of the infrared ray, and the mid-infrared region, which is the wavelength band of 3 to 5 μm or 4 to 5 μm, and 8 to 13 μm among the infrared rays emitted from the object 3. The target extraction method according to claim 1, wherein the target extraction method is combined with a method of extracting a target by using a difference in radiation amount in a far-infrared region which is a wavelength band.