JP7644659B2 - Image processing device and vehicle - Google Patents

Description

This disclosure relates to an image processing device that estimates the position of an object based on a captured image, and a vehicle equipped with such an image processing device.

Images captured by an imaging device include images of various objects. For example, Patent Literature 1 discloses an image processing device that estimates the position of an object based on such captured images.

JP 2008-123141 A

However, in such image processing devices, there is a demand for speeding up the process of estimating object positions. It is desirable to provide an image processing device that can speed up the process of estimating object positions, and a vehicle equipped with such an image processing device.

An image processing device according to an embodiment of the present disclosure includes an area setting unit that sets one or more image areas in a captured image, an extraction unit that extracts feature amounts arranged in a two-dimensional arrangement contained in the image areas of the captured image, and a position estimation unit that estimates the position of a tracked object by performing a correlation calculation using a convolution calculation on the feature amounts between image areas of captured images obtained in previous and subsequent frame periods along the time axis. When performing the convolution calculation, the position estimation unit uses a Fourier transform on the feature amounts, and performs the Fourier transform and the convolution calculation on the two-dimensionally arranged feature amounts, one dimension at a time.

A vehicle according to one embodiment of the present disclosure includes an image processing device according to the embodiment of the present disclosure, and a vehicle control unit that controls the vehicle using the estimated position of the tracked object obtained from the position estimation unit.

1 is a block diagram illustrating an example of a schematic configuration of a vehicle according to an embodiment of the present disclosure. FIG. 2 is a top view illustrating an example of the external configuration of the vehicle illustrated in FIG. 1 . 2 is a schematic diagram illustrating an example of a left image and a right image generated by the stereo camera illustrated in FIG. 1 . 2 is a schematic diagram illustrating an example of an image region set by a region setting unit illustrated in FIG. 1 . FIG. 1 is a schematic diagram for explaining an overview of object tracking using correlation calculation. 5A and 5B are schematic diagrams illustrating an example of a correlation calculation process according to the embodiment. 11 is a flowchart illustrating an example of a process including a correlation calculation according to an embodiment. 11A and 11B are schematic diagrams illustrating examples of correlation calculation processes according to a comparative example and an embodiment of the present invention.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order.
1. Embodiment (Example of setting image area using distance information or machine learning)
2. Modifications

1. Preferred embodiment
[composition]
Fig. 1 is a block diagram showing an example of a schematic configuration of a vehicle (vehicle 10) according to an embodiment of the present disclosure. Fig. 2 is a schematic top view showing an example of an external configuration of the vehicle 10 shown in Fig. 1.

As shown in FIG. 1, the vehicle 10 is equipped with a stereo camera 11, an image processing device 12, and a vehicle control unit 13. Note that in FIG. 1, the driving power source (engine, motor, etc.) of the vehicle 10 is omitted. The vehicle 10 is, for example, an electric vehicle such as a hybrid electric vehicle (HEV) or an electric vehicle (EV), or a gasoline vehicle.

(A. Stereo camera 11)
The stereo camera 11 is a camera that captures an image of the area ahead of the vehicle 10 to generate a pair of images (a left image PL and a right image PR) having a parallax therebetween, as shown in Fig. 2 for example. The stereo camera 11 includes a left camera 11L and a right camera 11R, as shown in Figs. 1 and 2.

The left camera 11L and the right camera 11R each include, for example, a lens and an image sensor. The left camera 11L and the right camera 11R are arranged near the top of the windshield 19 of the vehicle 10, for example, as shown in FIG. 2, at a predetermined distance apart along the width direction of the vehicle 10. The left camera 11L and the right camera 11R are configured to perform imaging operations in synchronization with each other. Specifically, as shown in FIG. 1, the left camera 11L generates a left image PL, and the right camera 11R generates a right image PR. The left image PL includes a plurality of pixel values, and the right image PR includes a plurality of pixel values. The left image PL and the right image PR constitute a stereo image PIC, as shown in FIG. 1.

Figure 3 shows an example of such a stereo image PIC. Specifically, Figure 3(A) shows an example of a left image PL, and Figure 3(B) shows an example of a right image PR. Note that x and y shown in Figure 3 represent the x-axis and y-axis, respectively. In this example, another vehicle (preceding vehicle 90) is traveling ahead of vehicle 10 on the road on which vehicle 10 is traveling. The left camera 11L captures an image of the preceding vehicle 90 to generate the left image PL, and the right camera 11R captures an image of the preceding vehicle 90 to generate the right image PR.

The stereo camera 11 is configured to generate a stereo image PIC that includes the left image PL and the right image PR. The stereo camera 11 also generates a series of stereo images PIC by performing an imaging operation at a predetermined frame rate (e.g., 60 fps).

(B. Image Processing Device 12)
The image processing device 12 is a device that performs various image processing (such as tracking processing of an object ahead of the vehicle 10) based on the stereo image PIC supplied from the stereo camera 11. As shown in FIG. 1, the image processing device 12 has an image memory 121, a distance information generating unit 122, an area setting unit 123, a feature extraction unit 124, and a position estimation unit 125.

Such an image processing device 12 is configured to include, for example, one or more processors (CPU: Central Processing Unit) that execute programs, and one or more memories communicatively connected to these processors. In addition, such memory is configured, for example, with a RAM (Random Access Memory) that temporarily stores processing data, and a ROM (Read Only Memory) that stores programs, etc.

The feature extraction unit 124 described above corresponds to a specific example of an "extraction unit" in this disclosure.

(Image memory 121)
1, the image memory 121 is a memory for temporarily storing the left image PL and the right image PR included in the stereo image PIC. The image memory 121 sequentially supplies at least one of the left image PL and the right image PR thus stored as a captured image P to the distance information generating unit 122 and the feature amount extracting unit 124 (see FIG. 1).

(Distance information generation unit 122)
The distance information generating unit 122 generates distance information Iz by performing predetermined image processing including stereo matching processing and filtering processing based on the captured image P (here, the left image PL and the right image PR) read out from the image memory 121 (see FIG. 1). Specifically, the distance information generating unit 122 generates a distance image including a plurality of pixel values based on the left image PL and the right image PR. In this example, each of the plurality of pixel values is a parallax value. In other words, each of the plurality of pixel values corresponds to a distance to a point corresponding to each pixel in a three-dimensional real space. Note that this example is not limited, and for example, each of the plurality of pixel values may be a distance value indicating a distance to a point corresponding to each pixel in a three-dimensional real space. In this way, the distance information generating unit 122 generates distance information Iz, which is information indicating a distance to a point corresponding to each pixel.

(Area setting unit 123)
The region setting unit 123 sets one or more image regions R in the captured image P based on the distance information Iz supplied from the distance information generating unit 122. Specifically, the region setting unit 123 specifies a plurality of pixels that are located close to each other in the captured image P and have approximately the same parallax value based on the distance information Iz, and sets a rectangular region including the plurality of pixels as the image region R. That is, when an object is present in the captured image P, the pixels in the region corresponding to the object are located close to each other and have approximately the same parallax value. Thus, by setting the image region R in this manner, the region setting unit 123 sets the image region R so as to surround the object.

Figure 4 is a schematic diagram showing an example of an image region R set by the region setting unit 123. In the example shown in Figure 4, an image region R is set for each of two vehicles in the captured image P (here, one of the left image PL and the right image PR). Note that in this example, the region setting unit 123 sets the image region R for the vehicles, but is not limited to this example, and may also set the image region R for, for example, a person, a guardrail, a wall, etc.

In this manner, information about one or more image regions R set by the region setting unit 123 is supplied to the feature extraction unit 124, as shown in FIG. 1.

In the example shown in FIG. 1, the region setting unit 123 sets the image region R using the distance information Iz, but this is not limited to the example. That is, the region setting unit 123 may use a trained model such as a DNN (Deep Neural Network) to identify an object in the captured image P and output the coordinates of the identified object to set the image region R, which is a rectangular region. That is, the region setting unit 123 may set the image region R using machine learning.

(Feature Extraction Unit 124)
The feature extraction unit 124 extracts feature values F contained in one or more image regions R in a captured image P (here, one of the left image PL and the right image PR) (see FIG. 1). The feature values F are configured from pixel values of a plurality of pixels arranged in a matrix (two-dimensional arrangement), as will be described in detail later (FIG. 6, etc.). Examples of such feature values F include RGB (Red, Green, Blue) feature values and HOB (Histograms of Oriented Gradients) feature values.

The feature extraction unit 124 extracts such feature F, for example, by using a trained model of DNN. In this case, for example, each of the neural networks in the feature extraction unit 124 has multiple convolution layers and multiple pooling layers.

The feature F extracted in this manner by the feature extraction unit 124 is supplied to the position estimation unit 125 (see Figure 1).

(Position estimation unit 125)
The position estimation unit 125 estimates the position of the tracked object by performing a correlation calculation using a convolution calculation on the above-mentioned feature amount F between image regions R of captured images P obtained in previous and subsequent frame periods along the time axis. In other words, the position estimation unit 125 performs object tracking by estimating the position of the tracked object using the feature amount F for each captured image P in each frame period.

In addition, such object tracking is performed using object tracking methods that utilize a CNN (Convolutional Neural Network), such as KCF (Kernelized Correlation Filter), template matching, and Siamese Network.

Figure 5 shows a schematic overview of object tracking using such correlation calculations. Note that t in Figure 5 represents the time axis, and this also applies to the following examples.

For example, as shown in Figures 5(A) and 5(B), in object tracking using correlation calculations, how the position (position coordinate x0) of the tracked object has moved in each captured image P0, P1, ... Pn at each timing t0, t1, ... tn along the time axis t is estimated (see movement amount Δx). Specifically, in such object tracking, the position where the correlation value is the largest (indicates the maximum value) among each captured image P0, P1, ... Pn is estimated as the position coordinate x0 of the tracked object, and is updated as needed.

As an example, when tracking an object using the above-mentioned KCF, various calculations are performed as follows. That is, first, machine learning of the coefficient w is performed so that the value shown in the following equation (1) is minimized. Then, if the value of this coefficient w can be optimized, it becomes an equation that outputs the position coordinates of the tracked object in the input image. Note that when optimizing the coefficient w in this way, for example, the kernel trick described below is used.

Also, FIG. 6 is a schematic diagram showing an example of correlation calculation processing according to this embodiment. Specifically, it shows an example of correlation calculation processing using a feature amount F extracted from an image region R of a captured image Pn obtained in the current frame period (see FIG. 6(A)) and a feature amount Fn+1 obtained in an image region R of a captured image Pn+1 obtained in the immediately following frame period (one frame later: see FIG. 6(B)).

In addition, in each of the above feature quantities Fn and Fn+1, 16 pixels PX (4 pixels in the vertical direction corresponding to the y-axis direction × 4 pixels in the horizontal direction corresponding to the x-axis direction) are arranged in a matrix (two-dimensional arrangement) (see Figures 6(A) and 6(B)). Also, Figure 6 shows a correlation map Mc obtained from a correlation calculation by a convolution calculation using a Fourier transform, which will be described below.

In this embodiment, the position estimation unit 125 uses a Fourier transform on the feature F when performing the convolution operation on the feature F described above. In addition, the position estimation unit 125 uses, for example, the KCF described above to perform a correlation operation using such a Fourier transform. Furthermore, when this KCF is used, as shown in FIG. 5, for example, a Fourier transform is used to convert the feature indicating the color space (such as the RGB feature described above) into a frequency space, thereby speeding up the processing.

When the kernel trick described above is used in such a KCF, the function F(x) that performs the Fourier transform is defined by the following equation (2). Furthermore, when the above-mentioned convolution operation is expressed as (f*g), it is defined by the following equation (3).

Then, the convolution operation between the results of Fourier transform is expressed by the following equation (4). In other words, as can be seen from the last equation in equation (4), the convolution operation between the results of Fourier transform is equal to the product of the results of Fourier transform. Specifically, in the example of FIG. 6, when performing a convolution operation on feature quantities Fn and Fn+1 that have been transformed into frequency space using a Fourier transform, the operation is performed by taking the product of these feature quantities Fn and Fn+1 (matrices). By using KCF in this way, the amount of calculation is reduced, and processing speed is increased.

In this embodiment, the position estimation unit 125 performs such Fourier transform and convolution calculations on the two-dimensionally arranged feature amount F, one dimension at a time (see, for example, the dashed arrows shown for feature amounts Fn and Fn+1 in Figs. 6(A) and 6(B)). A detailed example of the processing in such a position estimation unit will be described later (Figs. 7 and 8).

(C. Vehicle control unit 13)
The vehicle control unit 13 performs various vehicle controls on the vehicle 10 using the result of the position estimation (estimated position Pe of the tracked object) supplied from the position estimation unit 125 (see FIG. 1). Specifically, the vehicle control unit 13 performs, for example, driving control of the vehicle 10 and operation control of various members in the vehicle 10 based on the information on the estimated position Pe.

Similar to the image processing device 12, such a vehicle control unit 13 is configured to include, for example, one or more processors (CPUs) that execute programs, and one or more memories communicatively connected to these processors. Also, like the image processing device 12, such memories are configured, for example, with a RAM that temporarily stores processing data, and a ROM that stores programs, etc.

[Actions, actions and effects]
Next, the operation, functions and effects of this embodiment will be described in detail.

(A. Position Estimation Processing, etc., of the Present Embodiment)
First, an example of the process (position estimation process, etc.) including the correlation calculation described above in this embodiment will be described with reference to Figures 1 to 6 as well as Figures 7 and 8. Figure 7 shows a flow chart of such a process example including correlation calculation. Figure 8 shows a schematic comparison (see the dashed line CF) of a comparative example (Figure 8A) and an example of this embodiment (Figure 8B).

In the processing example shown in FIG. 7, first, the stereo camera 11 captures an image of the area in front of the vehicle 10 to generate a stereo image PIC (left image PL and right image PR) (step S100). Specifically, a stereo image PIC (left image PL and right image PR) such as that shown in FIG. 3 is generated.

Next, the image memory 21 in the image processing device 12 temporarily stores the stereo image PIC (left image PL and right image PR) thus generated as the captured image P (step S101). Next, the area setting unit 123 determines whether or not the image area R in the captured image P has been set (step S102). If it is determined that such an image area R has been set (step S102: Y), the process proceeds to step S104, which will be described later.

On the other hand, if it is determined that such an image region R has not been set (step S102: N), the region setting unit 123 then uses the above-mentioned method (distance information Iz or machine learning, etc.) to set one or more image regions R in the captured image P (step S103). Then, in step S104, the feature extraction unit 124 extracts the above-mentioned feature F (two-dimensional arrangement) included in the image region R of the captured image P.

Next, the position estimation unit 125 estimates the position of the tracked object by performing a correlation calculation using a convolution calculation on the feature amount F between the image regions R of the captured images Pn and Pn+1 (see FIG. 6) described above (step S105). Specifically, the position estimation unit 125 performs the Fourier transform and convolution calculation described above on the two-dimensionally arranged feature amount F, one dimension at a time, to estimate the position of such a tracked object.

More specifically, in the embodiment of FIG. 8B corresponding to the example of FIG. 6, the position estimation unit 125 performs position estimation as follows. That is, the position estimation unit 125 first performs one-dimensional processing of the Fourier transform and convolution operation as described above along the x-axis direction (row direction) of the feature Fn (see dashed arrow). Then, the position estimation unit 125 performs Fourier transform and convolution operation on the two-dimensionally arranged feature Fn by adding the results of such one-dimensional processing along the x-axis direction to multiple sets (four sets in this example) along the y-axis direction (column direction).

Here, the x-axis direction (row direction) described above corresponds to a specific example of the "first direction" in this disclosure. Also, the y-axis direction (column direction) described above corresponds to a specific example of the "second direction" in this disclosure.

If it is determined in step S105 that it is difficult to estimate the position of the tracked object, then the position estimation ends, and the example process shown in FIG. 7 also ends.

Next, the vehicle control unit 13 uses the position estimation result obtained in step S105 (the estimated position Pe of the tracked object described above) to perform various vehicle controls on the vehicle 10 (such as the driving control of the vehicle 10 described above and the operation control of various components) (step S106).

This completes the series of processing examples shown in Figure 7.

(B. Actions and Effects)
In this manner, in this embodiment, the following processing is performed when estimating the position of a tracked object by performing a correlation calculation using a convolution calculation on extracted feature amounts F (two-dimensional arrangement) between image regions R of captured images P obtained in adjacent frame periods along the time axis. That is, when performing a convolution calculation on feature amounts F, a Fourier transform is used on feature amounts F, and these Fourier transforms and convolution calculations are each performed one-dimensionally on feature amounts F arranged in a two-dimensional arrangement. As a result, in this embodiment, the amount of calculation processing (calculation amount) is reduced compared to, for example, a case in which these Fourier transforms and convolution calculations are each performed two-dimensionally (corresponding to the case of the comparative example below).

Specifically, in the comparative example shown in FIG. 8A, when considering the case where a Discrete Fourier Transform (DFT) is used, the amount of calculation is as follows. That is, first, the order of calculation amount in one-dimensional DFT for pixels PX arranged two-dimensionally (N×N) is O(N ² ). Therefore, as described above, in this comparative example, since the Fourier transform is performed two-dimensionally (along the x-axis direction (row direction) and the y-axis direction (column direction): see the dashed arrow), the order of calculation amount in two-dimensional DFT is O(N ² ×N ² )=O(N ⁴ ). That is, in the example of FIG. 8A, since the pixels PX are arranged two-dimensionally (4×4), the order of calculation amount in two-dimensional DFT is O(4 ² ×4 ² )=O(4 ⁴ ).

In contrast, in the embodiment shown in FIG. 8B, when DFT is used in the same manner, the amount of calculation is as follows. That is, in this embodiment, the results of one-dimensional DFT (see dashed arrow) performed along the x-axis are added to each other for multiple sets (four sets) along the y-axis direction, so the order of the total amount of calculation is O(N ² ×N)=O(N ³ )=O(4 ³ ). That is, in this embodiment, the amount of calculation during DFT for the feature amount Fn is reduced while maintaining the calculation accuracy, compared to the above-mentioned comparative example. Note that such a reduction in the amount of calculation is not limited to the above-mentioned DFT, and is similar to other types of Fourier transform, such as fast Fourier transform (FFT).

As a result, this embodiment makes it possible to speed up the object position estimation process (shorten the processing time).

In addition, in this embodiment, the results of one-dimensional processing along the x-axis direction are added together for multiple sets along the y-axis direction to perform a Fourier transform and convolution operation on the two-dimensionally arranged feature quantity F, as shown below. In other words, by using such a specific calculation method, it is possible to actually achieve the reduction in the amount of calculation described above.

Furthermore, in this embodiment, when estimating the position of a tracked object, a correlation calculation is performed using the KCF described above, and a Fourier transform is used to convert the feature value F indicating the color space into a frequency space, which makes it possible to achieve even faster processing (even shorter processing time).

In addition, in this embodiment, the image region R in the captured image P is set using the distance information Iz or machine learning described above, so that such an image region R can be easily set, which also makes it possible to achieve further speedup (further shortening of processing time).

2. Modified Examples
Although the present disclosure has been described above by way of the embodiment, the present disclosure is not limited to this embodiment and various modifications are possible.

For example, the configuration (type, shape, arrangement, number, etc.) of each component in the vehicle 10 and the image processing device 12 is not limited to that described in the above embodiment. In other words, the configuration of each of these components may be of other types, shapes, arrangements, numbers, etc. Furthermore, the values, ranges, magnitude relationships, etc. of the various parameters described in the above embodiment are not limited to those described in the above embodiment, and may be other values, ranges, magnitude relationships, etc.

Specifically, for example, in the above embodiment, the stereo camera 11 is configured to capture an image in front of the vehicle 10, but this is not limited to the above configuration, and for example, the stereo camera 11 may be configured to capture an image of the side or rear of the vehicle 10. Also, in the above embodiment, an example in which the stereo camera 11 is used is described, but this is not limited to the above example, and for example, a monocular camera may be used to perform the various processes described in the above embodiment.

In addition, for example, in the above embodiment, specific examples of various processes performed in the vehicle 10 and the image processing device 12 are described, but the present invention is not limited to these specific examples. In other words, these various processes may be performed using other methods. Specifically, for example, the one-dimensional processing of the Fourier transform and convolution calculation described above is not limited to the method described in the above embodiment (one-dimensional processing along the x-axis direction), and may be, for example, one-dimensional processing along the y-axis direction or one-dimensional processing along other directions (such as diagonal directions). In addition, in the above embodiment, an example of a processing method using KCF is described, but the present invention is not limited to this example, and other processing methods may be used.

Furthermore, the series of processes described in the above embodiment may be performed by hardware (circuits) or by software (programs). When performed by software, the software is composed of a group of programs for causing a computer to execute each function. Each program may be, for example, pre-installed in the computer and used, or may be installed in the computer from a network or recording medium and used.

In addition, in the above embodiment, an example in which the image processing device 12 is provided in a vehicle has been described, but this is not limited to the example, and such an image processing device 12 may be provided, for example, in a moving body other than a vehicle or in a device other than a moving body.

Furthermore, the various examples described above may be applied in any combination.

The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

The present disclosure can also be configured as follows.
(1)
a region setting unit that sets one or more image regions in a captured image;
an extraction unit that extracts a feature amount of a two-dimensional arrangement included in the image region of the captured image;
a position estimation unit that estimates a position of a tracked object by performing a correlation calculation using a convolution operation on the feature amount between the image regions of the captured images obtained in previous and subsequent frame periods along a time axis,
The position estimation unit is
When performing the convolution operation, a Fourier transform is used for the feature amount,
the image processing device performing the Fourier transform and the convolution operation on the feature amounts arranged in the two-dimensional array, one dimension at a time.
(2)
The position estimation unit is
performing one-dimensional processing of the Fourier transform and the convolution operation along a first direction of the feature amount,
The results of the one-dimensional processing along the first direction are added to each other in a plurality of sets along a second direction different from the first direction,
The image processing device according to (1) above, further comprising: a processor for executing the Fourier transform and the convolution operation on the feature amounts arranged in the two-dimensional array.
(3)
The position estimation unit is
The correlation calculation is performed using a KCF (Kernelized Correlation Filter),
The image processing device according to (1) or (2) above, wherein the feature quantity indicating a color space is transformed into a frequency space by the Fourier transform.
(4)
The image processing device according to any one of (1) to (3) above, wherein the area setting unit sets the image area using distance information generated based on a left image and a right image as the captured image obtained from a stereo camera, or machine learning.
(5)
An image processing device according to any one of (1) to (4) above;
a vehicle control unit that controls the vehicle by using the estimated position of the tracked object provided by the position estimation unit.
(6)
one or more processors; and one or more memories communicatively coupled to the one or more processors,
The one or more processors:
Setting one or more image areas in a captured image;
extracting a feature amount of a two-dimensional arrangement included in the image region of the captured image;
and estimating a position of a tracked object by performing a correlation calculation using a convolution calculation on the feature amount between the image regions of the captured images obtained in the previous and next frame periods along a time axis, and
When performing the convolution operation, a Fourier transform is used for the feature amount,
the image processing device performing the Fourier transform and the convolution operation on the feature amounts arranged in the two-dimensional array, one dimension at a time.

10...vehicle, 11...stereo camera, 11L...left camera, 11R...right camera, 12...image processing device, 121...image memory, 122...distance information generation unit, 123...area setting unit, 124...feature extraction unit, 125...position estimation unit, 13...vehicle control unit, 19...windshield, 90...preceding vehicle, PL...left image, PR...right image, PIC...stereo image, P, P0 to Pn, Pn+1...captured image, R...image area, Iz...distance information, F, Fn, Fn+1...feature, Mc...correlation map, Pe...estimated position, x0...position coordinate, PX...pixel, t...time axis, t0 to tn...timing.

Claims (5)

a region setting unit that sets one or more image regions in a captured image;
an extraction unit that extracts a feature amount of a two-dimensional arrangement included in the image region of the captured image;
a position estimation unit that estimates a position of a tracked object by performing a correlation calculation using a convolution operation on the feature amount between the image regions of the captured images obtained in previous and subsequent frame periods along a time axis,
The position estimation unit is
When performing the convolution operation, a Fourier transform is used for the feature amount,
the image processing device performing the Fourier transform and the convolution operation on the feature amounts arranged in the two-dimensional array, one dimension at a time. The position estimation unit is
performing one-dimensional processing of the Fourier transform and the convolution operation along a first direction of the feature amount,
The results of the one-dimensional processing along the first direction are added to each other in a plurality of sets along a second direction different from the first direction,
The image processing device according to claim 1 , further comprising: a processor configured to execute the Fourier transform and the convolution operation on the feature amounts in the two-dimensional arrangement. The position estimation unit is
The correlation calculation is performed using a KCF (Kernelized Correlation Filter),
The image processing apparatus according to claim 1 , wherein the feature quantity indicating a color space is transformed into a frequency space by the Fourier transform. The image processing device according to claim 1 , wherein the region setting unit sets the image region by using distance information generated based on a left image and a right image as the captured image obtained from a stereo camera, or by using machine learning. An image processing device according to any one of claims 1 to 4,
a vehicle control unit that controls the vehicle by using the estimated position of the tracked object provided by the position estimation unit.

Images