JP7548183B2 - MOVING OBJECT DETERMINATION DEVICE AND MOVING OBJECT DETERMINATION PROGRAM - Google Patents

Description

The disclosure of this specification relates to a moving object determination device and a moving object determination program.

Conventionally, there is known a technique for capturing images of the surroundings of a vehicle with a camera and determining whether a target object included in the captured image is moving. For example, Patent Document 1 describes a moving object determination device that compares an optical flow shown as a time series of the positions of feature points with an optical flow of a spatial feature point shown as a time series of the positions of the spatial feature points corresponding to the feature points, to determine whether a target object represented by the feature points is moving.

JP 2018-185694 A

In object detection systems, determining whether a target in an image is moving or stationary plays an important role. Existing technologies, including the technology described in Patent Document 1, are configured to determine whether a target is moving or stationary based on a single parameter such as optical flow. As a result, there are concerns that the determination may not be made accurately, and there is thought to be room for improvement.

The present invention has been made in consideration of the above problems, and its purpose is to provide a moving object determination device and a moving object determination program that can accurately determine whether a target is moving or stationary.

The present invention relates to
A moving object determination device that is applied to a vehicle equipped with a camera that captures images of the vehicle's surroundings and performs a moving/stationary determination based on images captured by the camera to determine whether or not a target included in the images is moving, comprising:
an acquisition unit that acquires the images at a predetermined period;
a recognition unit that recognizes the same target by a plurality of target recognition methods different from each other using the image acquired by the acquisition unit;
an existence reliability calculation unit that calculates, for each of the plurality of target recognition methods, an existence reliability indicating a likelihood that the target exists as a moving object candidate, based on a recognition result of the target in each of the plurality of target recognition methods and a predetermined weighting coefficient determined for each of the target recognition methods;
an integrated reliability calculation unit that integrates the presence reliability calculated for each of the target recognition methods and calculates an integrated reliability of the target;
A determination unit that performs a moving/stationary determination based on the integrated reliability;
The present invention is characterized by comprising:

According to the above moving object determination device, the same target is recognized by a plurality of target recognition methods from images acquired at a predetermined cycle, and an existence reliability indicating the likelihood that the target exists as a moving object candidate is calculated for each target recognition method based on the target recognition results in each of the plurality of different target recognition methods and a predetermined weighting coefficient determined for each target recognition method. Then, an integrated reliability of the target is calculated by integrating the existence reliability calculated for each target recognition method, and a moving/stationary determination is performed based on the integrated reliability. In this case, the accuracy of the moving/stationary determination can be improved by using the recognition results of the plurality of different target recognition methods and integrating them while performing the moving/stationary determination. In addition, since a weighting coefficient is determined for each target recognition method, the integrated reliability of the target can be properly calculated while reflecting which of the plurality of target recognition methods the target was recognized by, unlike the case where a uniform weighting coefficient is determined for each target recognition method. As a result, the moving/stationary determination of the target can be performed with high accuracy.

The present invention also provides a method for producing a method for manufacturing a semiconductor device comprising the steps of:
A moving object determination program that is applied to a vehicle equipped with a camera that photographs the surroundings of the vehicle and is executed by a control device to perform a moving/stationary determination as to whether a target included in images photographed by the camera is moving or not, the program comprising:
An acquisition step of acquiring the image at a predetermined period;
a recognition step of recognizing the same target by a plurality of target recognition methods different from each other using the images acquired in the acquisition step;
an existence reliability calculation step of calculating, for each of the plurality of target recognition methods, an existence reliability indicating a likelihood that the target exists as a moving object candidate, based on a recognition result of the target in each of the plurality of target recognition methods and a predetermined weighting coefficient determined for each of the target recognition methods;
an integrated reliability calculation step of integrating the presence reliability calculated for each of the target recognition methods and calculating an integrated reliability of the target;
A determination step of determining whether or not the movement or stationary state is performed based on the integrated reliability;
The present invention is characterized by comprising:

The above moving object determination program can accurately determine whether a target is moving or stationary, just like the moving object determination device.

FIG. 1 is a diagram showing an object detection system for a vehicle including a moving object determination device. FIG. 2 is a diagram showing the configuration of an image processing unit. FIG. 4 is a diagram showing the relationship between a target recognition parameter and instantaneous reliability. FIG. 4 is a diagram showing weighting coefficients for each target recognition method. FIG. 13 is a diagram showing instantaneous value data, continuity reliability, and integrated reliability. 13 is a flowchart showing an object detection process. 4 is a time chart for explaining an outline of a target recognition takeover process; 11 is a flowchart showing a takeover process of target recognition.

First Embodiment
1 shows an object detection system that is mounted on a vehicle and detects objects around the vehicle. The object detection system includes a camera 30 and a moving object determination device 10. The camera 30 includes a front camera 31, a rear camera 32, a left camera 33, and a right camera 34. The front camera 31, the rear camera 32, the left camera 33, and the right camera 34 are each a monocular digital camera. In the following, the front camera 31, the rear camera 32, the left camera 33, and the right camera 34 may be referred to as the cameras 31, 32, 33, and 34 for simplicity. The cameras 31, 32, 33, and 34 each employ a wide-angle lens with a viewing angle set to 180 degrees.

The front camera 31 is attached, for example, to the front end of the vehicle in the direction of travel (e.g., the front grille) and captures the area ahead of the vehicle in the direction of travel. The rear camera 32 is attached to the rear end of the vehicle in the direction of travel and captures the area behind the vehicle. The left side camera 33 is attached, for example, to the left door mirror of the vehicle and captures the left side of the vehicle in the width direction of the vehicle. The right side camera 34 is attached, for example, to the right door mirror of the vehicle and captures the right side of the vehicle in the width direction of the vehicle.

The cameras 31 to 34 each capture images of the vehicle's surroundings in chronological order, and output image data representing the captured images to the video signal input unit 11 of the moving object determination device 10. The video signal input unit 11 acquires the image data output from each of the cameras 31, 32, 33, and 34, and outputs the acquired image data to the image processing unit 13.

The moving object determination device 10 is mainly composed of a microcomputer and includes a CPU, ROM, RAM, I/O, and an in-vehicle communication bus 40 that connects these components. Specifically, the moving object determination device 10 includes a video signal input unit 11, a communication interface (communication I/F) 12, an image processing unit 13, a memory 15, and a power supply unit 16.

The communication I/F 12 is connected to a vehicle speed sensor, steering angle sensor, sonar, shift range, etc. via the in-vehicle communication bus 40, and acquires vehicle information such as the vehicle speed, steering angle, shift range, sonar information, etc. More specifically, the vehicle speed sensor inputs a signal corresponding to the host vehicle speed, which is the speed of the vehicle. The steering angle sensor inputs a signal corresponding to the steering angle of the vehicle based on the steering operation by the driver of the vehicle. The sonar inputs a signal indicating the results of the exploration of obstacles around the vehicle using ultrasonic waves as exploration waves. The shift range inputs range information of the automatic transmission.

The memory 15 corresponds to a storage unit and includes a RAM, a ROM, a writable non-volatile storage medium, etc. The memory 15 stores the computer program executed by the image processing unit 13.

When the vehicle's power switch is turned on, the power supply unit 16 supplies power to the image processing unit 13 so that the image processing unit 13 can execute a program.

The image processing unit 13 performs object detection processing by executing a computer program pre-stored in the memory 15.

Figure 2 is a diagram showing the configuration of the image processing unit 13. The image processing unit 13 includes an acquisition unit 21, a target recognition unit 22, an instantaneous value calculation unit 23, a continuing reliability calculation unit 24, an integrated reliability calculation unit 25, and a determination unit 26.

The acquisition unit 21 acquires captured images (frames) of the surroundings of the vehicle input to the video signal input unit 11 at a predetermined interval. This allows images captured by each of the cameras 31 to 34 to be acquired. The acquisition unit 21 also acquires vehicle information input to the communication I/F 12.

The target recognition unit 22 uses the images acquired by the acquisition unit 21 to recognize targets using a number of target recognition methods. In this embodiment, the target recognition unit 22 is configured to perform target recognition using four target recognition methods, and has a type recognition unit 22A, an optical flow recognition unit 22B, an SS (Semantic Segmentation) recognition unit 22C, and an SfM (Structure from Motion) recognition unit 22D as recognition units that perform individual target recognition.

The type recognition unit 22A is a recognition unit that recognizes targets by determining the type of targets contained in an image, and calculates a type score indicating the likelihood of the target type as a target recognition parameter. For example, the type recognition unit 22A predetermines multiple type determination targets, and estimates a type score for each target candidate present in the image using learning data that defines the recognition targets. The form of the learning data is arbitrary, but it is preferable to use learning data generated by deep learning, for example. The type score is preferably expressed as the probability that each target falls into which type.

Specifically, for example, four types of types are predefined as type determination targets: four-wheeled vehicles, two-wheeled vehicles, pedestrians, and other objects. Then, for each target candidate present in the image, the type recognition unit 22A calculates the probability that it is a four-wheeled vehicle, a two-wheeled vehicle, a pedestrian, or another object, and calculates the probability of which type each target candidate in the image is as a type score. When a pedestrian is captured in the image, the probabilities are calculated as, for example, "four-wheeled vehicles: 10%," "two-wheeled vehicles: 3%," "pedestrians: 85%," and "other objects: 2%." In this case, since the probability of a pedestrian is the highest, the target candidate in the image is type determined to be a pedestrian, and a type score is calculated according to the ratio of each type determination target.

The optical flow recognition unit 22B is a recognition unit that recognizes the movement of targets contained in an image using an optical flow density recognition method, and calculates optical flow density as a target recognition parameter. Optical flow is a vector representation of the movement of feature points of a target, and the optical flow recognition unit 22B calculates optical flow from feature points of past targets and feature points of the current target, and calculates the density of the movement vectors of the feature points as the optical flow density. The optical flow density is correlated with the distance from the camera that captured the image to the target, and as the distance from the camera to the target becomes closer, the number of feature points increases, and the optical flow density becomes a high value.

The SS recognition unit 22C is a recognition unit that recognizes targets by associating labels and categories with all pixels in an image, and calculates a target area ratio according to the area occupied by classified targets in the image as a target recognition parameter. SS is an algorithm that uses deep learning, and is used to recognize groups of pixels that form characteristic categories. The SS recognition unit 22C creates an attribute image in which targets included in an image are classified into either movable or non-movable classes, and calculates the target area ratio of the classified targets to the entire image.

The SfM recognition unit 22D is a recognition unit that performs target recognition using a three-dimensional recognition method (SfM), which creates a three-dimensional estimated image that shows the three-dimensional positions of feature points in an image and performs target recognition based on the three-dimensional estimated image, and calculates the number of points for each target in the three-dimensional estimated image as a target recognition parameter. SfM is a technology that restores the three-dimensional shape of a target by generating a point cloud that represents the depth of the target based on multiple images taken from different angles, i.e., multiple images with angle differences.

Next, the instantaneous value calculation unit 23 will be described. For each image acquired in time series, the instantaneous value calculation unit 23 calculates an instantaneous confidence level Tconf, which is the instantaneous reliability of the presence of the target, for the target recognition result of each target recognition method in the target recognition unit 22, and calculates instantaneous value data by multiplying the instantaneous confidence level Tconf by a weighting factor W determined for each target recognition method. That is, the instantaneous value calculation unit 23 has calculation units 23A to 23D provided for each recognition unit 22A to 22D, and each of these calculation units 23A to 23D calculates instantaneous confidence levels Tconf_A to Tconf_D and weighting factors WA to WD for each target recognition method, and calculates instantaneous value data by multiplying these values.

In this embodiment, the presence confidence level is calculated as an index showing the likelihood that a target exists as a moving object candidate, and the presence confidence level at each moment (instantaneous presence confidence level) is the instantaneous confidence level Tconf. The continuous presence confidence level of the target is the continuous confidence level Cconf, and the presence confidence level obtained by integrating the continuous confidence levels Cconf for each target recognition method is the integrated confidence level Iconf.

Each calculation unit 23A to 23D calculates instantaneous confidences Tconf_A to Tconf_D from each target recognition parameter for each target recognition method, for example, based on the relationships shown in Figures 3(a) to 3(d). The instantaneous confidences Tconf_A to Tconf_D for each target recognition method may be calculated within the same numerical range, for example, with a minimum value of 0 and a maximum value of 3.

Each calculation unit 23A to 23D calculates the weighting coefficients WA to WD for each target recognition method based on the relationships shown in, for example, Figures 4(a) to (d). In Figures 4(a) to (d), the values of the weighting coefficients WA to WD are determined for each target recognition method according to whether the vehicle is stationary or moving, and whether the target to be recognized is a four-wheeled vehicle, a two-wheeled vehicle, or a pedestrian. Information on the stationary or moving state of the vehicle and the distance to the target may be sequentially acquired together with the image. Whether the target to be recognized is a four-wheeled vehicle, a two-wheeled vehicle, or a pedestrian may be determined using the type determination result by the type recognition unit 22A. In Figures 4(a) to (d), the values of the weighting coefficients WA to WD can be set arbitrarily and are not limited to the values shown in the figures. Below, we will explain the calculation method for the instantaneous confidences Tconf_A to Tconf_D and the calculation method for the weighting coefficients WA to WD for each calculation unit 23A to 23D.

In the following explanation, the "n" added to the end of each instantaneous confidence Tconf_A to Tconf_D means the number of image frames from when recognition of the target being recognized this time started to the current time. For example, "Tconf_An" indicates the current value of instantaneous confidence Tconf_A. For example, if the number of frames from the start of target recognition to the current time is 50, then n = 50, and if it is 100, then n = 100. The same applies to the weighting coefficients WAn to WDn.

The calculation unit 23A uses the relationship in FIG. 3(a) to calculate the instantaneous confidence Tconf_An based on the type score calculated by the type recognition unit 22A. According to FIG. 3(a), the higher the type score, the higher the value calculated as the instantaneous confidence Tconf_An.

The calculation unit 23A also uses the relationship in FIG. 4(a) to set a weighting factor WAn for the instantaneous reliability Tconf_An of the type score. In this case, the weighting factor WAn is set to "1" under all conditions, regardless of whether the vehicle is stationary or moving, or whether the target is a four-wheeled vehicle, a two-wheeled vehicle, or a pedestrian.

Then, the calculation unit 23A calculates instantaneous value data (Tconf_An x WAn) regarding the target type determination by multiplying the instantaneous confidence Tconf_An of the type score, which is a target recognition parameter, by the weighting coefficient WAn.

The calculation unit 23B uses the relationship in FIG. 3(b) to calculate the instantaneous confidence Tconf_Bn based on the optical flow density calculated by the optical flow recognition unit 22B. According to FIG. 3(b), the higher the optical flow density, the higher the value of the instantaneous confidence Tconf_Bn that is calculated.

The calculation unit 23B also uses the relationship in FIG. 4(b) to set a weighting factor WBn for the instantaneous reliability Tconf_Bn of the optical flow density. In this case, the weighting factor WBn is set to a different value depending on whether the host vehicle is stationary or moving, and is also set to a different value depending on the distance from the host vehicle (on-board camera) to the target. Specifically, the weighting factor WBn is set to a larger value when the host vehicle is stationary compared to when the host vehicle is moving. The weighting factor WBn is set to a smaller value the longer the distance from the host vehicle to the target. The weighting factor WBn is the same value for all cases, whether the host vehicle is stationary or moving, whether the vehicle is a four-wheeled vehicle, a two-wheeled vehicle, or a pedestrian.

In comparison with the weighting coefficient WAn of the category score, the weighting coefficient WBn is set to a value smaller than the weighting coefficient WAn except when the vehicle is stationary and the distance to the target is within the range close to the vehicle (0 to 5 m).

Then, the calculation unit 23B calculates instantaneous value data (Tconf_Bn×WBn) regarding target recognition by optical flow by multiplying the instantaneous confidence Tconf_Bn of the optical flow density, which is a target recognition parameter, by the weighting coefficient WBn.

The calculation unit 23C uses the relationship in FIG. 3(c) to calculate the instantaneous confidence Tconf_Cn based on the target area ratio calculated by the SS recognition unit 22C. According to FIG. 3(c), the larger the target area ratio, the higher the value calculated as the instantaneous confidence Tconf_Cn.

The calculation unit 23C also uses the relationship in FIG. 4(c) to set a weighting factor WCn for the instantaneous reliability Tconf_Cn of the target area ratio. In this case, the weighting factor WCn is the same when the host vehicle is stationary and when it is moving. In addition, in this embodiment, the target recognition result by the SS is not used for four-wheeled vehicles, and if the target is a four-wheeled vehicle, the weighting factor WCn is set to 0.

In comparison with the weighting coefficient WAn of the category score, the weighting coefficient WCn is set to a value smaller than the weighting coefficient WAn for four-wheeled vehicles, and is set to the same value as the weighting coefficient WAn for two-wheeled vehicles and pedestrians.

Then, the calculation unit 23C calculates instantaneous value data (Tconf_Cn x WCn) regarding target recognition by the SS by multiplying the instantaneous reliability Tconf_Cn of the target area ratio, which is a target recognition parameter, by the weighting coefficient WCn.

The calculation unit 23D uses the relationship in FIG. 3(d) to calculate the instantaneous confidence Tconf_Dn based on the number of point clouds calculated by the SfM recognition unit 22D. According to FIG. 3(d), the larger the number of point clouds, the higher the value calculated as the instantaneous confidence Tconf_Dn.

The calculation unit 23D also uses the relationship in FIG. 4(d) to set a weighting factor WDn for the instantaneous reliability Tconf_Dn of the point cloud number. In this case, the weighting factor WDn has different values when the host vehicle is stationary and when it is moving, and is set to WDn=0 when the host vehicle is stationary and WDn=1 when the host vehicle is moving.

In comparison with the weighting coefficient WAn of the category score, the weighting coefficient WDn is set to a value smaller than the weighting coefficient WAn when the vehicle is stationary, and is set to the same value as the weighting coefficient WAn when the vehicle is moving.

Then, the calculation unit 23D calculates instantaneous value data (Tconf_Dn x WDn) regarding target recognition by SfM by multiplying the instantaneous confidence Tconf_Dn of the point cloud number, which is a target recognition parameter, by the weighting coefficient WDn.

In each of the calculation units 23A to 23D, the instantaneous reliabilities Tconf_A to Tconf_D and the weighting factors WA to WD are calculated for each image acquired in time series, and the instantaneous reliabilities Tconf_A to Tconf_D and the weighting factors WA to WD are sequentially stored in the memory 15 with the frame numbers in time series assigned. In this embodiment, the current frame number is n, and the image numbers from the current time to the past are 1 to n-1.

Figure 5 shows the instantaneous value data (instantaneous reliabilities Tconf_A to Tconf_D and weighting coefficients WA to WD) calculated by each calculation unit 23A to 23D of the instantaneous value calculation unit 23, as well as the continuous reliabilities Cconf_A to Cconf_D and the integrated reliability Iconf, which will be described later.

In FIG. 5, (a) shows instantaneous confidences Tconf_A1 to Tconf_An and weighting factors WA1 to WAn as time series data related to type scores. (b) shows instantaneous confidences Tconf_B1 to Tconf_Bn and weighting factors WB1 to WBn as time series data related to optical flow density. (c) shows instantaneous confidences Tconf_C1 to Tconf_Cn and weighting factors WC1 to WCn as time series data related to target area ratios. (d) shows instantaneous confidences Tconf_D1 to Tconf_Dn and weighting factors WD1 to WDn as time series data related to SfM point cloud counts.

Next, the continuity reliability calculation unit 24 will be described. The continuity reliability calculation unit 24 calculates an integrated value of instantaneous value data for each target recognition method within a recognition period, which is the period from the start of target recognition, and calculates a continuity reliability Cconf, which is the reliability of the continued presence of the target, based on the integrated value. In other words, the continuity reliability calculation unit 24 calculates the continuity reliability Cconf based on the integrated value of instantaneous value data of the same target accumulated in time series.

The continuing reliability calculation unit 24 has calculation units 24A to 24D provided for the calculation units 23A to 23D of the instantaneous value calculation unit 23, and each of these calculation units 24A to 24D calculates the continuing reliability Cconf_A to Cconf_D based on the integrated value of the instantaneous value data for each target recognition method. The instantaneous value calculation unit 23 and the continuing reliability calculation unit 24 correspond to the "presence reliability calculation unit."

The calculation unit 24A calculates the continuity reliability Cconf_A for the target type determination by a weighted average. Specifically, the calculation unit 24A calculates the integrated value of the time series of instantaneous value data (Tconf_A x WA) and divides the integrated value by the number of instantaneous value data to calculate the continuity reliability Cconf_A. For example, the calculation unit 24A calculates the continuity reliability Cconf_A using the following formula (1). In formula (1), nvalid is the number of valid frames, i.e., the number of valid images among each image in the time series.

算出部２４Ａは、上記式（１）の「Ｔｃｏｎｆｉ」に「Ｔｃｏｎｆ＿Ａｉ」を代入し、「Ｗｉ」に「ＷＡｉ」を代入することで、継続信頼度Ｃｃｏｎｆ＿Ａを算出する。

The calculation unit 24A calculates the continuation reliability Cconf_A by substituting "Tconf_Ai" for "Tconfi" and "WAi" for "Wi" in the above formula (1).

Similarly, the other calculation units 24B to 24D calculate the continuation reliability Cconf_B to Cconf_D by weighted averaging. For target recognition by optical flow, calculation unit 24B uses the above formula (1) to calculate the continuation reliability Cconf_B based on the time series integrated value of the instantaneous value data (Tconf_B x WB). For target recognition by SS, calculation unit 24C uses the above formula (1) to calculate the continuation reliability Cconf_C based on the time series integrated value of the instantaneous value data (Tconf_C x WC). For target recognition by SfM, calculation unit 24D uses the above formula (1) to calculate the continuation reliability Cconf_D based on the time series integrated value of the instantaneous value data (Tconf_D x WD).

For example, in target recognition using SfM, in principle, it is impossible to calculate instantaneous value data when the vehicle is stationary. Therefore, when a stationary period of the vehicle is included in the target recognition period, the number of valid frames nvalid is the total number of frames in the target recognition period minus the number of frames when the vehicle is stationary.

The continuous reliability Cconf of each target recognition method may be calculated in the same numerical range as the instantaneous reliability Tconf, for example, with a minimum value of 0 and a maximum value of 3. An integer conversion process may be performed on the continuous reliability Cconf, for example, the continuous reliability Cconf may be calculated as an integer value in the range of 0 to 3.

in particular,
If 0≦Cconf<3/4, then Cconf=0;
If 3/4≦Cconf<6/4, then Cconf=1;
If 6/4≦Cconf<9/4, then Cconf=2;
If 9/4≦Cconf≦3, then set Cconf=3.

Next, the integrated reliability calculation unit 25 will be described. The integrated reliability calculation unit 25 integrates the continuing reliabilities Cconf_A to Cconf_D calculated for each target recognition method to calculate the integrated reliability Iconf of the target. In this case, the integrated reliability calculation unit 25 calculates the integrated reliability Iconf by applying a predetermined weighting to the continuing reliabilities Cconf_A to Cconf_D calculated by each calculation unit 24A to 24D of the continuing reliability calculation unit 24. Specifically, the integrated reliability Iconf is calculated using the following formula (2).

In the following formula (2), m is a number corresponding to the number of target recognition methods used in the system, and m = 4 when four target recognition methods are used. WX is a weighting coefficient for allocating weights to each of the continuity reliabilities Cconf_A to Cconf_D of the multiple target recognition methods. The weighting coefficient WX is determined according to the number of target recognition methods applied to the calculation of the integrated reliability Iconf.

本実施形態では、各物標認識手法における継続信頼度Ｃｃｏｎｆ＿Ａ～Ｃｃｏｎｆ＿Ｄの重み付けを均等にしており、４つの物標認識手法が用いられる場合には、各継続信頼度Ｃｃｏｎｆ＿Ａ～Ｃｃｏｎｆ＿Ｄに乗算される重み係数ＷＸは一律で「１／４」とされる。ただし、例えば図４（ａ）～（ｄ）において、重み係数Ｗ（瞬時値データ）が常に０になるパターン（すなわち、継続信頼度Ｃｃｏｎｆが０になるパターン）に当てはまる場合には、４つの物標認識手法のうち常にＷ＝０になる物標認識手法を除外した残りの物標認識手法の数に応じて重み係数ＷＸが定められるとよい。

In this embodiment, the weighting of the continuity reliabilities Cconf_A to Cconf_D in each target recognition method is uniform, and when four target recognition methods are used, the weighting factor WX multiplied by each of the continuity reliabilities Cconf_A to Cconf_D is uniformly set to "1/4." However, for example, in the case of a pattern in which the weighting factor W (instantaneous value data) is always 0 (i.e., a pattern in which the continuity reliability Cconf is 0) in Figures 4(a) to 4(d) applies, it is preferable to determine the weighting factor WX according to the number of remaining target recognition methods excluding the target recognition methods among the four target recognition methods for which W = 0 is always satisfied.

A specific example is described below. For example, if the vehicle is constantly moving and the target object is a motorcycle or a pedestrian, the weighting factors WA to WD in the four target recognition methods will all be values other than 0 (see Figures 4(a) to (d)). Therefore, the weighting factor WX multiplied by each of the continuity reliabilities Cconf_A to Cconf_D is uniformly set to "1/4", and the product of each of the continuity reliabilities Cconf_A to Cconf_D and "1/4" is added together to form the integrated reliability Iconf.

In contrast, if the vehicle is always moving and the target is a four-wheeled vehicle, the weighting factor WC is 0 (see FIG. 4(c)). Therefore, the weighting factor WX multiplied by the continuation reliability Cconf_C of the target recognition result by the SS is set to 0, and the weighting factors WX multiplied by the remaining three continuation reliability Cconf_A, Cconf_B, and Cconf_D are each set to "1/3". In addition, if the vehicle is always stationary and the target is a four-wheeled vehicle, the weighting factors WC and WD are 0 (see FIG. 4(c) and (d)). Therefore, the weighting factors WX multiplied by the continuation reliability Cconf_C and Cconf_D of the target recognition result by the SS and SfM are each set to 0, and the weighting factors WX multiplied by the remaining two continuation reliability Cconf_A and Cconf_B are each set to "1/2".

The integrated confidence Iconf may be calculated in the same numerical range as the instantaneous confidence Tconf, for example with a minimum value of 0 and a maximum value of 3. An integer conversion process may be performed on the integrated confidence Iconf, for example, the integrated confidence Iconf may be calculated as an integer value in the range of 0 to 3. The integer conversion process has been described above, so a description thereof will be omitted here.

The determination unit 26 performs a moving/stationary determination of the target based on the integrated reliability Iconf. When the integrated reliability Iconf is calculated as an integer value in the range of 0 to 3, if the integrated reliability Iconf is 3, the target is determined to be a moving object, and if the integrated reliability Iconf is 0 to 2, the target is determined to be a stationary object. Note that the value of the integrated reliability Iconf that determines the target to be a moving object and the value that determines the target to be a stationary object can be set arbitrarily and is not limited to the relationship shown above.

In the moving object detection system shown in FIG. 1, the image processing unit 13 of the moving object determination device 10 executes a moving object detection program, which is a computer program stored in the memory 15, to detect moving objects around the vehicle and control the vehicle. FIG. 6 shows a flowchart of the object detection process executed by the image processing unit 13. This process is executed continuously at a predetermined interval. This process is executed for each image captured by each camera 30 (cameras 31 to 34).

In step S11, image information and vehicle information are acquired. At this time, an image captured by the camera 30 is acquired as image information, and the speed of the vehicle and the distance from the vehicle to a target object, etc. are acquired as vehicle information. Then, the process proceeds to step S12.

In step S12, a type determination is performed for the target included in the image acquired in step S11, and a type score is calculated as a target recognition parameter for the type determination. This process corresponds to the type score calculation process performed by the type recognition unit 22A shown in FIG. 2.

In step S13, the relative position and moving speed of the target determined in step S12 with respect to the vehicle are calculated. The position and speed of the target may be calculated using the tracking results based on optical flow.

In step S14, the optical flow density is calculated as a target recognition parameter of the optical flow for the target whose type was determined in step S12. This process corresponds to the optical flow density calculation process by the optical flow recognition unit 22B shown in FIG. 2.

In step S15, the SS performs target recognition for the target determined in step S12, and calculates the target area ratio as a target recognition parameter of the SS. This process corresponds to the target area ratio calculation process by the SS recognition unit 22C shown in FIG. 2.

In step S16, it is determined whether the vehicle is moving. If the vehicle is moving, the process proceeds to step S17. If the vehicle is not moving, step S17 is skipped.

In step S17, target recognition by SfM is performed for the target whose type was determined in step S12, and the number of point clouds is calculated as a target recognition parameter of SfM. This process corresponds to the calculation process of the number of point clouds by the SfM recognition unit 22D shown in FIG. 2.

Then, in step S18, instantaneous reliabilities Tconf_A to Tconf_D and weighting factors WA to WD are calculated for each of the target recognition parameters by the above-mentioned target recognition methods, and instantaneous value data is calculated from the instantaneous reliabilities Tconf_A to Tconf_D and weighting factors WA to WD. This process corresponds to the calculation process of the instantaneous value data (instantaneous reliability Tconf × weighting factor W) by the instantaneous value calculation unit 23 (calculation units 23A to 23D) shown in FIG. 2.

Specifically, the following processes are carried out in step S18.
An instantaneous confidence Tconf_An and a weighting factor WAn are calculated for the type score, which is a target recognition parameter for type determination, and instantaneous value data regarding the type determination is calculated by multiplying the instantaneous confidence Tconf_An and the weighting factor WAn.
An instantaneous confidence Tconf_Bn and a weighting factor WBn are calculated for the optical flow density, which is a target recognition parameter of the optical flow, and instantaneous value data regarding the optical flow is calculated by multiplying the instantaneous confidence Tconf_Bn by the weighting factor WBn.
An instantaneous reliability Tconf_Cn and a weighting factor WCn are calculated for the target area ratio, which is a target recognition parameter for the SS, and instantaneous value data for the SS is calculated by multiplying the instantaneous reliability Tconf_Cn by the weighting factor WCn.
An instantaneous reliability Tconf_Dn and a weighting factor WDn are calculated for the point cloud number, which is a target recognition parameter of SfM, and instantaneous value data for SfM is calculated by multiplying the instantaneous reliability Tconf_Dn by the weighting factor WDn.

Then, in step S19, the continuing reliability Cconf_A to Cconf_D are calculated based on the average value (weighted average) of the instantaneous value data for each target recognition method within the target recognition period. This process corresponds to the calculation process of the continuing reliability Cconf by the continuing reliability calculation unit 24 (calculation units 24A to 24D) shown in FIG. 2. Specifically, the continuing reliability Cconf_A, Cconf_B, Cconf_C, and Cconf_D are calculated using the above formula (1).

Then, in step S20, the continuity confidence Cconf for each target recognition method is integrated to calculate the integrated confidence Iconf. This process corresponds to the calculation process of the integrated confidence Iconf by the integrated confidence calculation unit 25 shown in FIG. 2. Specifically, the integrated confidence Iconf is calculated using the above formula (2).

If the continuity reliability Cconf by each target recognition method includes a continuity reliability Cconf by a target recognition method whose weighting coefficient is always 0, the continuity reliability Cconf whose weighting coefficient is always 0 is excluded from the integration. In this case, the remaining continuity reliability Cconf that is not excluded from the integration is weighted equally to calculate the integrated reliability Iconf.

Then, in step S21, a determination is made as to whether the target is moving or stationary based on the integrated reliability Iconf. In this case, as described above, if the integrated reliability Iconf is 3, the target is determined to be moving, and if the integrated reliability Iconf is 0 to 2, the target is determined to be stationary.

When the camera 30 is equipped with a first camera and a second camera that have different shooting directions and partially overlapping shooting ranges, it is possible that the same target (e.g., a pedestrian) is photographed by the first camera and the second camera. For example, a configuration in which the shooting ranges of the front camera 31 and the right camera 34 partially overlap each other is conceivable. In this case, one of the front camera 31 and the right camera 34 corresponds to the first camera and the other corresponds to the second camera. In other combinations such as the front camera 31 and the left camera 33, one corresponds to the first camera and the other corresponds to the second camera.

In this way, when the shooting ranges of the two cameras overlap, for example, it is conceivable that the target will first be photographed by only the first camera out of the first and second cameras, and then the state will transition to one in which the target is photographed by both the first and second cameras. It is also conceivable that thereafter the state will transition to one in which the target is photographed by only the first camera or only the second camera. The following describes the process of taking over target recognition when the camera photographing the target is switched.

Figure 7 is a time chart for explaining an overview of the target recognition handover process. In Figure 7, at timing t1, target recognition begins using the first camera image. Timing t1 and onwards is the first recognition period in which the target is recognised using the first camera image. After timing t1, the frame number n1 of the first camera image gradually increases from 0. During the first recognition period, target recognition for each target recognition method using the first camera image, calculation of instantaneous value data, and calculation of the continuation reliability Cconf are performed sequentially. In addition, an integrated reliability Iconf is calculated based on the continuation reliability Cconf of each target recognition method, and a moving/stationary determination of the target is made based on the integrated reliability Iconf.

Meanwhile, at time t2, target recognition begins using the second camera image. The period from time t2 onwards is the second recognition period in which the target is recognised using the second camera image. After time t2, the frame number n2 of the second camera image gradually increases from 0. During the second recognition period, target recognition for each target recognition method using the second camera image, calculation of instantaneous value data, and calculation of the continuing confidence Cconf are performed sequentially. In this case, at time t2 (the start of the second recognition period), target recognition using the first camera image continues, and after time t2, the calculation of the continuing confidence Cconf in the first recognition period and the calculation of the continuing confidence Cconf in the second recognition period are performed in an overlapping manner.

However, if the recognition periods overlap, the integrated confidence Iconf is calculated using the continuous confidence Cconf calculated for each image from the first camera and the second camera, whichever has the larger number of image frames. In the illustrated situation, from time t2 onwards, the integrated confidence Iconf is calculated using the continuous confidence Cconf calculated from the first camera image.

After that, at timing t3, the target leaves the shooting range of the first camera, and the first recognition period ends. That is, target recognition for each target recognition method using the first camera image, calculation of instantaneous value data, and calculation of the continuity reliability Cconf end. At this time, since the continuity reliability Cconf is being calculated using the second camera image for the target to be recognized, the calculation of the integrated reliability Iconf and the determination of whether the target is moving or stationary based on the integrated reliability Iconf continue by using the current continuity reliability Cconf using the second camera image instead of the continuity reliability Cconf using the first camera image.

After that, at timing t4, target recognition is started again using the first camera image, and from timing t4 onwards, the frame number n1 of the first camera image again gradually increases from 0. Also, similar to the period from timing t1 to t3, target recognition for each target recognition method using the first camera image, calculation of instantaneous value data, and calculation of the continuing reliability Cconf are performed sequentially. Then, when the target moves out of the shooting range of the second camera at timing t5, the continuing reliability Cconf calculated using the first camera image is used instead of the continuing reliability Cconf calculated using the second camera image. As a result, calculation of the integrated reliability Iconf and determination of whether the target is moving or stationary based on the integrated reliability Iconf are performed again continuously.

Figure 8 is a flowchart showing the target recognition handover process executed by the image processing unit 13 of the moving object determination device 10. Here, we will explain the case where the combination of the front camera 31 and the right camera 34 is the first camera and the second camera.

Figure 8 includes a process for calculating the continuation confidence Cconf by four target recognition methods using an image from the front camera 31 (steps S34, S36) and a process for calculating the continuation confidence Cconf by four target recognition methods using an image from the right camera 34 (steps S35, S37), and each of these processes corresponds to the processes of steps S11 to S19 in Figure 6. Also, in Figure 8, the process for calculating the integrated confidence Iconf based on the continuation confidence Cconf (steps S39, S40) corresponds to the process of step S20 in Figure 6. For convenience of explanation, the continuation confidence Cconf for each target recognition method calculated from the front camera image is referred to as the "first continuation confidence Cconf1", and the continuation confidence Cconf for each target recognition method calculated from the right camera image is referred to as the "second continuation confidence Cconf2". In addition, the integrated reliability Iconf calculated using the first continuous reliability Cconf1 is referred to as the "first integrated reliability Iconf1," and the integrated reliability Iconf calculated using the second continuous reliability Cconf2 is referred to as the "second integrated reliability Iconf2."

In step S31, it is determined whether or not a target is recognized by the image from the front camera 31. In steps S32 and S33, it is determined whether or not a target is recognized by the image from the right camera 34.

In this case, if the target is recognized only from the image of the front camera 31, the process proceeds to step S34, where the first continuity reliability Cconf1 for each target recognition method is calculated. After processing in step S34, the process proceeds to step S39. In step S39, the first integrated reliability Iconf1 is calculated based on the first continuity reliability Cconf1 for each target recognition method.

If the target is recognized only from the image of the right camera 34, the process proceeds to step S35, where the second continuing reliability Cconf2 is calculated. After processing in step S35, the process proceeds to step S40. In step S40, the second integrated reliability Iconf2 is calculated based on the second continuing reliability Cconf2 for each target recognition method.

If the target is recognized by both cameras 31 and 34, the process proceeds to step S36. Then, in step S36, the first continuity reliability Cconf1 is calculated, and in step S37, the second continuity reliability Cconf2 is calculated.

Then, in step S38, it is determined whether the frame number nf of the front camera image is greater than the frame number nr of the right camera image. If nf>nr, the process proceeds to step S39, where the first integrated reliability Iconf1 is calculated based on the first continuation reliability Cconf1 for each target recognition method. If nf≦nr, the process proceeds to step S40, where the second integrated reliability Iconf2 is calculated based on the second continuation reliability Cconf2 for each target recognition method.

Then, in step S41, a determination is made as to whether the target is moving or stationary based on the first integrated reliability Iconf1 or the second integrated reliability Iconf2 (see step S21 in FIG. 6).

According to each of the above embodiments, the following effects can be obtained.

The same target is recognized by a plurality of target recognition methods from images acquired at a predetermined period, and a continuation confidence Cconf (existence confidence) indicating the likelihood that the target exists as a moving object candidate is calculated for each target recognition method based on the target recognition results in each of the plurality of different target recognition methods and a predetermined weighting coefficient W determined for each target recognition method. Then, the target's integrated confidence Iconf is calculated by integrating the continuation confidence Cconf calculated for each target recognition method, and a moving/stationary judgment is made based on the integrated confidence Iconf. In this case, the accuracy of the moving/stationary judgment can be improved by using the recognition results by the plurality of different target recognition methods and integrating them to make the moving/stationary judgment. In addition, since the weighting coefficient W is determined for each target recognition method, unlike the case where the weighting coefficient W is determined uniformly for each target recognition method, the target's integrated confidence Iconf can be properly calculated while reflecting which of the plurality of target recognition methods the target was recognized by. This allows targets to be accurately determined to be stationary or moving.

For each target recognition method, instantaneous value data is calculated, which is the product of instantaneous confidence Tconf and weighting factor W, and the continuous confidence Cconf is calculated by accumulating the instantaneous value data. As a result, even if the state of target recognition in one of the target recognition methods among the multiple target recognition methods changes or target recognition is interrupted during a period in which target recognition is continuing, the weighting factor W for the recognition result of the corresponding target recognition method can be temporarily set to zero, and the integrated confidence Iconf in an integrated sense can be continuously and appropriately calculated.

When targets are continuously recognized, movement of at least one of the vehicle and the target may affect target recognition by each target recognition method. For example, depending on the target recognition method, the weighting factor W may be changed to take into account the difference in recognition accuracy when the vehicle is moving and when it is stopped, or the weighting factor W may be changed to take into account the difference in recognition accuracy depending on the distance from the vehicle to the target.

In this regard, a weighting factor W is set based on the type of target and the vehicle's moving or stationary state, and the weighting factor W is used to calculate instantaneous value data, while the continuation reliability Cconf is calculated by integrating the time-series instantaneous value data. This makes it possible to properly calculate the continuation reliability Cconf (presence reliability) while taking into account the situation that changes from moment to moment during the period in which target recognition continues. This therefore makes it possible to improve the accuracy of the target's integrated reliability Iconf.

When multiple target recognition methods are used, the weighting factor W may be set to zero depending on the type of target and the moving/stationary state of the vehicle. Taking this into consideration, when calculating the integrated reliability Iconf by integrating the continuation reliability Cconf of multiple target recognition methods, the continuation reliability Cconf of the target recognition method for which the weighting factor W is always zero is excluded from the integration, and the remaining continuation reliability Cconf that has not been excluded from the integration is weighted equally to calculate the integrated reliability Iconf. This makes it possible to properly calculate the integrated reliability Iconf while distinguishing between the multiple target recognition methods that are capable of target recognition and those that are not.

When a first camera and a second camera are mounted as cameras with different shooting directions and partially overlapping shooting ranges, it is possible that the same target is photographed by the first camera and the second camera. In such a case, calculation of the continuation reliability Cconf for each target recognition method for the recognized target is started from the point in time when target recognition is performed from the image of the first camera. After that, when the same target is also recognized in the image of the second camera, calculation of the continuation reliability Cconf for each target recognition method for the recognized target by the image of the second camera is started from that point in time, overlapping with the processing on the first camera side. Then, the moving/stationary determination is made based on the continuation reliability Cconf calculated using the image of the first camera and the continuation reliability Cconf calculated using the image of the second camera, whichever has the larger number of image frames. In this case, even if the camera capturing the target is switched due to the movement of either the vehicle or the target, the continuity confidence Cconf, which is the target recognition data, can be appropriately inherited, and the movement/stationary determination can be performed appropriately.

The multiple target recognition methods include a type recognition method and an optical flow recognition method, and the continuation confidence Cconf_A calculated from the type determination result of the type recognition method and the continuation confidence Cconf_B calculated from the target tracking result of the optical flow recognition method are integrated to calculate the integrated confidence Iconf of the target. In this case, unlike a configuration in which a moving/still determination of a target is simply performed using optical flow, the accuracy of the moving/still determination can be improved by taking into account the reliability of the type determination.

When the host vehicle is moving, it is considered that the accuracy of target recognition using optical flow is reduced compared to when the host vehicle is stationary. Taking this into consideration, the weighting coefficient WB of the optical flow recognition method is set smaller when the host vehicle is moving than when the host vehicle is stationary. This makes it possible to improve the accuracy of the integrated reliability Iconf of the target, which is calculated by integrating the continuity reliability Cconf of multiple target recognition methods.

When the vehicle is stationary, it becomes difficult to grasp the three-dimensional positions of feature points on the image, and it is thought that the accuracy of target recognition using the three-dimensional estimated image (SfM image) will decrease. In consideration of this, the weighting coefficient WD of the three-dimensional recognition method is set to zero when the vehicle is stationary. This makes it possible to improve the accuracy of the integrated reliability Iconf of the target, which is calculated by integrating the continuity reliability Cconf of multiple target recognition methods.

Other Embodiments
The above embodiment may be modified, for example, as follows.

- In the above embodiment, four target recognition methods are used to calculate the instantaneous confidence Tconf, set the weighting coefficient W, and calculate the continuous confidence Cconf, but this may be changed. For example, three or two target recognition methods may be used. Alternatively, five or more target recognition methods may be used. When changing the combination of multiple target recognition methods, it is desirable to combine a target recognition method based on type determination with another target recognition method.

When changing the combination of multiple target recognition methods, the set values of the weighting coefficients shown in Figures 4(a) to (d) may be changed appropriately according to the combination. Note that it is also possible to use only one of the target type and the vehicle's moving/stationary state as a parameter for setting the weighting coefficients.

- In the above embodiment, when calculating the integrated reliability Iconf, the continuous reliability Cconf is calculated using instantaneous value data (instantaneous reliability Tconf × weighting coefficient W) of the time series for each target recognition method, and the integrated reliability Iconf is calculated based on the continuous reliability Cconf. However, this may be changed. For example, the integrated reliability Iconf may be calculated based on the current value of the instantaneous value data for each target recognition method. In this case, the instantaneous value data (instantaneous reliability Tconf × weighting coefficient W) may be used instead of the continuous reliability Cconf as a variable in the above formula (2). The instantaneous reliability Tconf corresponds to "existence reliability". In this configuration, the target recognition accuracy is correctly evaluated for each target recognition method, and then the integrated reliability Iconf of the target can be properly calculated.

The control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be realized by one or more dedicated computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transient tangible recording medium as instructions executed by the computer.

10...moving object determination device, 30...camera.

Claims (10)

A moving object determination device (10) is applied to a vehicle equipped with a camera (30) that photographs the surroundings of the vehicle, and performs a moving/stationary determination based on images photographed by the camera to determine whether a target included in the images is moving, comprising:
an acquisition unit that acquires the images at a predetermined period;
a recognition unit that recognizes the same target by a plurality of target recognition methods different from each other using the image acquired by the acquisition unit;
an existence reliability calculation unit that calculates, for each of the plurality of target recognition methods, an existence reliability indicating a likelihood that the target exists as a moving object candidate, based on a recognition result of the target in each of the plurality of target recognition methods and a predetermined weighting coefficient determined for each of the target recognition methods;
an integrated reliability calculation unit that integrates the presence reliability calculated for each of the target recognition methods and calculates an integrated reliability of the target;
A determination unit that performs a moving/stationary determination based on the integrated reliability;
A moving object determination device comprising: The moving object determination device according to claim 1, wherein the presence reliability calculation unit calculates the presence reliability using the weighting coefficient determined for each target recognition method based on at least one of the type of the target and the moving or stationary state of the host vehicle. The presence reliability calculation unit
an instantaneous value calculation unit that calculates, for each of the images acquired in time series, an instantaneous reliability that is the instantaneous reliability of the presence of the target for a recognition result of the target in each of the plurality of target recognition methods, and calculates instantaneous value data by multiplying the instantaneous reliability by the weighting coefficient determined for each of the target recognition methods;
a continuation reliability calculation unit that calculates an integrated value of the instantaneous value data within a recognition period of the target for each of the target recognition methods, and calculates a continuation reliability, which is a reliability of the continuous existence of the target, based on the integrated value;
The moving object determination device according to claim 1 , wherein the integrated reliability calculation unit calculates the integrated reliability by integrating the continuing reliability calculated for each of the target recognition methods by the continuing reliability calculation unit. a weight setting unit that sets the weight coefficient based on at least one of a type of the target and a moving or stationary state of the host vehicle for each of the target recognition methods,
the instantaneous value calculation unit calculates the instantaneous value data based on the instantaneous reliability calculated from the recognition results of the target by each of the target recognition techniques and the weighting coefficient set by the weight setting unit;
The moving object determination device according to claim 3 , wherein the continuation reliability calculation unit calculates the continuation reliability based on an integrated value of the instantaneous value data of the same target accumulated in time series. the weight setting unit sets a numerical value within a range including zero as the weight coefficient based on at least one of a type of the target and a moving or stationary state of the host vehicle,
5. The moving body determination device according to claim 4, wherein when the integrated reliability calculation unit calculates the integrated reliability by integrating the continuation reliability for each of the target recognition methods, the integrated reliability calculation unit excludes from the integration the continuation reliability by a target recognition method whose weighting coefficient is always zero among the continuation reliability for each of the target recognition methods, and calculates the integrated reliability by equally weighting the remaining continuation reliability that has not been excluded from the integration. The present invention is applied to a vehicle equipped with a first camera and a second camera having different shooting directions and partially overlapping shooting ranges as the cameras,
In a situation where the target is recognized by the image of the first camera and the continuation reliability is calculated for the target for each target recognition method, when the same target is recognized by the image of the second camera, the continuation reliability is calculated for the recognized target by the image of the second camera for each target recognition method in a manner overlapping with the processing on the first camera side;
The moving body determination device according to any one of claims 3 to 5, wherein the determination unit performs a moving/still determination based on the continuing reliability calculated using the camera image having a greater number of image frames, between the continuing reliability calculated using the image of the first camera and the continuing reliability calculated using the image of the second camera. the plurality of target recognition methods include a type recognition method for recognizing the target by determining a type of the target included in the image, and an optical flow recognition method for recognizing a movement of the target included in the image by an optical flow,
the presence reliability calculation unit calculates the presence reliability for a result of the type determination performed by the type recognition method, and calculates the presence reliability for a result of tracking the target performed by the optical flow recognition method;
The moving object determination device according to any one of claims 1 to 6, wherein the integrated reliability calculation unit integrates the presence reliability calculated from the type determination result of the type recognition method and the presence reliability calculated from the target tracking result of the optical flow recognition method to calculate an integrated reliability of the target. the plurality of target recognition methods include an optical flow recognition method that recognizes a movement of the target included in the image by an optical flow;
The moving body determination device according to any one of claims 1 to 7, wherein the presence reliability calculation unit calculates the presence reliability by making the weighting coefficient of the optical flow recognition method smaller when the vehicle is moving than when the vehicle is stationary. the plurality of target recognition methods include a three-dimensional recognition method for creating a three-dimensional estimated image representing a three-dimensional position of a feature point on the image and recognizing the target based on the three-dimensional estimated image;
The moving object determination device according to any one of claims 1 to 8, wherein the presence reliability calculation unit calculates the presence reliability by setting the weighting coefficient of the three-dimensional recognition method among the plurality of target recognition methods to zero when the host vehicle is stationary. A moving object determination program is applied to a vehicle equipped with a camera (30) that photographs the surroundings of the vehicle, and is executed by a control device (10) to perform a moving/stationary determination based on images photographed by the camera to determine whether a target included in the images is moving or not, comprising:
An acquisition step of acquiring the image at a predetermined period;
a recognition step of recognizing the same target by a plurality of target recognition methods different from each other using the images acquired in the acquisition step;
an existence reliability calculation step of calculating, for each of the plurality of target recognition methods, an existence reliability indicating a likelihood that the target exists as a moving object candidate, based on a recognition result of the target in each of the plurality of target recognition methods and a predetermined weighting coefficient determined for each of the target recognition methods;
an integrated reliability calculation step of integrating the presence reliability calculated for each of the target recognition methods and calculating an integrated reliability of the target;
A determination step of determining whether or not the movement or stationary state is performed based on the integrated reliability;
A moving object determination program including:

Images