JP7764764B2 - Sensor calibration device and sensor calibration method - Google Patents

Description

The present invention relates to a sensor calibration device and a sensor calibration method.

Sensor calibration devices that calibrate the orientation and position of sensors have been known for some time (see, for example, Patent Document 1). Sensor calibration devices set a reference within the detection range of the sensor and measure the position of an object relative to the reference, thereby calibrating the position and orientation parameters of the sensor.

Patent No. 5311365

However, depending on the sensor installation environment, it may be difficult to measure the position of an object relative to a reference. In this case, calibrating the sensor's position and orientation parameters requires obtaining a lot of information about the object's position, which takes time.

The present invention has been made to solve the above-mentioned problems, and aims to provide technology that can easily calibrate the position and orientation of a sensor in a sensor calibration device.

The present invention has been made to solve at least some of the above-mentioned problems, and can be realized in the following forms.

(1) According to one aspect of the present invention, there is provided a sensor calibration device used to calibrate the position and orientation of a sensor. This sensor calibration device includes a reference line setting unit that sets a reference line; a reference point setting unit that sets a reference point on the reference line that will be the center of a position of an object moving in three-dimensional space; a position measurement unit that measures the position of the object using pre-set position and orientation parameters that represent the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a memory unit that stores, as a set of measurement points, the position measurement results of the object moving on the reference line at each time, measured by the position measurement unit; and a density of measurement points in the set of measurement points. The system includes a stay point calculation unit that calculates estimated stay points based on the distribution; a line approximation unit that creates an approximate line of the set of measurement points using the set of measurement points stored in the storage unit; and a parameter calibration unit that calibrates the position and orientation parameters using the reference line, the reference point, the approximate line, and the estimated stay points, wherein the parameter calibration unit calibrates the position parameters by translating the approximate line so that the estimated stay points approach the reference points, and calibrates the orientation parameters by rotating the approximate line so that the approximate line approaches the reference line.

With this configuration, an approximate line is created from the position measurement results at each time of an object moving on the reference line, relative to both the reference line and a reference point set on the reference line, and an estimated stay point is calculated. The parameter calibration unit can calibrate the sensor's orientation parameters by rotating the approximate line so that it approaches the reference line. The estimated stay point is calculated based on the density distribution of measurement points in a measurement point set that combines the object's position measurement results at each time, and serves as the center of stay of the object moving using the reference point as a landmark. The parameter calibration unit can thus calibrate the position parameters by translating the approximate line so that the calculated estimated stay point approaches the reference point. This allows the sensor's position and orientation parameters to be calibrated by the object moving on the reference line within the sensor's detection range, with the reference point as the center of stay. Therefore, the sensor's position and orientation parameters can be calibrated even if the position measurement unit has few object position measurement results, allowing the position and orientation parameters to be calibrated in a short time.

(2) In the sensor calibration device of the above aspect, the reference point setting unit sets a plurality of the reference points on the reference line and sets the reference point where the object stays the most among the plurality of set reference points as a representative reference point; the stay point calculation unit calculates the plurality of estimated stay points based on the density distribution of the measurement points in the set of measurement points and sets the estimated stay point with the highest density distribution of the measurement points in the set of measurement points as an estimated longest stay point among the plurality of estimated stay points; and the parameter calibration unit may calibrate the position parameters by translating the approximation line so that the estimated longest stay point approaches the representative reference point. According to this configuration, when setting a plurality of reference points on the reference line, the reference point setting unit sets the reference point where the object stays the most as a representative reference point. When calculating a plurality of estimated stay points based on the density distribution of the measurement points in the set of measurement points, the stay point calculation unit sets the estimated stay point with the highest density distribution of the measurement points as an estimated longest stay point. This means that even if multiple reference points can be set, by setting a representative reference point and an estimated longest stay point, the approximate line can be translated so that the representative reference point and the estimated longest stay point approach each other, thereby calibrating the sensor's position parameters.

(3) In the sensor calibration device of the above aspect, the line approximation unit may create an approximate line for the set of measurement points by performing principal component analysis on the estimated longest stay point. According to this configuration, the approximate line is created by principal component analysis performed on the estimated longest stay point. As a result, the approximate line passes through the estimated longest stay point, so that by translating the approximate line, the representative reference point and the estimated longest stay point can be superimposed, and by rotating the approximate line, the approximate line can be superimposed on the reference line. Therefore, the position and orientation parameters of the sensor can be reliably calibrated.

(4) In the sensor calibration device of the above aspect, the reference point setting unit may set a plurality of the reference points on the reference line, the stay point calculation unit may calculate the same number of estimated stay points as the number of the reference points set by the reference point setting unit, and the parameter calibration unit may calibrate the position parameters by translating the approximation line so that each of the plurality of estimated stay points approaches each of the corresponding plurality of reference points. According to this configuration, the reference point setting unit sets a plurality of reference points on the reference line. The stay point calculation unit calculates a plurality of estimated stay points based on the density distribution of the measurement points in the set of measurement points. The parameter calibration unit calibrates the position parameters by translating the approximation line so that each of the plurality of estimated stay points approaches a plurality of reference points. This can improve the calibration accuracy of the position and orientation parameters of the sensor compared to translating the approximation line so that one estimated stay point approaches one reference point.

(5) In the sensor calibration device of the above aspect, the parameter calibration unit may calibrate the position parameters by translating the approximation line so as to minimize the sum of absolute values of distances between the plurality of reference points and the plurality of estimated stay points corresponding to each of the plurality of reference points. According to this configuration, the parameter calibration unit calibrates the position parameters using the least absolute value method, which is less susceptible to the influence of outliers. This can further improve the calibration accuracy of the sensor's position parameters.

(6) In the sensor calibration device of the above aspect, the sensor calibration device is connected to two sensors installed in different locations, the reference line setting unit sets intersecting reference lines for each of the two sensors, the reference point setting unit sets a reference point on each of the two reference lines, and the line approximation unit acquires information regarding the distance between the reference point and the intersection point on a reference broken line formed by the two intersecting reference lines and the intersection point of the two reference lines, and creates an approximated line for the set of measurement points using the information regarding the distance between the reference point and the intersection point. According to this configuration, the line approximation unit sets a reference broken line formed by the two reference lines set for each of the two sensors. For each of the two reference lines, the line approximation unit acquires the distance from the intersection point of the two reference lines to each reference point on the two reference lines, and calibrates the position and orientation parameters of the sensor using the position of the reference point and the distance from the reference point to the intersection point. This allows the approximate line to be determined using two points, the reference point and the intersection point, improving the accuracy of the approximate line. This in turn improves the accuracy of calibration of the sensor's position and orientation parameters.

(7) According to another aspect of the present invention, a sensor calibration method is provided that is used to calibrate the position and orientation of a sensor. a step of calibrating the position and orientation parameters using the reference line, the reference point, the reference point being the center of stay of an object moving in three-dimensional space; a step of measuring the position of the object using position and orientation parameters that are preset as parameters representing the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a step of storing position measurement results of the object moving on the reference line at each time by the position measurement unit as a measurement point set; a step of calculating an estimated stay point based on a density distribution of measurement points in the measurement point set; a step of creating an approximation line of the measurement point set using the stored measurement point set; and a calibration step of calibrating the position and orientation parameters using the reference line, the reference point, the approximation line, and the estimated stay point. In the calibration step, the position parameters are calibrated by translating the approximation line so that the estimated stay point approaches the reference point, and the orientation parameters are calibrated by rotating the approximation line so that the approximation line approaches the reference line. With this configuration, the sensor calibration method creates an approximate line from the position measurement results at each time of an object moving on a reference line. The sensor calibration method calculates an estimated stay point based on the density distribution of measurement points in a set of measurement points that combines the position measurement results of the object at each time relative to the reference point. This allows the position and orientation parameters of the sensor to be calibrated simply by the object moving on the reference line within the sensor's detection range, with the reference point as the center of stay, thereby enabling the position and orientation parameters to be calibrated in a short time.

The present invention can be realized in various forms, such as a system including a sensor calibration device, a control method for these devices and systems, a computer program for executing the sensor calibration method in these devices and systems, a server device for distributing the computer program, and a non-transitory storage medium on which the computer program is stored.

1 is a schematic diagram illustrating a schematic configuration of a sensor calibration device according to a first embodiment; 1 is a flowchart illustrating a sensor calibration method. FIG. 10 is a diagram illustrating a reference setting process. FIG. 10 is a diagram illustrating a stay point calculation process. FIG. 1 is a first diagram illustrating a parameter calibration process. FIG. 10 is a second diagram illustrating the parameter calibration process. FIG. 10 is a third diagram illustrating the parameter calibration process. 10A and 10B are diagrams illustrating how to overlap a reference line and an approximation line. 10A and 10B are diagrams illustrating a method for creating an approximate straight line in a first comparative example. FIG. 10 is a first diagram illustrating a method for creating an approximate straight line in a second comparative example. FIG. 10 is a second diagram illustrating a method for creating an approximate straight line in the second comparative example. 5A to 5C are diagrams illustrating a method for creating an approximate straight line according to the first embodiment. 10A and 10B are diagrams illustrating a sensor calibration method according to a second embodiment. FIG. 10 is a schematic diagram illustrating a schematic configuration of a sensor calibration device according to a third embodiment. FIG. 2 is a diagram illustrating the positional relationship between two monocular cameras.

First Embodiment
FIG. 1 is a schematic diagram illustrating the overall configuration of a sensor calibration device 1 according to a first embodiment. The sensor calibration device 1 according to this embodiment is a device for calibrating a sensor that is fixed in one location. The sensor calibration device 1 is used, for example, when correcting position parameters representing the position and attitude parameters representing the attitude of a sensor that is temporarily installed in conjunction with the establishment of an event. The sensor calibration device 1 according to this embodiment calibrates a single monocular camera 5 as a sensor. The monocular camera 5 is installed, for example, near a road on which vehicles travel, continuously captures images of vehicles traveling on the road and pedestrians 6 (see FIG. 1 ) on the side of the road, and outputs the captured images (multidimensional information). The sensor calibration device 1 includes a measuring instrument 10 and a calibration processor 20. While the sensor according to this embodiment is a monocular camera 5, the type of sensor is not limited thereto and may be a stereo camera, radar, LiDAR (laser radar), sonar, or the like.

The measuring instrument 10 includes a position measurement unit 11 and a reference setting unit 12. The measuring instrument 10 is electrically connected to the monocular camera 5, and receives images captured by the monocular camera 5 as input.

The position measurement unit 11 measures the position of an object (pedestrian 6) included in an image captured by the monocular camera 5. Provisional position and orientation parameters, including both position parameters and orientation parameters that are preset as parameters representing the provisional position and orientation of the monocular camera 5, are input to the position measurement unit 11. The position measurement unit 11 measures the position of the moving pedestrian 6 from these position and orientation parameters and the image captured by the monocular camera 5. The position measurement unit 11 outputs the measurement results and the provisional position and orientation parameters to the calibration processor 20, to which it is electrically connected.

The reference setting unit 12 sets a reference line and a reference point on the captured image, which is the output information of the monocular camera 5. The reference line is a line that can be set arbitrarily, and specifically, it is set on the image based on a linear object included in the image captured by the monocular camera 5, such as a row of markers on a sidewalk to guide the visually impaired or a white line dividing a sidewalk into a pedestrian lane and a bicycle lane. The reference point is a point set on the reference line in the image captured by the monocular camera 5, and indicates the position of an object that has been preset as the center of pedestrian 6's location. Specifically, it is set on the image corresponding to a location in the image captured by the monocular camera 5 where an object that is easily recognizable from the pedestrian's perspective is installed, such as a fixed bus stop sign or a warning marker among markers for guiding the visually impaired. Note that the objects that serve as the reference for setting the reference line and the objects that are preset for setting the reference point are not limited to these. For example, the object used as the reference for setting the reference line may be a white line that distinguishes lanes on a road, or a curb that distinguishes the roadway from the sidewalk, and the object set in advance as the reference point may be a bench or trash can on the sidewalk.

The calibration processor 20 includes a memory unit 21 and a CPU 22. The calibration processor 20 calibrates the position and orientation parameters of the monocular camera 5 using the object position measurement results measured by the position measurement unit 11, the provisional position and orientation parameters used in the measurement, and the reference line set by the reference setting unit 12. The calibrated position and orientation parameters are output by the calibration processor 20 to the position measurement unit 11 of the measuring device 10, which is electrically connected to the calibration processor 20.

The memory unit 21 is composed of a hard disk, flash memory, memory card, etc. The memory unit 21 stores the position measurement results at each time of an object moving on a reference line measured by the position measurement unit 11 as a set of measurement points. The memory unit 21 also stores various calculation results from the CPU 22. The memory unit 21 outputs this stored information to the CPU 22 as appropriate.

The CPU 22 controls each part of the sensor calibration device 1, including the measuring instrument 10, by loading a computer program stored in ROM into RAM and executing it. In addition, the CPU 22 functions as a stay point calculation unit 22a, a straight line approximation unit 22b, and a parameter calibration unit 22c, and calculates an estimated stay point and creates an approximated line from the position measurement results of a moving object measured by the position measurement unit 11, and calibrates position and orientation parameters from the relationship between the estimated stay point and the reference point, and the relationship between the approximated line and the reference line.

The stay point calculation unit 22a calculates an estimated stay point based on the density distribution of measurement points in the set of measurement points stored in the memory unit 21. Specifically, the stay point calculation unit 22a calculates a point in the set of measurement points stored in the memory unit 21 where the density distribution of measurement points is relatively high, and sets this point as the estimated stay point. The detailed function of the stay point calculation unit 22a will be described later.

The straight line approximation unit 22b creates an approximate straight line of the set of measurement points using the set of measurement points of the object position measurement results stored in the memory unit 21. The detailed function of the straight line approximation unit 22b will be described later.

The parameter calibration unit 22c calibrates the position and orientation parameters of the monocular camera 5 using the approximate line created by the straight line approximation unit 22b, the reference line, and the provisional position and orientation parameters. The detailed functions of the parameter calibration unit 22c will be described later.

Figure 2 is a flowchart illustrating a sensor calibration method. Next, the sensor calibration method of this embodiment will be described. The sensor calibration method of this embodiment is performed by the sensor calibration device 1, for example, when the monocular camera 5 is installed in a predetermined location. The sensor calibration method checks the installation position of the monocular camera 5 and the orientation of the monocular camera 5, which affect the direction of the optical axis of the monocular camera 5, and corrects these during installation of the monocular camera 5 (on-site adjustment work). This on-site adjustment work is performed to reduce false detections (determining that a measurement target object is present within the detectable range of the monocular camera 5 when it is not) and non-detections (determining that a measurement target object is not present within the detectable range of the monocular camera 5 when it is). Therefore, on-site calibration is required after installation of the monocular camera 5. However, measuring the position and orientation of the monocular camera 5 itself or performing calibration using a calibration board is not sufficient for on-site adjustment work. Furthermore, position and orientation estimation using other sensors, such as positioning satellite reception or a gyroscope, is not suitable for the monocular camera 5 because it cannot properly estimate the orientation when the camera is stationary.

First, a reference line and a reference point are set (reference setting process: step S1). In step S1, the reference setting unit 12 sets objects arranged in a straight line at the site as a reference line in the coordinate system of the position measurement unit 11, and sets a reference point on the set reference line. In this embodiment, the reference setting unit 12 sets the row 7 of guiding blocks for visually impaired people on the sidewalk as the reference line (see Figure 1). The reference point set by the reference setting unit 12 is preferably a landmark for pedestrians 6 traveling along the row 7 of guiding blocks for visually impaired people. In this embodiment, the reference points are multiple points (not shown), including the location 81 (see Figure 1) where a sign 8 installed along the row 7 of guiding blocks for visually impaired people is fixed, and the location 81 where the sign 8 is fixed (hereinafter referred to as the "location of the sign") is set as the representative reference point. Note that the objects that serve as the reference line and the reference points are not limited to these.

Figure 3 is a diagram illustrating the reference setting process. Figure 3 shows a portion of an image captured by the monocular camera 5. In the reference setting process, the reference setting unit 12 sets a reference line Ls on the x-axis of the coordinate system of the image shown in Figure 3, and sets a representative reference point Ps0 on the reference line Ls that corresponds to the location 81 of the sign.

Next, the position of pedestrian 6 is measured (position measurement process: step S2). In step S2, the position measurement unit 11 measures the position of pedestrian 6 moving on row 7 of guiding blocks for visually impaired persons using the preset provisional position and orientation parameters of monocular camera 5 and the output of monocular camera 5. In this embodiment, when pedestrian 6 walks on row 7 of guiding blocks for visually impaired persons, he moves so that the time spent at location 81 of the sign serving as the representative reference point is longer than the time spent at other locations.

Here, the provisional position and orientation parameters of the monocular camera 5 in the sensor calibration method of this embodiment refer to position parameters expressed by coordinate values s _t (vector notation) for each of the x-axis, y-axis, and z-axis, and orientation parameters expressed by azimuth angle a _t and elevation angle e _t . In other words, the position measurement unit 11 measures the position of the pedestrian 6 output by the monocular camera 5 in a state in which the provisional position parameter s _t (vector notation) and orientation parameters e _t and a _t have been set. The position measurement unit 11 outputs the position measurement results of the pedestrian 6 to the storage unit 21. Note that when characters shown in the text are vectors, "(vector notation)" is written after the target character.

Next, the position measurement results at each time of the pedestrian 6 moving on the row 7 of guiding blocks for visually impaired persons, measured in step S2, are stored as a set of measurement points (storage process: step S3). The memory unit 21 stores the position measurement results of the pedestrian 6 output by the position measurement unit 11 as a set of measurement points in three-dimensional space. The memory unit 21 also stores the position information of the row 7 of guiding blocks for visually impaired persons set by the reference setting unit 12 as position information of the reference straight line.

Next, an estimated stay point is calculated from the position measurement results of the pedestrian 6 stored in step S3 (stay point calculation process: step S4). The stay point calculation unit 22a uses the set of measurement points stored in the memory unit 21 to calculate a position where the pedestrian 6 stayed for a relatively long time as an estimated stay point.

Figure 4 is a diagram illustrating the stay point calculation process and shows an example of a measurement point set. Figure 4 shows a two-dimensional plane (xy plane) on which the position measurement results of pedestrian 6 stored in memory unit 21 are plotted. Each of the multiple circles on the xy plane shown in Figure 4 represents the position measurement results of pedestrian 6 at each time, and indicates the position (measurement point M) of pedestrian 6 measured at regular time intervals within a specified period. In other words, memory unit 21 stores a set of position measurement results of pedestrian 6 at each time (measurement point set Gm). Note that for ease of explanation, Figure 4 shows the measurement point set on a two-dimensional plane, and also shows the reference line Ls and representative reference point Ps0 set in the reference setting process.

Now, the method for calculating the estimated stay points in step S4 will be described. In this embodiment, the stay point calculation unit 22a calculates the estimated stay points using a well-known hierarchical clustering shortest distance method, which aggregates the measurement points M included in the measurement point set Gm into groups called "clusters" in order of proximity. The stay point calculation unit 22a terminates clustering when the maximum distance within a cluster reaches a threshold value that takes into account the variance of position measurements, for example, 0.1 m, and selects clusters in order of the number of points within the cluster until the number of points within the cluster becomes the same as the reference point. In this embodiment, for each cluster, the median value of the positions of each point within the cluster is set as the estimated stay point Pe, and the coordinate point with the highest distribution density of measurement points M among the multiple clusters selected in the measurement point set Gm is set as the estimated longest stay point LPe.

Next, in step S5, an approximate straight line of the set of measurement points is created using the set of measurement points stored in memory unit 21 (straight line approximation step: step S5). Line approximation unit 22b creates an approximate straight line for calibrating the position parameter s _t (vector notation) and the attitude parameters e _t and a _t of monocular camera 5 using the set of measurement points of the position measurement results of pedestrian 6 stored in memory unit 21.

In the straight line approximation step, the straight line approximation unit 22b first generates an approximated line candidate using a method that combines principal component analysis and the RANSAC method. In this embodiment, instead of the usual principal component analysis that uses the average of coordinate values of multiple measurement points, the straight line approximation unit 22b performs principal component analysis using the coordinate values of the estimated longest stay point to generate an approximated line candidate. Specifically, assuming that the coordinate value matrix of each of the multiple measurement points is matrix P, the coordinate value of the estimated longest stay point is coordinate value p0 _t (vector notation), and the number of measurement points is n, the following formula (1) is subjected to eigenvalue decomposition to obtain the maximum eigenvalue and the first eigenvector fev (vector notation) corresponding to the maximum eigenvalue. Note that T in formula (1) is a transpose operator.

In the linear approximation process, the linear approximation unit 22b determines whether to adopt a combination of the maximum eigenvalue and the first eigenvector fev (vector representation) calculated by the principal component analysis described above, based on the square root of the maximum eigenvalue. The linear approximation unit 22b adopts the created approximated line candidate if the square root of the maximum eigenvalue is equal to or greater than a threshold set based on the length of the line segment included in the position-measurable range of the reference line, and rejects the created approximated line candidate if the square root is less than the threshold. Specifically, assuming that the length of the line segment included in the position-measurable range of the reference line is 4 m and the threshold is 1.8 m, half the measurable range of 3.6 m that takes detection omissions into account, the created approximated line candidate is adopted if the length is 1.8 m or greater, and is rejected if the length is less than 1.8 m. The linear approximation unit 22b repeatedly calculates combinations of the maximum eigenvalue and the first eigenvector fev (vector representation) calculated by the principal component analysis described above and makes decisions based on the square root of the maximum eigenvalue until the number of times that the combination has not been rejected reaches a predetermined number. When the number of times that rejection has reached a preset number, the straight line approximation unit 22b selects, as an approximate straight line, a straight line that has the smallest approximation error from among the combinations of the maximum eigenvalue that was not rejected and the first eigenvector fev (vector notation). The straight line approximation unit 22b stores the maximum eigenvalue and first eigenvector fev (vector notation) of the selected approximate straight line in the memory unit 21.

Returning to FIG. 2, after step S5, the position parameters and attitude parameters of the monocular camera 5 are calibrated using the reference line set in step S1 and the approximate line selected in step S5 (parameter calibration process: step S6). In this embodiment, the parameter calibration unit 22c calibrates the attitude parameters a and e, and then calibrates the position parameter s.

FIG. 5 is a first diagram illustrating the parameter calibration process. Here, a method for calibrating the position and orientation parameters in the parameter calibration process will be described. FIG. 5( a) shows an xy plane in a three-dimensional space, and shows projection points Mxy obtained by projecting each of the multiple measurement points M onto the xy plane, and a projected approximate line Laxy obtained by projecting the approximate line La onto the xy plane. FIG. 5( a) also shows a provisional position parameter s _t (vector notation) of the monocular camera 5 on the xy plane, and an azimuth _angle at representing the provisional attitude. Here, the provisional azimuth angle is assumed to be, for example, azimuth angle _at = 15 degrees. FIG. 5( b) shows an xz plane in a three-dimensional space, and shows projection points Mxz obtained by projecting each of the multiple measurement points M onto the xz plane, and a projected approximate line Laxz obtained by projecting the approximate line La onto the xz plane. 5B also shows a provisional position parameter s _t (vector notation) of the monocular camera 5 on the xz plane and an elevation angle e _t representing a provisional attitude. Here, the provisional elevation angle is assumed to be, for example, e _t = 0 degrees.

The reference line Ls is set on the x-axis in step S1 (see FIG. 5). At this time, each of the multiple measurement points M is determined by first rotating a point in a coordinate system centered on the monocular camera 5 (the monocular camera 5 is at the origin of the three-dimensional space, and the front axis of the monocular camera 5 is the x-axis, which is the optical axis of the monocular camera 5) by an elevation angle e _t and then an azimuth angle a _t , and then translating the point so that the position of the monocular camera 5 has a coordinate value s _t (vector notation).

Furthermore, in step S5, the coordinate value p0 _t (vector notation) of the estimated longest stay point and the first eigenvector fev (vector notation) of the approximation line La are stored in the storage unit 21. The parameter calibration unit 22c derives the elevation angle e c and azimuth angle a c after calibration using this information and the two coordinate values p0 _t (vector notation) and p1 _t (vector notation) shown in Figures 5(a ₎ and 5(b ₎ , respectively. Here, the coordinate value p1 _t is expressed by the following equation (2) using the coordinate value p0 _t (vector notation) and the first eigenvector fev (vector notation) of the approximation line La.
Here, in a coordinate system centered on the monocular camera 5, a function E(e) is introduced to find a rotation matrix for rotating the measurement point around the origin according to the elevation angle e, and a function A(a) is introduced to find a rotation matrix for rotating the measurement point around the origin according to the azimuth angle a. The elevation angle e is defined as shown in equation (3) because the x-axis is rotated counterclockwise as viewed from the south around the y-axis (axis from south to north).

The azimuth angle a is defined as in equation (4) by rotating the x-axis counterclockwise as viewed from above around the z-axis (axis from bottom to top).

The post-calibration elevation angle e _c and azimuth angle a _c to be derived are the elevation angle and azimuth angle at which the projected approximate lines Laxy and Laxz become parallel to the x-axis, which is the reference line Ls, when the influence of the provisional position and attitude parameters is canceled out and the plane is rotated again by the elevation angle and azimuth angle. In this case, the difference between the post-calibration coordinate value p0 _p (vector notation) obtained by calibrating only the attitude parameters of the coordinate value p0 _t (vector notation) and the post-calibration coordinate value p1 _p (vector notation) obtained by calibrating only the attitude parameters of the coordinate value p1 _t (vector notation) becomes zero for both the y and z components. The post-calibration coordinate value p0 _p (vector notation) is expressed by equation (5), and the post-calibration coordinate value p1 _p (vector notation) is expressed by equation (6).

To derive the elevation angle e _c and azimuth angle a _c after calibration, the simultaneous equations consisting of the following equations (7) and (8) are solved for the elevation angle difference Δe and the azimuth angle a _c . Equation (7) is an equation for the y component, and equation (8) is an equation for the z component. The elevation angle difference Δe is (e _c - e _t ), and since the elevation angle e _t is a known value as the provisional elevation angle, the elevation angle e _t can be found once the elevation angle difference Δe is found.

By solving the simultaneous equations consisting of equations (7) and (8), the azimuth angle a _c and the elevation angle difference Δe are derived as shown in equations (9) and (10). Using these equations, the azimuth angle a _c and the elevation angle e _c after calibration are calculated.

FIG. 6 is a second diagram illustrating the parameter calibration process. The diagrams shown in FIGS. 6(a) and 6(b) correspond to FIGS. 5(a) and 5(b), respectively, and show the projected approximate lines Laxy and Lxxz corrected using the calibrated azimuth angle a _c and elevation angle e _c calculated using equations (9) and (10). As shown in FIG. 6, each of the projected approximate lines Laxy and Laxz is parallel to the reference line Ls. Specifically, the azimuth angle a is calibrated to be between 15 degrees and 6.1 degrees, and the elevation angle e is calibrated to be between 0 degrees and minus 3 degrees.

FIG. 7 is a third diagram illustrating the parameter calibration process. Next, the parameter calibration unit 22c calibrates the position parameters. In calibrating the position parameters, the monocular camera 5 is translated so that the y-component and z-component of the coordinate value p0 _p after calibration, in which only the attitude parameters have been calibrated, are each zero. For example, for FIG. 5(a) in which only the azimuth angle a has been calibrated, the projected approximate line Laxy is moved in the negative direction of the y-axis to coincide with the reference line Ls (see FIG. 5(a)). For FIG. 6(b) in which only the elevation angle e has been configured, the projected approximate line Laxz is moved in the positive direction of the z-axis to coincide with the reference line Ls (see FIG. 7(a)). As a result, the approximate line La coincides with the x-axis, which is the reference line Ls. That is, the position s _c of the monocular camera 5 after translation can be expressed by equation (11).

FIG. 8 is a diagram illustrating how the reference line and the approximate line are superimposed in the sensor calibration method of this embodiment. Here, the calibration of the position and orientation parameters in the above-described sensor calibration method will be described using the positional relationships among the reference line Ls, the representative reference point Ps0, the estimated longest stay point LPe, and the approximate line La0. In the sensor calibration method of this embodiment, the line approximation unit 22b first creates the approximate line La0 that passes through the estimated longest stay point LPe. Next, the parameter calibration unit 22c rotates the approximate line La0 (arrow R1 in FIG. 8 ) by calibrating the orientation parameters e _t and a _t , thereby creating a virtual line La1 that is parallel to the reference line Ls. Next, the parameter calibration unit 22c translates the virtual line La1 so that the virtual line La1 approaches the reference line Ls, thereby creating a virtual line La2 (arrow T1 in FIG. 8 ). Finally, the parameter calibration unit 22c translates the virtual line La2 so that the estimated longest stay point LPe on the virtual line La2 approaches the representative reference point Ps0 (arrow T2 in FIG. 8 ). In this way, the parameter calibration unit 22c calibrates the attitude parameters and the position parameters by rotating the approximate line La0 so that the approximate line La0 is parallel to the reference line Ls and translating the approximate line La0 so that the estimated longest stay point LPe approaches the representative reference point Ps0. Note that the term "approaching" here also includes the meaning of bringing the estimated longest stay point LPe closer to the representative reference point Ps0, thereby overlapping the estimated longest stay point LPe with the representative reference point Ps0.

As described above, in step S6, the parameter calibration unit 22c calibrates the orientation parameters and then the position parameters. The parameter calibration unit 22c outputs the calibrated position and orientation parameters to the measuring instrument 10. The measuring instrument 10 uses the input, calibrated position and orientation parameters to perform subsequent object position measurements.

Here, the effects of the sensor calibration device 1 of this embodiment, particularly with regard to the accuracy of the approximated line created by the linear approximation unit, will be explained by comparing it with several comparative examples. The first comparative example is an example of creating an approximated line using principal component analysis, and the second comparative example is an example of creating an approximated line using a method that combines principal component analysis and the RANSAC method.

Figure 9 is a diagram illustrating a method for creating an approximate line in the first comparative example. The circles in Figure 9 represent multiple measurement points M. In principal component analysis, an approximate line L01 is created using all measurement points M. For this reason, the approximate line L01 is subject to the influence of outliers, such as measurement point Mx within ellipse R10, and is therefore prone to having a large slope and deviating from the reference line Ls.

Figure 10 is the first diagram illustrating a method for creating an approximate line in the second comparative example. In the second comparative example, first, in the first step, a portion of all measurement points M are randomly selected. For example, as shown in Figure 10, multiple thick circles M2 are selected. A provisional approximate line L02 is obtained for these selected measurement points M2 using principal component analysis. Next, in the second step, of the measurement points M that were not selected, points whose distance from the provisional approximate line L02 exceeds a threshold are considered outliers, and the approximate line and approximation error are obtained for points other than the outliers.

Figure 11 is a second diagram illustrating a method for creating an approximate line in the second comparative example. In the second comparative example, the first and second steps are repeated as a third step, and the line resulting in the smallest approximation error is taken as approximate line L12. The plotted point "x" in Figure 11 is the measurement point M that was deemed an outlier in the second step. The approximate line L12 has a greater slope than the first comparative example, and also deviates more from the reference line Ls.

Figure 12 is a diagram illustrating the method for creating an approximate line in the first embodiment. Figure 14 shows an approximate line L03 created by the above-described method for creating an approximate line using the same set of measurement points as those used in the first and second comparative examples. As shown in Figure 12, it is clear that the approximate line L03 created by this embodiment has a smaller slope than the approximate line L01 of the first comparative example, and produces reasonable results that are closer to the reference line Ls. Note that in Figures 9, 11, and 13, the solid lines indicated by the symbols L01, L12, and L03 are line segments representing portions of the approximate lines obtained by the respective methods, and the lengths of the line segments are proportional to the standard deviation obtained from the orthogonal projection of each measurement point M onto the approximate line.

According to the sensor calibration device 1 of the present embodiment described above, an approximate line La is created from the position measurement results at each time of a pedestrian 6 moving on the reference line Ls, with respect to each of the reference line Ls and the representative reference point Ps0 set on the reference line Ls, and an estimated longest stay point LPe is calculated. The parameter calibration unit 22c can calibrate the attitude parameters a and e of the monocular camera 5 by rotating the approximate line La so that it approaches the reference line Ls. Furthermore, the estimated longest stay point LPe is calculated based on the density distribution of the measurement points M in the measurement point set Gm, which combines the position measurement results of the pedestrian 6 at each time, and is the center of stay of the pedestrian 6 moving using the representative reference point Ps0 as a landmark. As a result, the parameter calibration unit 22c can calibrate the position parameter _st (vector notation) by translating the approximate line La so that the calculated estimated longest stay point LPe approaches the representative reference point Ps0. This makes it possible to calibrate the position and orientation parameters of the monocular camera 5 by having the pedestrian 6 move on the reference line Ls with the representative reference point Ps0 as the center of movement within the detection range of the monocular camera 5. Therefore, even if there are only a few position measurement results of the pedestrian 6 by the position measurement unit 11, the position and orientation parameters of the monocular camera 5 can be calibrated, and therefore the position and orientation parameters can be calibrated in a short time.

Furthermore, according to the sensor calibration device 1 of this embodiment, when setting multiple reference points on the reference line Ls, the reference setting unit 12 sets the representative reference point Ps0 where the pedestrian 6 stays the most as the sign location 81. When calculating multiple estimated stay points Pe based on the density distribution of measurement points in the measurement point set Gm, the stay point calculation unit 22a sets the estimated stay point Pe with the highest density distribution of measurement points as the estimated longest stay point LPe. As a result, even when multiple reference points can be set, by setting the sign location 81 and the estimated longest stay point LPe, the approximate line La can be translated so that the sign location 81 and the estimated longest stay point LPe approach each other, thereby calibrating the position parameters of the monocular camera 5.

Furthermore, according to the sensor calibration device 1 of this embodiment, the approximate line La is created by principal component analysis performed on the estimated longest stay point LPe. As a result, the approximate line La passes through the estimated longest stay point LPe, so by translating the approximate line La, the sign location 81 and the estimated longest stay point LPe can be superimposed, and by rotating the approximate line La, the approximate line La can be superimposed on the reference line Ls. Therefore, the position and orientation parameters of the monocular camera 5 can be reliably calibrated.

Furthermore, according to the sensor calibration method of this embodiment, an approximate line La is created from the position measurement results at each time of the pedestrian 6 moving on the reference line Ls relative to the reference line Ls. The sensor calibration method calculates an estimated stay point Pe based on the density distribution of measurement points in the measurement point set Gm, which combines the position measurement results of the pedestrian 6 at each time relative to the representative reference point Ps0. As a result, the position and orientation parameters of the monocular camera 5 can be calibrated simply by the pedestrian 6 moving on the reference line Ls within the detection range of the monocular camera 5, with the representative reference point Ps0 as the center of stay, thereby enabling the position and orientation parameters to be calibrated in a short time.

Second Embodiment
13 is a diagram illustrating a sensor calibration method according to the second embodiment. The sensor calibration method according to the sensor calibration device of the second embodiment differs from the sensor calibration method according to the first embodiment ( FIG. 1 ) in that the position and orientation parameters of the sensor are calibrated using a plurality of estimated stay points selected in the stay point calculation step.

In the sensor calibration method using the sensor calibration device of the second embodiment, as shown in FIG. 13, in the reference setting step, multiple reference points Ps1, Ps2, Ps3, and Ps4 are set on the reference line Ls. As in the first embodiment, the multiple reference points Ps1, Ps2, Ps3, and Ps4 are preferably landmarks for pedestrians 6 moving along the row 7 of guiding blocks for the visually impaired, and may be warning blocks included in the row 7 of guiding blocks for the visually impaired, manholes on the road, etc. In the position measurement step, pedestrians 6 moving along the row 7 of guiding blocks for the visually impaired move so that their stay time at each of the multiple reference points Ps1, Ps2, Ps3, and Ps4 is longer than their stay time at other positions.

In the stay point calculation process, which follows the position measurement process, the stay point calculation unit 22a uses the set of measurement points stored in the memory unit 21 to calculate positions where the pedestrian 6 stayed for a relatively long time as estimated stay points. In this embodiment, the same number of clusters as the number of reference points set in the reference setting process are selected using a hierarchical clustering shortest distance method. In this embodiment, for each cluster, the median value of the cluster is set as the estimated stay points Pe1, Pe2, Pe3, and Pe4.

In the parameter calibration process, as in the first embodiment, the posture parameters are calibrated, and then the approximate line Lp is translated so that the estimated staying points Pe1, Pe2, Pe3, and Pe4 approach each of the multiple reference points Ps1, Ps2, Ps3, and Ps4, respectively, to calibrate the position parameters. In this embodiment, the position parameters are calibrated using the least absolute value method, which is more resistant to outliers than the least squares method. Specifically, the approximate line Lp is translated so that the sum of the absolute values of the distances between the estimated staying points Pe1, Pe2, Pe3, and Pe4 corresponding to each of the reference points Ps1, Ps2, Ps3, and Ps4 and the reference points Ps1, Ps2, Ps3, and Ps4 is minimized.

In the sensor calibration device of this embodiment described above, the reference setting unit 12 sets multiple reference points Ps1, Ps2, Ps3, and Ps4 on the reference line Ls. The stay point calculation unit 22a calculates multiple estimated stay points Pe1, Pe2, Pe3, and Pe4 based on the density distribution of measurement points M in the measurement point set Gm. The parameter calibration unit 22c calibrates the position parameters by translating the approximate line La so that each of the multiple estimated stay points Pe1, Pe2, Pe3, and Pe4 approaches each of the multiple reference points Ps1, Ps2, Ps3, and Ps4. This improves the calibration accuracy of the position and orientation parameters of the monocular camera 5 compared to translating the approximate line so that one estimated stay point approaches and overlaps with one reference point.

Furthermore, according to the sensor calibration device of this embodiment, the parameter calibration unit 22c calibrates the position parameters using the least absolute value method, which is less susceptible to the influence of outliers. This further improves the calibration accuracy of the position parameters of the monocular camera 5.

Third Embodiment
14 is a schematic diagram showing a schematic configuration of a sensor calibration device according to the third embodiment. The sensor calibration device according to the third embodiment differs from the sensor calibration device according to the first embodiment (FIG. 1) in that it includes two measuring instruments and calibrates the position and orientation parameters of two sensors.

The sensor calibration device 3 of this embodiment includes two measuring instruments 10a and 10b and a calibration processor 30. Each of the two measuring instruments 10a and 10b of this embodiment is connected to a monocular camera 5a and 5b, respectively. The two measuring instruments 10a and 10b are connected to the calibration processor 30, for example, via wireless communication. The calibration processor 30 receives the measurement and setting details of each of the two measuring instruments 10a and 10b, and can output calibrated position and orientation parameters to each of the two measuring instruments 10a and 10b.

Measuring instrument 10a includes a position measurement unit 11a and a reference setting unit 12a. Measuring instrument 10a is electrically connected to monocular camera 5a and receives images captured by monocular camera 5a. The position measurement unit 11a measures the position of pedestrian 6a. The reference setting unit 12a sets a reference line 7a and a reference point 81a on reference line 7a on the image captured by monocular camera 5a. Measuring instrument 10b includes a position measurement unit 11b and a reference setting unit 12b. Measuring instrument 10b is electrically connected to monocular camera 5b and receives images captured by monocular camera 5b. The position measurement unit 11b measures the position of pedestrian 6b. The reference setting unit 12b sets a reference line 7b and a reference point 81b on reference line 7b on the image captured by monocular camera 5b.

Figure 15 is a diagram illustrating the positional relationship between two monocular cameras 5a and 5b. Figure 15 shows a map of an urban area 9 as an example. The two monocular cameras 5a and 5b configured with the sensor calibration device 3 are installed at separate locations, for example, in one block 9a of the urban area 9. In this embodiment, the two reference setting units 12a and 12b corresponding to the two monocular cameras 5a and 5b respectively set reference lines 7a and 7b so that they intersect. Here, the intersection of the reference lines 7a and 7b is referred to as intersection point 7c (see Figure 15).

The calibration processor 30 includes a memory unit 31 and a CPU 32. The calibration processor 30 calibrates the position and orientation parameters of the monocular cameras 5a and 5b using the position measurement results of the two pedestrians 6a and 6b measured by the measuring devices 10a and 10b, respectively, the provisional position and orientation parameters, and the reference lines 7a and 7b and reference points 81a and 81b set by the reference setting unit 12.

The memory unit 31 stores, as a set of measurement points, the position measurement results at each time of pedestrians 6a and 6b moving along the reference lines 7a and 7b, measured by the two position measurement units 11a and 11b, respectively. The memory unit 31 also stores various calculation results from the CPU 32 and outputs this stored information to the CPU 32 as appropriate.

The CPU 32 functions as a stay point calculation unit 32a, a straight line approximation unit 32b, and a parameter calibration unit 32c, and controls each unit of the sensor calibration device 3, including the two measuring devices 10a and 10b. The CPU 32 calculates estimated stay points from the position measurement results of pedestrians 6a and 6b measured by the two position measurement units 11a and 11b, and creates approximate straight lines, and calibrates position and orientation parameters from the relationship between the estimated stay points and the reference point, and the relationship between the approximate straight line and the reference straight line.

The stay point calculation unit 32a has the same function as the stay point calculation unit 22a in the first embodiment. The stay point calculation unit 32a calculates estimated stay points using two sets of measurement points that are the position measurement results from the two position measurement units 11a and 11b.

The straight line approximation unit 32b includes a candidate creation unit 321b, a determination unit 322b, and a selection unit 323b. The candidate creation unit 321b performs principal component analysis using the coordinate values of the reference points of each of the two sets of measurement points to create approximate straight line candidates. The determination unit 322b determines whether to adopt the combination of approximate straight line candidates created by the candidate creation unit 321b. The selection unit 323b selects a combination of approximate straight lines to be used to calibrate the position and orientation parameters from the combination of approximate straight line candidates adopted by the determination unit 322b. The detailed functions of the straight line approximation unit 32b will be described later.

The parameter calibration unit 32c has the same functions as the parameter calibration unit 22c in the first embodiment. The parameter calibration unit 32e calibrates the position and orientation parameters using the approximate line selected by the selection unit 32d, the reference line, and the provisional position and orientation parameters.

Next, the sensor calibration method of this embodiment will be described. In the sensor calibration method of this embodiment, the reference setting units 12a and 12b first set, for each of the two monocular cameras 5a and 5b, reference lines 7a and 7b, a location 81a on the reference line 7a where a marker 8a is fixed as a reference point 81a, and a location 81b on the reference line 7b where a marker 8b is fixed as a reference point 81b. In this embodiment, the reference lines 7a and 7b are set so that they intersect, as shown in FIG. 15. The provisional position and orientation parameters of the monocular camera 5a are set in a three-dimensional space with the reference line 7a as the x-axis and the reference point 81a as the origin. The provisional position and orientation parameters of the monocular camera 5b are set in a three-dimensional space with the reference line 7b as the x-axis and the reference point 81b as the origin. In other words, the reference lines 7a and 7b and the reference points 81a and 81b are set in different coordinate systems.

Next, the position measurement units 11a and 11b measure the positions of the pedestrians 6a and 6b moving on the reference lines 7a and 7b, respectively, and the memory unit 31 stores the position measurement results of the pedestrians 6a and 6b at each time as a set of measurement points. The staying point calculation unit 32a then calculates the estimated staying points of the pedestrians 6a and 6b from the position measurement results stored in the memory unit 31 using a method similar to that used in the first embodiment.

Next, the candidate creation unit 321b creates approximate line candidates for each of the measurement point sets for the two monocular cameras 5a and 5b using a method that combines principal component analysis and the RANSAC method for the coordinate values of the reference points, as in the first embodiment. In this embodiment, the candidate creation unit 321b calculates approximate line candidates until the number of times that they have not been rejected reaches a preset number, as in the line approximation process of the first embodiment. The candidate creation unit 321b creates a combination of approximate line candidates by combining the approximate line candidates created for each of the two monocular cameras 5a and 5b.

Next, the determination unit 322b first converts the reference lines 7a, 7b and the reference points 81a, 81b into a common coordinate system, for example, the coordinate system of the Geospatial Information Authority of Japan map, and sets a reference broken line 9c formed from the two reference lines 7a, 7b (see FIG. 15). When setting the reference broken line 9c, the determination unit 322b acquires information on the distances Dma, Dmb (hereinafter referred to as "measured distances") from each of the two reference points 81a, 81b on the reference broken line 9c to the intersection 7c of the reference lines 7a, 7b (see FIG. 15). The measured distances Dma, Dmb may be distances actually measured using, for example, a roller distance meter, or may be distances calculated from a map.

Next, the determination unit 322b converts the approximate line candidates for each of the two monocular cameras 5a and 5b created by the candidate creation unit 321b into a coordinate system common to the reference broken line 9c, and calculates the distance from the converted estimated stay point to the intersection of the two approximate line candidates (hereinafter referred to as the "estimated distance"). The determination unit 322b compares the absolute value of the difference between the measured distance and the estimated distance with an error (hereinafter referred to as the "intersection rejection threshold") that takes into account the procedures for distance measurement and setting the reference broken line. If the error is less than or equal to the intersection rejection threshold, the determination unit 322b adopts the combination of the two approximate line candidates, and if the error is greater than the intersection rejection threshold, it rejects the combination of the two approximate line candidates.

For example, suppose the candidate creation unit 321b creates a combination of approximate line candidate Lpa1 (estimated stay point Pea1) for monocular camera 5a and approximate line candidate Lpb1 (estimated stay point Peb1) for monocular camera 5b. For this combination of approximate line candidates (Lpa1, Lpb1), the estimated distance Dea is the distance from the intersection of approximate line candidate Lpa1 and approximate line candidate Lpb1 to estimated stay point Pea1, and the estimated distance Deb is the estimated distance from the intersection of approximate line candidate Lpa1 and approximate line candidate Lpb1 to estimated stay point Peb1. In this case, if both the difference between the estimated distance Dea and the measured distance Dma and the difference between the estimated distance Deb and the measured distance Dmb are less than or equal to the intersection rejection threshold, the combination of approximate line candidates (Lpa1, Lpb1) is adopted. On the other hand, even if the estimated distance Dea nearly matches the measured distance Dma, if the difference between the estimated distance Deb and the measured distance Dmb is greater than the intersection rejection threshold, the combination of approximate line candidates (Lap1, Lpb1) is rejected. Note that the intersection rejection threshold may be set to a uniform value if the two measured distances are nearly the same. The intersection rejection threshold may also be set as a ratio to the measured distance. In this way, the determination unit 322b determines whether to accept or reject the combination of multiple approximate line candidates created by the candidate creation unit 321b.

Next, the selection unit 323b selects one combination from the combinations of approximate line candidates adopted by the determination unit 322b. In this embodiment, the selection unit 323b selects, from the adopted combinations of approximate line candidates, the combination that minimizes the aggregate value of the approximation errors of the two approximate line candidates. The aggregate value of the errors of the approximate line candidates may be a simple average or a weighted average based on the number of measurement points.

According to the sensor calibration device 3 of this embodiment described above, the linear approximation unit 32b sets a reference broken line 9c formed from two reference lines 7a, 7b set for each of the two monocular cameras 5a, 5b. For each of the two reference lines 7a, 7b, the linear approximation unit 32b acquires the measured distance from the intersection 7c of the two reference lines 7a, 7b to the respective reference points 81a, 81b on the two reference lines 7a, 7b. The linear approximation unit 32b also acquires the estimated distance from the estimated stay point to the intersection of the two approximate line candidates, and calibrates the position and orientation parameters of the monocular cameras 5a, 5b using the positions of the reference points 81a, 81b, the measured distance, and the estimated distance. This allows the reference lines 7a, 7b to be determined by two points: the reference point and the intersection, thereby improving the accuracy of the approximate line. This improves the calibration accuracy of the sensor's position and orientation parameters.

<Modification of this embodiment>
The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[Modification 1]
In the above-described embodiment, the approximate line candidate is created using a method that combines the principal component analysis method and the RANSAC method in the line approximation process. However, the method for creating the approximate line candidate is not limited to this. The approximate line candidate may be created using only the principal component analysis method.

[Modification 2]
In the first embodiment, the parameter calibration unit 22c calibrates the orientation parameters a and e and then the position parameter s. However, the order in which the position and orientation parameters of the monocular camera 5 are calibrated is not limited to this. The orientation parameter may be calibrated by translating the approximate line so that the estimated longest stay point LPe overlaps with the reference point Ps, thereby calibrating the position parameter s, and then by rotating the approximate line so that the approximate line La overlaps with the reference line Ls.

[Modification 3]
In the first and second embodiments, the sensor calibration device 1 calibrates one monocular camera 5. However, the "sensor" does not have to be the monocular camera 5, and as described above, may be a stereo camera, radar, LiDAR (laser radar), sonar, etc. The same applies to the third embodiment in which two monocular cameras 5 are configured.

[Modification 4]
In the second embodiment, the parameter calibration unit calibrates the location parameters by bringing a plurality of estimated stay points closer to a plurality of reference points using the least absolute value method, which is less susceptible to the influence of outliers. The method of calibrating the location parameters by the parameter calibration unit is not limited to this. For example, the least squares method may be used.

[Modification 5]
In the third embodiment, the selector 323b selects, from among the combinations of adopted approximate line candidates, the combination for which the aggregated value of the approximation errors of the two approximate line candidates is the smallest. However, the method for selecting combinations by the selector 323b is not limited to this. For example, one approximate line candidate is calculated for each set of measurement points, and those combinations are retained unless they are rejected. When the number of retained combinations reaches a predetermined number, the combination for which the aggregated value of the approximation errors is the smallest may be selected. This method can improve the accuracy of the approximate line depending on the rejection conditions, thereby reducing the effort required to calculate approximate line candidates.

[Modification 6]
In the first and second embodiments, the sensor calibration device 1 is configured from two devices, namely, the measuring device 10 and the calibration processor 20. However, the sensor calibration device may be configured such that the measuring device and the calibration processor are integrated into one device, or may be installed in different locations as in the third embodiment.

This aspect has been described above based on embodiments and variations, but the above-described embodiments are intended to facilitate understanding of this aspect and are not intended to limit this aspect. This aspect may be modified or improved without departing from its spirit or the scope of the claims, and equivalents are included in this aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate.

1, 3... Sensor calibration device 5, 5a, 5b... Monocular camera 6, 6a, 6b... Pedestrian 10, 10a, 10b... Measuring instrument 11, 11a, 11b... Position measurement unit 12, 12a, 12b... Reference setting unit 20, 30... Calibration processor 21, 31... Memory unit 22a, 32a... Stay point calculation unit 22b, 32b... Line approximation unit 22c, 32c... Parameter calibration unit La... Approximate line Ls... Reference line LPe... Estimated longest stay point Ps0... Representative reference point Ps1, Ps2, Ps3, Ps4... Reference point Pe, Pe1, Pe2, Pe3, Pe4, Pea1, Peb1... Estimated stay point a, a _c , a _t ... Azimuth angle (attitude parameter)
e, e _c , e _t ... elevation angle (attitude parameter)
s, s _c , s _t ... coordinate values (position parameters)

Claims (7)

A sensor calibration device used to calibrate the position and orientation of a sensor, comprising:
a reference line setting unit that sets a reference line;
a reference point setting unit that sets a reference point on the reference line that will be the center of a position of an object moving in a three-dimensional space;
a position measurement unit that measures the position of the object using position and orientation parameters that are set in advance as parameters that represent the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space;
a storage unit that stores, as a set of measurement points, position measurement results at each time of the object moving on the reference straight line obtained by the position measurement unit;
a stay point calculation unit that calculates an estimated stay point of the object based on a density distribution of the measurement points in the measurement point set;
a straight line approximation unit that creates an approximation straight line of the set of measurement points using the set of measurement points stored in the storage unit;
a parameter calibration unit that calibrates the position and orientation parameters using the reference line, the reference point, the approximate line, and the estimated stay point;
The parameter calibration unit
calibrating position parameters by translating the approximation line so that the estimated stay point approaches the reference point;
calibrating the attitude parameters by rotating the approximated line so that the approximated line approaches the reference line;
Sensor calibration equipment. 2. The sensor calibration device of claim 1,
The reference point setting unit
A plurality of the reference points are set on the reference line;
Among the plurality of set reference points, the reference point where the object stays most often is set as a representative reference point;
The staying point calculation unit
calculating the plurality of estimated stay points based on a density distribution of the measurement points in the set of measurement points;
Among the plurality of estimated stay points, an estimated stay point having the highest density distribution of measurement points in the set of measurement points is set as an estimated longest stay point;
the parameter calibration unit calibrates the position parameters by translating the approximation line so that the estimated longest stay point approaches the representative reference point;
Sensor calibration equipment. 3. The sensor calibration device of claim 2,
the straight line approximation unit performs principal component analysis on the estimated longest stay point to create an approximate straight line of the set of measurement points.
Sensor calibration equipment. 2. The sensor calibration device of claim 1,
the reference point setting unit sets a plurality of the reference points on the reference straight line;
the stay point calculation unit calculates the same number of estimated stay points as the number of reference points set by the reference point setting unit,
the parameter calibration unit calibrates the position parameters by translating the approximation line so that each of the plurality of estimated stay points approaches each of the corresponding plurality of reference points;
Sensor calibration equipment. 5. The sensor calibration device of claim 4,
the parameter calibration unit calibrates the position parameters by translating the approximation line so as to minimize a sum of absolute values of distances between a plurality of the reference points and a plurality of the estimated stay points corresponding to the plurality of the reference points, respectively.
Sensor calibration equipment. 6. A sensor calibration device according to claim 1, comprising:
the sensor calibration device is connected to each of two sensors installed at different locations;
the reference line setting unit sets reference lines that intersect with each other for each of the two sensors;
the reference point setting unit sets a reference point on each of the two reference straight lines;
The linear approximation unit
acquire information about the distance between the reference point and the intersection point on a reference broken line formed by the two intersecting reference straight lines and the intersection point of the two reference straight lines;
creating an approximation line of the set of measurement points using information about the distance between the reference point and the intersection point;
Sensor calibration equipment. A sensor calibration method for calibrating a position and orientation of a sensor using a sensor calibration device , comprising:
A step of setting a reference line;
setting a reference point on the reference line as a center of stay of an object moving in three-dimensional space;
measuring the position of the object using position and orientation parameters that are preset as parameters representing a temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space;
a step of storing , as a set of measurement points, position measurement results at each time of the object moving on the reference straight line measured in the step of measuring the position of the object ;
calculating an estimated stay point of the object based on a density distribution of the measurement points in the set of measurement points;
creating an approximation line of the set of measurement points using the set of measurement points stored;
a calibration step of calibrating the position and orientation parameters using the reference line, the reference point, the approximate line, and the estimated stay point,
In the calibration step,
calibrating position parameters by translating the approximation line so that the estimated stay point approaches the reference point;
calibrating the attitude parameters by rotating the approximated line so that the approximated line approaches the reference line;
Sensor calibration methods.