JP7579137B2 - Observation device and method for calibrating observation device - Google Patents

Description

This disclosure relates to an observation device and a method for calibrating the observation device.

In recent years, safe driving support systems have been developed that install observation devices on the roadside to detect vehicles and pedestrians, and notify approaching vehicles of dangers based on the distance to the vehicle or pedestrian. Observation devices are generally installed close to the road and can be affected by vibrations from passing vehicles. If the installation angle of the observation device fluctuates due to vibrations, the distance to the vehicle or pedestrian detected by the observation device will no longer be detected accurately. For this reason, the observation device needs to be calibrated to maintain the accuracy of its detection. For example, Patent Document 1 discloses a technology for calibrating a roadside unit, which is a type of observation device, using a measurement vehicle equipped with a marker.

JP 2012-10036 A

However, there is a demand for technology that can calibrate observation equipment without using measurement vehicles, etc. Furthermore, such technology would be even more useful if it could detect the appropriate timing for calibration.

The purpose of this disclosure, made in consideration of these circumstances, is to provide an observation device and a calibration method for an observation device that can detect the occurrence of a deviation that requires calibration without using a measurement vehicle or the like.

An observation device according to an embodiment of the present disclosure includes:
A detection device that captures an image of a target including a marker;
a storage unit that stores positions of a plurality of recognition points in the photographed image, the recognition points being intersections between an outline of the marker included in the photographed image and an end portion of the photographed image;
and a control unit that determines that a shift has occurred in the position or orientation of the detection device when the number of recognition points among the plurality of recognition points that have changed from their stored positions by a predetermined distance or more satisfies a predetermined condition.

A method for calibrating an observation device according to an embodiment of the present disclosure includes:
A method for calibrating an observation device including a detection device, comprising:
Obtaining a photographed image of a photographing target including a marker;
storing positions of a plurality of recognition points in the photographed image, the recognition points being points at which an outline of the marker included in the photographed image intersects with an end portion of the photographed image;
determining that a shift has occurred in the position or orientation of the detection device when the number of recognition points among the plurality of recognition points that have changed from their stored positions by a predetermined distance or more satisfies a predetermined condition.

The present disclosure provides an observation device and a calibration method for an observation device that can detect the occurrence of a deviation that requires calibration without using a measurement vehicle or the like.

FIG. 1 is a diagram showing an example of the configuration of a communication system including an observation device according to an embodiment. FIG. 2 is an example of a functional block diagram of an observation device according to an embodiment. FIG. 3 is a diagram illustrating an example of the configuration of an observation device according to an embodiment. FIG. 4 is a diagram illustrating an example of installation of an observation device according to an embodiment. FIG. 5 is a diagram for explaining the deviation and calibration of the detection device. FIG. 6 is a diagram for explaining the deviation and calibration of the detection device. FIG. 7 is a diagram illustrating recognition points and edges. FIG. 8 is a diagram illustrating the amount of change in the position or orientation of the detection device. FIG. 9 is a flowchart showing an example of a calibration method for an observation apparatus according to an embodiment.

(Communication Systems)
1 shows a configuration example of a communication system 1 including an observation device 5 according to an embodiment. The communication system 1 is, for example, a safe driving support communication system of an intelligent transport system (ITS). The safe driving support communication system is also called a safe driving support system or a safe driving support wireless system.

The observation device 5 is a device that observes vehicles, objects, and people on the road, and is, for example, a roadside unit or a surveillance camera device. In this embodiment, the observation device 5 is a roadside unit that is placed near an intersection where multiple roads 7 (roadways) intersect, and observes the road surface. Here, the observation device 5 may be placed on the roadside other than at the intersection. Details of the observation device 5 will be described later.

In the communication system 1, the observation device 5 and a vehicle 6, such as an automobile traveling on a road 7, can communicate wirelessly with each other. Furthermore, multiple vehicles 6 may be capable of communicating wirelessly with each other. The communication system 1 may also be capable of communicating wirelessly with electronic devices carried by pedestrians 9. The electronic devices are, for example, mobile terminal devices such as smartphones.

The observation device 5 can notify the vehicle 6 of safe driving support information for supporting the driver of the vehicle 6 in safe driving. The safe driving support information may include, for example, information on the illumination of the traffic lights 4, information on road regulations, road linear information indicating the shape of the intersection where the observation device 5 is located (the shape of the road 7), and the like. The observation device 5 also detects the vehicles 6 and pedestrians 9 on the road being observed. The observation device 5 can detect, for example, the pedestrians 9 crossing the crosswalk 3. The observation device 5 can detect the vehicles 6 approaching the intersection. The observation device 5 can detect the distance to the detected objects such as the vehicles 6 and pedestrians. The observation device 5 can include information on the detected vehicles 6 and pedestrians 9 in the safe driving support information and notify the vehicle 6. The observation device 5 can also include information notified from the vehicle 6 in the safe driving support information and notify other vehicles 6. As described above, the observation device 5 may also be able to notify such information to an electronic device carried by the pedestrian 9.

The vehicle 6 may transmit vehicle information to the observation device 5 or the like periodically, for example. The vehicle information may include, for example, information related to the position, speed, and blinker of the vehicle 6. The vehicle 6 may receive various information notified from the observation device 5 or the like by the electronic device mounted on it. The electronic device mounted on the vehicle 6 may be, for example, a car navigation system. The electronic device mounted on the vehicle 6 may support the driver in safe driving by notifying the driver of a warning or the like based on the information notified from the observation device 5. The notification to the driver may be, for example, when turning at an intersection, that another vehicle 6 is approaching from the front, or that there is a pedestrian 9 on the crosswalk 3 ahead after the turn.

In this way, the communication system 1 assists the driver of the vehicle 6 in driving safely. Here, the vehicle 6 is not limited to an automobile. For example, the vehicle 6 may include a motorcycle, a bus, a streetcar, and a bicycle.

(Functional block diagram of the observation device)
2 is a functional block diagram of the observation device 5 according to this embodiment. The observation device 5 includes a detection device 50, a communication unit 51, a storage unit 52, and a control unit 53. There may be a plurality of detection devices 50. Each of the plurality of detection devices 50 may be capable of operating independently.

The detection device 50 is a device that detects vehicles 6 and pedestrians 9. The detection device 50 includes a sensor. The detection device 50 may be, for example, a visible light camera including an image sensor. The detection device 50 may also be an infrared camera including an infrared sensor. In this embodiment, the detection device 50 includes at least a visible light camera, and captures an image of a subject including a marker M, which will be described later.

Here, the sensors included in the detection device 50 are not limited to image sensors and infrared sensors. The sensors included in the detection device 50 may include, for example, a 3D laser scanner. The detection device 50 outputs the detection signals detected by the sensors, the above-mentioned captured images, etc. to the control unit 53.

The communication unit 51 is controlled by the control unit 53 to perform wireless communication with the vehicle 6. The communication unit 51 may be composed of a communication circuit and an antenna. The antenna may be, for example, an omnidirectional antenna. The communication unit 51 may perform wireless communication using, for example, the 700 MHz band allocated to ITS. The communication unit 51 may also perform wireless communication using, for example, a wireless LAN (Local Area Network).

The communication unit 51 performs various processes, such as amplification, on the signal received by the antenna, and outputs the processed received signal to the control unit 53. The control unit 53 performs various processes on the input received signal to obtain information contained in the received signal. The control unit 53 also outputs a transmission signal containing information to the communication unit 51. The communication unit 51 performs various processes, such as amplification, on the input transmission signal, and wirelessly transmits the processed transmission signal from the antenna.

The storage unit 52 may function as a memory that stores various types of information. The storage unit 52 may store, for example, a program executed in the control unit 53 and the results of processing executed in the control unit 53. The storage unit 52 may also function as a work memory for the control unit 53. The storage unit 52 may be configured, for example, with semiconductor memory such as ROM and RAM, but is not limited to this and may be any storage device. The storage unit 52 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

In this embodiment, the memory unit 52 stores data related to the calibration executed by the control unit 53. As data related to the calibration, the memory unit 52 may store recognition point data 521 including the positions of the recognition points described below. The memory unit 52 may also store ridge line data 522 described below. The memory unit 52 may also store coordinate-distance correspondence data indicating the correspondence between the position of the subject in the captured image and the distance to the subject. The memory unit 52 may also store a program for causing the control unit 53 to calculate the amount of change in the detection device 50 and execute the calibration described below.

The control unit 53 controls the other components of the observation device 5 to comprehensively manage the operation of the observation device 5. The control unit 53 may include one or more processors. The processor may load a program from an accessible memory to realize various functions of the observation device 5. The processor may include at least one of a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA (Field-Programmable Gate Array). The control unit 53 may include at least one of a System-on-a-Chip (SoC) and a System in a Package (SiP) in which one or more processors work together.

The control unit 53 can generate safe driving support information based on the acquired detection signal from the detection device 50. For example, the control unit 53 may measure the distance to the pedestrian 9 included in the captured image acquired from the detection device 50, and warn the vehicle 6 when it determines that the pedestrian 9 is approaching. In addition, in this embodiment, when the number of recognition points described below satisfies a predetermined condition, the control unit 53 calculates the amount of change in the position or orientation of the detection device 50 and performs calibration based on the calculated amount of change. Details of these processes will be described later.

(Configuration of observation device)
Fig. 3 is a diagram showing an example of the configuration of the observation device 5 according to this embodiment. As shown in Fig. 3, the observation device 5 may be provided as an on-road observation unit including a plurality of detection devices 50, which are connected to a connection portion 57 to determine the relative positional relationship between them.

The connection part 57 is a member for connecting the detection device 50. In this embodiment, the connection part 57 is a metal frame-shaped member. However, the material and shape of the connection part 57 are not limited. Here, as shown in FIG. 3, orthogonal coordinates can be defined for the observation device 5, which is a road observation unit. For example, the u-axis direction, v-axis direction, and w-axis direction each correspond to the extending directions of the sides that intersect at right angles at the vertex of the connection part 57. As shown in FIG. 3, in the observation device 5 according to this embodiment, multiple detection devices 50 are arranged side by side in the w-axis direction.

Figure 4 shows an example of the installation of the observation device 5. The observation device 5 is installed on a pillar 59 near an intersection where the road 7 to be observed intersects, for example. A connection part 57 is fixed to the pillar 59, and the orientation of the detection device 50 may be adjusted with respect to the connection part 57. Here, when the observation device 5 is installed to observe the road, the w-axis direction is the vertical direction. Here, in Figure 4, the z-axis direction is the vertical direction, and the x-axis and y-axis indicate horizontal directions perpendicular to the z-axis.

Here, the detection device 50 may be attached to the connection part 57 by, for example, a screw 56 (see FIG. 3), a fitting, or the like, and the orientation may be adjusted. The detection device 50 may have an adjustment mechanism for changing the installation angle with respect to the connection part, and may be capable of rotating in, for example, the pan, tilt, and yaw directions. The adjustment mechanism may be a structure such as a camera platform, but is not limited to this. The detection device 50 may have a structure that allows the movable part to be adjusted in three axial directions relative to the base part, for example, by having axes in the u-axis, v-axis, and w-axis directions shown in FIG. 3. When the connection part 57 is fixed to the pillar 59, the detection device 50 is adjusted to observe the road at an appropriate depression angle, and is fixed by screws or the like so that the orientation does not change.

(calibration)
As described above, the observation device 5 is installed, for example, near an intersection where the road 7 to be observed intersects, and therefore may be affected by vibrations from traveling vehicles, etc. For example, continued exposure to vibrations from traveling vehicles, etc. may cause the screws 56 for fixing the detection device 50 to the connection part 57, the fitting parts, etc. to loosen or rotate, causing the detection direction of the detection device 50 to shift. Alternatively, the detection direction of the detection device 50 may shift due to an impact being applied to the pillar 59 on which the observation device 5 is attached. In such cases, calibration is required so that the observation device 5 can continue to perform accurate distance measurements.

5 and 6 are diagrams for explaining the deviation and calibration of the detection device 50. FIG. 5 illustrates an example of the detection device 50 at the initial position photographing a marker M 50 m ahead, and the photographed image. The number of markers M is not limited to one, and may be multiple. Here, the marker M is an object located at a specific distance from the observation device 5 among the photographed objects of the detection device 50. Therefore, it is possible to use the marker M to correspond the coordinates in the photographed image to the distance from the observation device 5 to the photographed object. As the marker M, for example, an object that is difficult to move, such as a building, an installation indicating a bus stop (hereinafter, a bus stop), a signboard, or a white line drawn on the road 7, is selected. In the photographed image shown in FIG. 5, the marker M is located at the coordinate (50, 40), and this coordinate is associated with a position 50 m ahead from the observation device 5. Here, the coordinates of the photographed image are the horizontal direction as the X axis and the vertical direction as the Y axis, and the upper left vertex is the coordinate (0, 0). Information on the coordinates in the captured image and the distance from the observation device 5 may be stored in the storage unit 52 as coordinate-distance correspondence data. The coordinate-distance correspondence data may be data in which the distance from the observation device 5 is associated with the coordinates of all pixels in the captured image, or may be data in which the coordinates of some pixels are associated with each other.

Figure 6 illustrates an example of the detection device 50 photographing a marker M 50 m ahead after it has shifted in the tilt direction from the state of Figure 5, and the photographed image. As shown in Figure 6, the marker M, which was at the coordinate (50, 40) in Figure 5, has moved to the coordinate (45, 45). When the detection device 50 changes to the state of Figure 6, the observation device 5 cannot perform accurate distance measurement with the initial correspondence of the coordinate (50, 40) as a position 50 m ahead. Therefore, calibration is required to newly correspond the coordinate (45, 45) as a position 50 m ahead. Calibration is a correction of the correspondence between the position (coordinate) in the photographed image and the distance between the photographed object and the observation device at that position. In this embodiment, calibration is performed by updating the coordinate-distance correspondence data stored in the memory unit 52. The coordinate-distance correspondence data is data that defines the correspondence between the position (coordinate) in the photographed image and the distance to the photographed object at that coordinate. When performing distance measurement, the control unit 53 of the observation device 5 calculates the distance to the subject to be photographed based on the coordinate-distance correspondence data read from the memory unit 52.

Here, depending on the installation location of the observation device 5, a movable object such as a signboard attached to the road may be selected as the marker M. Or, an object may be placed on the white line selected as the marker M, making it impossible to correctly recognize it, or it may disappear or change shape due to deterioration over time. In such a case, if the detection device 50 erroneously determines that the marker M has moved or that it has been recognized as having a different shape and performs calibration, the observation device 5 cannot perform accurate distance measurement. Therefore, it is necessary to appropriately determine that the detection device 50 has shifted. In addition, the detection device 50 may have a shift other than the tilt direction (for example, a shift in the pan direction, a shift in one direction, etc.). Therefore, it is necessary to appropriately determine the direction and amount of the shift. The observation device 5 according to this embodiment detects the occurrence of a shift that requires calibration and calculates the amount of change in the position or orientation of the detection device 50 using the method described below.

(Recognition point)
The control unit 53 of the observation device 5 sets the marker M according to the installation location of the observation device 5. Then, the control unit 53 sets the intersection point where the outline of the marker M included in the photographed image acquired from the detection device 50 intersects with the end of the photographed image as the recognition point. The control unit 53 determines one or more markers M so that there are multiple recognition points. The control unit 53 also sets a ridge line. The ridge line is a part of the outline of the marker M in the photographed image and is a line extending from the recognition point. The ridge line is selected so as to be a straight line or a line that can be approximated to a straight line. Here, the marker M forming the recognition point and the ridge line may be an object used to generate the coordinate-distance correspondence data, or may be another object. Various objects included in the photographed image acquired from the detection device 50 can be the marker M forming the recognition point and the ridge line.

Figure 7 is a diagram for explaining recognition points and ridge lines. Figure 7 includes an example of a captured image acquired from the detection device 50, an example of recognition point data 521, and an example of ridge line data 522. In the captured image shown in Figure 7, the marker M is a building in the upper left, a white line on the road 7 in the center, and a bus stop in the lower right. Here, the ends of the captured image constitute four sides (upper side, right side, lower side, and left side). Recognition point P1 is an intersection point where the outline of the bus stop intersects with the lower side. Recognition point P2 is an intersection point where the outline of the building intersects with the left side. Also, recognition point P3 is an intersection point where the outline of the white line on the road 7 intersects with the upper side. In the captured image shown in Figure 7, other circled parts can be recognition points. The control unit 53 may set all such intersection points as recognition points, or may select some of them as recognition points. The control unit 53 sets multiple recognition points. As an example, the control unit 53 may set 10 to 20 recognition points. Here, the recognition points must be present on three or more of the four sides that constitute the end of the captured image so that the amount of change in the position or orientation of the detection device 50 can be calculated. The recognition points may be distributed so that there are an equal number of recognition points on each side of the captured image. The positions of the recognition points on each side may be set so that they are separated by a predetermined distance or more in the captured image. This is because adjacent recognition points are often formed by the same object, or by different objects that are close to each other. It is desirable that the objects in the captured image that form the recognition points and ridgelines are different objects for each recognition point and ridgeline. This is because if many recognition points are set by one object, the positional deviation of the object may be erroneously determined as a deviation of the detection device. Here, the recognition points may be set automatically by the control unit by image recognition of the object in the captured image, or may be set by the operator selecting them from the captured image.

In the captured image shown in FIG. 7, ridge line L1 is part of the contour of the bus stop and extends from recognition point P1. Furthermore, ridge line L2 is part of the contour of the white line of road 7 and extends from recognition point P3. Parts of the contours of other markers M may be set as ridge lines. The control unit 53 may set all such parts as ridge lines, or may select parts as ridge lines. The control unit 53 may set one or more ridge lines. As an example, the control unit 53 may set 10 to 20 ridge lines.

The control unit 53 stores information about the set recognition points in the memory unit 52 as recognition point data 521. The recognition point data 521 includes the positions (coordinates) of multiple recognition points in the captured image. In the example of FIG. 7, the recognition point data 521 associates the recognition point number, the position on the captured image, and the angle change of the detection device 50 for a one-pixel shift of the recognition point. Here, the X-direction value of the angle change indicates the amount of change in the pan direction. Also, the Y-direction value of the angle change indicates the amount of change in the tilt direction.

The control unit 53 also stores information about the set ridge line in the memory unit 52 as ridge line data 522. The ridge line data 522 includes the positions (coordinates) of the start and end points of the ridge line in the captured image. In the example of FIG. 7, the ridge line data 522 associates the number in the ridge line data, the positions of the start and end points, and the angle that the ridge line makes with the edge of the captured image (i.e., the outer periphery of the image) in the captured image.

When the control unit 53 acquires a new captured image from the detection device 50, it extracts the recognition points and ridge lines from the new captured image and determines their positions (coordinates). The control unit 53 determines whether the detection device 50 has shifted by comparing the recognition point data 521 and ridge line data 522 stored in the memory unit 52 with information such as their positions. If the control unit 53 determines that the detection device 50 has shifted, it calculates the amount of change in the detection device 50. Based on the calculated amount of change, the control unit 53 also performs calibration of the correspondence between the position of the subject in the captured image and the distance to the subject.

Figure 8 is a diagram illustrating the amount of change in the position or orientation of the detection device 50. In the example of Figure 8, the captured image It0 is a previously captured image that includes recognition points and ridgelines corresponding to the recognition point data 521 and ridgeline data 522 stored in the memory unit 52. The captured image It1 is a new captured image.

The control unit 53 sets multiple recognition points (dashed circles in FIG. 8) and ridgelines in the captured image It0. The control unit 53 specifies the positions of the multiple recognition points and stores recognition point data 521 in the memory unit 52. The control unit 53 also specifies the inclination (angle) of the ridgelines and stores ridgeline data 522 in the memory unit 52. When a new captured image It1 is acquired, the control unit 53 extracts recognition points (solid circles at the ends of the captured image It1 in FIG. 8) and ridgelines in the captured image It1. The objects that make up the recognition points in the captured image It1 are the same objects as the objects that make up the recognition points in the captured image It0. The positions of the recognition points and the inclination of the ridgelines in the captured images It0 and It1 are compared to determine whether the detection device 50 has shifted.

The control unit 53 determines that the detection device 50 has shifted when the number of recognition points that have changed by a predetermined distance or more from the stored position among the multiple recognition points set for the captured image It0 satisfies a predetermined condition. Here, the predetermined distance is set so that the detection device 50 is not mistakenly determined to have shifted due to small everyday vibrations of the detection device 50 caused by vibrations transmitted by a large vehicle running nearby, for example. The predetermined distance may be set to about 3 pixels, for example. In the example of FIG. 8, the recognition points that have changed by a predetermined distance or more are marked with arrows. For example, recognition points P1 and P2 are recognition points that have changed by a predetermined distance or more. Also, for example, recognition point P3 is a recognition point that has not changed by a predetermined distance or more. Here, in this embodiment, the predetermined condition is that the number of recognition points that have changed exceeds half of the multiple recognition points. If the number of recognition points that have changed is the majority of the total, it is considered that the detection device 50 has shifted, rather than that some of the markers M have moved. The observation device 5 according to this embodiment can determine with sufficient accuracy that a deviation requiring calibration has occurred in the detection device 50. Here, the threshold value of the ratio of the number of recognition points that have changed to the total number used in the specified condition is not limited to 50%. For example, this threshold value may be set to a value less than 50% of the total (e.g., 30%) so that calibration is performed more frequently. From a similar perspective, the specified condition may be that the number of recognition points that have changed exceeds a specified value (e.g., 5). Here, the amount of change in the recognition points due to the deviation of the detection device 50 is usually greater for recognition points with markers M that are farther away. Therefore, rather than setting the specified distance to the same value for all recognition points, the specified distance may be set to be greater for recognition points with markers M that are farther away.

When the control unit 53 determines that the detection device 50 has shifted, it calculates the amount of change in the position or orientation of the detection device 50 based on the comparison results of the position of the recognition point and the inclination of the ridgeline in the captured image It0 and the captured image It1. If the recognition point has changed in the X direction in the captured image, it can be determined that the detection device has rotated in the pan direction. If the recognition point has changed in the Y direction in the captured image, it can be determined that the detection device has rotated in the tilt direction. If the angle of the ridgeline has changed in the captured image, it can be determined that the detection device has rotated in the roll direction. The control unit 53 may calculate the amount of change in the detection device 50 using a known method used in coordinate rotation processing.

The control unit 53 updates the correspondence between the position of the subject in the captured image and the distance to the subject based on the calculated amount of change. Specifically, the control unit 53 updates the coordinate-distance correspondence data stored in the memory unit 52 as a calibration. This allows the observation device 5 to continue to perform accurate distance measurement. Even if it is determined that a deviation requiring calibration has occurred in the detection device 50, calibration may not be performed in certain cases, and a notification may be sent to the administrator who manages the observation device 5. The certain cases include, for example, when the calculated amount of change is very large, when there is an inconsistency in the direction of change in the position of the recognition point, and when there is an inconsistency in the direction of change in the ridge line. An example of an inconsistency in the changes in the recognition points and ridge lines is when there are a certain number or more of recognition points and ridge lines that are changing in opposite directions. Or, an example of an inconsistency in the changes in the recognition points and ridge lines is when many recognition points have disappeared. In such cases, the observation device 5 may have broken down or a serious problem may have occurred, so the administrator needs to check the status.

The control unit 53 also updates the recognition point data 521 and edge line data 522 stored in the memory unit 52 using the positions of the recognition points and the slopes of the edge lines extracted in the captured image It1.

(Method of Calibrating Observation Device)
9 is a flowchart showing an example of a calibration method for the observation device 5 according to one embodiment. The observation device 5 executes a series of processes for calibration in accordance with the flowchart shown in FIG.

The observation device 5 acquires a captured image (step S1). The captured image in step S1 corresponds to, for example, the above-mentioned captured image It0. The observation device 5 sets multiple recognition points and ridgelines in the captured image It0.

The observation device 5 stores the position of the recognition point (step S2). The observation device 5 stores the position of the set recognition point in the memory unit 52, for example, by including it in the above-mentioned recognition point data 521. The processing of steps S1 and S2 is also performed, for example, when the observation device 5 is first installed.

The observation device 5 acquires a captured image (step S3). The captured image in step S3 corresponds to, for example, the above-mentioned captured image It1. The observation device 5 identifies the positions of multiple recognition points and the inclination of ridge lines in the captured image It1.

The observation device 5 determines whether the number of recognition points that have changed by a predetermined distance or more from the position stored in step S2 among the multiple recognition points identified in the captured image It0 satisfies the above-mentioned predetermined condition (step S4).

If the number of recognition points that have changed does not satisfy the predetermined condition (No in step S4), the observation device 5 determines that there is no deviation in the detection device 50 and returns to the processing of step S3.

If the number of recognition points that have changed satisfies a predetermined condition (Yes in step S4), the observation device 5 determines that a deviation has occurred in the detection device 50 and proceeds to processing in step S5.

The observation device 5 determines that a shift has occurred in the position or orientation of the detection device 50 (step S5). The observation device 5 also calculates the amount of change in the position or orientation of the detection device 50 based on the comparison results of the positions of the recognition points and the inclination of the ridge lines between the captured images It0 and It1.

The observation device 5 performs calibration based on the calculated amount of change (step S6). Then, the observation device 5 returns to the processing of step S2. In the processing of step S2 after performing the calibration, the observation device 5 updates the recognition point data 521 using the position of the recognition point extracted in the captured image It1, and stores it in the memory unit 52.

Here, the observation device 5 may periodically (for example, once a day) execute the processing from step S3 onward in the flowchart of FIG. 9.

As described above, the observation device 5 and the calibration method for the observation device 5 according to this embodiment can detect the occurrence of a deviation that requires calibration by using the above-described configuration and process, without using a measurement vehicle or the like.

Although the present disclosure has been described based on various drawings and examples, it should be noted that a person skilled in the art would easily be able to make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included within the scope of the present disclosure.

For example, in the above embodiment, the control unit 53 included in the observation device 5 calculates the amount of change in the position or orientation of the detection device 50, and performs calibration based on the calculated amount of change. Here, as described above, the observation device 5 includes a communication unit 51. The observation device 5 may communicate with a server in the communication system 1 via the communication unit 51. The observation device 5 may transmit necessary data to the server, and the server may perform at least one of calculation of the amount of change in the detection device 50 and calibration. Here, the server in the communication system 1 is a computer that controls the entire communication system 1.

For example, in the above embodiment, the observation device 5 sets multiple recognition points and edge lines. Here, the observation device 5 may define only multiple recognition points without setting edge lines.

For example, in the above embodiment, the observation device 5 treats each of the multiple recognition points equally, but weighting may be performed depending on the position and type of the recognition point, etc.

REFERENCE SIGNS LIST 1 Communication system 3 Crosswalk 4 Traffic light 5 Observation device 6 Vehicle 7 Road 9 Pedestrian 50 Detection device 51 Communication unit 52 Memory unit 53 Control unit 56 Screw 57 Connection unit 59 Pillar 521 Recognition point data 522 Ridge line data

Claims (8)

A detection device that captures an image of the marker ;
a storage unit that stores positions of a plurality of recognition points in the photographed image, the recognition points being defined by contours of the markers included in the photographed image;
a control unit that determines that a position or orientation of the detection device has shifted when a recognition point that has changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition ,
The predetermined distance is a larger value as the distance of the marker forming the recognition point from the detection device increases.
Observation equipment. The observation device according to claim 1 , wherein the recognition point is an intersection point where an outline of the marker intersects with an edge of the captured image. the control unit determines that a position or orientation of the detection device has shifted when a number of the recognition points that have changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition.
3. The observation device according to claim 1 or 2. 4. The observation device according to claim 1, wherein when a deviation in a position or orientation of the detection device is detected, the control unit performs a calibration regarding a correspondence between a position in the captured image and a distance to a subject present at that position . The observation device according to claim 1 , wherein the predetermined condition is that the number of the recognition points at which the change has occurred exceeds half of the plurality of recognition points. The observation device according to claim 1 , wherein the control unit issues an alert when a predetermined number or more of the recognition points have changed in a direction opposite to a direction in which the positions of some of the recognition points have changed . A method for calibrating an observation device including a detection device , comprising:
a control unit that stores in a storage unit positions of a plurality of recognition points in the acquired photographed image, the recognition points being parts of the contours of the markers included in the photographed image;
the control unit determines that a position or orientation of the detection device has shifted when a recognition point that has changed by a predetermined distance or more from a stored position among the plurality of recognition points satisfies a predetermined condition,
A method for calibrating an observation device, wherein the specified distance is a larger value the farther the marker forming the recognition point is from the detection device . The control unit performs calibration on a correspondence between a position of the marker in the captured image and a distance to the marker based on the calculated amount of change in position or orientation of the detection device.
A method for calibrating an observation device according to claim 7 .

Images