JP4896115B2 - Automatic tracking imaging device from a moving body in the air - Google Patents

Description

  The present invention relates to an automatic tracking imaging device mounted on an aerial moving body. For example, when a disaster countermeasure or rescue operation is performed using an aerial moving body such as an aircraft or a helicopter, a target is captured by a camera mounted on the aerial moving body. This is related to an automatic tracking imaging device from an aerial moving body that is used when automatically shooting a specified target regardless of the position, posture or movement of the aerial moving body in the air. is there.

The conventional automatic tracking imaging device mounted on an aerial moving body is divided into a tracking system based on coordinate designation based on latitude / longitude information of a shooting location and a tracking system based on image analysis processing of camera-captured video. I know.
In the former tracking method by specifying coordinates, the latitude and longitude information of the shooting target is registered in the device in advance, and the position information in the air and the position information in the air collected by GPS or gyro every second This is a method in which the camera is directed to the latitude and longitude of the target based on the posture information.
The latter tracking method based on image analysis processing captures the shape of the target by capturing the target at the center of the captured image, and adjusts the camera orientation by extracting the shape from the subsequent image through image analysis. It is a method to do.
The former tracking method by specifying coordinates is mainly suitable for automatic shooting from a long distance, and the latter tracking method by image analysis processing is mainly suitable for shooting at a short distance. There is also a device that uses a coordinate designation method for designating and switches to an image analysis method as it approaches the target.

Japanese Patent Laying-Open No. 2006-121261 (FIG. 1 and description thereof)

Conventionally, in the coordinate specification tracking method, only latitude / longitude information is specified or only elevation information is specified. For this reason, there is a problem that an error may occur between the designated coordinates and the position of the target to be actually photographed, and it is difficult to perform correct tracking.
For example, it is difficult to estimate the height information for the designation of a shooting location such as the roof of a building.
In addition, since the position information of the aerial moving body is basically based on GPS information updated every second, the control interval in the camera direction is also every second, and the captured image is displayed with the target blurred. It has been done.

In the image analysis tracking method, it is possible to avoid the above-described problems in the coordinate specification tracking method, and by specifying the target object in the video, the specific position information is unclear. Even so, tracking photography was possible.
It was also possible to suppress blurring of the target in the video by setting the camera control interval by image analysis to a cycle of 1 second or less.
However, when it is difficult to capture the target due to image analysis processing performance problems, and when the aerial moving body is suddenly changed, there is a problem that the target is off the camera and cannot be tracked. there were.
Further, as described above, since there is a possibility that an error has occurred in the latitude and longitude information of the designated target, a situation may occur in which the target cannot be correctly captured by the camera depending on the position of the aerial moving body. In general, it is difficult to correct a target position based on latitude and longitude information, and a position correction function combined with another method, for example, an image analysis method is required.
Patent Document 1 proposes a method of correcting a target position in accordance with an operation for manually changing the camera direction in order to improve target position accuracy. In this method, a point on the camera optical axis that takes the shortest distance from the previous target position is specified as the next target position, but this is specified as position information with a certain degree of accuracy. If not, it is difficult to capture the correct target position.
In addition, in the operation of a camera equipped with a helicopter, the target position is not necessarily specified by latitude and longitude coordinates, and is often specified after a target is captured by the camera. In this case, tracking of the image analysis method is performed, but since the first coordinate position is not specified, the above correction method cannot be used.

  The present invention has been made in view of the above-described circumstances. By combining the tracking function of the coordinate designation method and the tracking function of the image analysis method and switching both methods, the smoothness on the viewer is achieved. It is intended to realize an improvement in the function of maintaining target tracking as much as possible when an aerial moving body suddenly changes its direction.

  An automatic tracking imaging apparatus from an aerial moving object according to the present invention is an automatic tracking imaging apparatus from an aerial moving object having a tracking function of a coordinate designation method and a tracking function of an image analysis method, and an image captured from a camera A video position display function for displaying the image on the viewer, and an image analysis function that operates when a tracking target is specified on the video displayed on the viewer. The tracking function of the specified method performs automatic tracking shooting, and when a tracking target is specified on the video displayed on the viewer, the tracking function of the image analysis method performs automatic tracking shooting, and the tracking function of the image analysis method. If automatic tracking is not possible due to image analysis by the image analysis function under automatic tracking shooting by switching to the tracking function of the coordinate designation method, automatic tracking shooting is performed. That.

  The present invention is an automatic tracking imaging device from an aerial moving body having a tracking function of a coordinate designation method and a tracking function of an image analysis method, and a video position display function for displaying a video captured from a camera on a viewer; It has an image analysis function that operates when a tracking target is specified on the image displayed on the viewer. When the position coordinates of the target are specified, automatic tracking shooting is performed by the tracking function of the coordinate specification method. When a tracking object is specified on the video displayed on the viewer, automatic tracking shooting is performed by the tracking function of the image analysis method, and under the automatic tracking shooting by the tracking function of the image analysis method, the image analysis function If automatic tracking becomes impossible due to image analysis, the tracking function of the coordinate specification method and the image analysis method By combining the tail function and switching between the two methods, it is possible to achieve smooth shooting on the viewer and to maintain the target tracking as much as possible when the mobile object suddenly changes direction. There is an effect that can be improved.

Embodiment 1 FIG.
A first embodiment of the present invention will be described below with reference to FIGS. FIG. 1 is a diagram schematically illustrating an example of the relationship between an automatic tracking device and its peripheral devices, FIG. 2 is a diagram schematically illustrating the internal functions of the automatic tracking device and the correlation between the functions, and FIG. FIG. 4 is a diagram illustrating a state coordinate diagram, FIG. 4 is a diagram illustrating a world coordinate system in which an aerial moving body moves, and FIG. FIG. 6 is a diagram illustrating the concept of the camera coordinate system, the pan angle and the tilt angle, and FIG. 7 is a diagram illustrating the image analysis function.

  In FIG. 1, a camera 1 mounted on an aerial moving body 100 such as a helicopter is equipped under the head of the aerial moving body if it is a helicopter. Hereinafter, the case where the aerial moving body 100 is a helicopter (hereinafter abbreviated as “heli”) will be described as an example.

The automatic tracking device 2 mounted on the helicopter 100, which is an aerial moving body, receives video information and camera shooting direction angle information (camera shooting direction and camera shooting angle information (specifically illustrated in FIG. 6) from the camera 1. 6 and the information on the tilt angle illustrated in FIG. 6), and is connected to the output end of each information of the camera 1 by the interface.
The automatic tracking device 2 has an interface that transmits a signal for controlling the shooting direction of the camera to the camera 1.
Further, the automatic tracking device 2 is connected to the GPS antenna 3 mounted on the helicopter 100 and the gyro 4 mounted on the helicopter 100, receives GPS information from the GPS antenna 3, and receives attitude information of the helicopter from the gyro 4 in the air. Receive.

  The video display device 5 mounted on the helicopter 100 is a viewer that displays video shot by the camera 1.

  The pointing device 6 is used for pointing a tracking target object on the video displayed on the viewer 5.

  The keyboard 7 mounted on the helicopter 100 is used for inputting the position coordinates of the target.

  As illustrated in FIG. 2, the automatic tracking device 2 has, as its main functions, a video capture function 11, a video position display function 12, a camera posture capture function 13, a host posture input function 14, and a coordinate input function. 15, a camera direction calculation function 16, a camera control function 17, a video position specifying function 18, and an image analysis function 19.

  The video capture function 11 sequentially transmits video transmitted from the camera 1 to the video position display function 12.

  The video position display function 12 sequentially displays the video transmitted from the camera 1 on the viewer 5.

  The posture information of the camera 1 is transmitted to the camera posture capturing function 13. Specifically, the posture information of the camera 1 indicates a pan angle α (see FIG. 6) and a tilt angle β (see FIG. 6).

The own apparatus attitude input function 14 takes in information from the GPS antenna 3 and the gyro 4 and calculates the attitude of the own apparatus. Specifically, the information from the GPS antenna 3 includes position information (latitude θ, longitude λ, altitude H), and information from the gyro 4 includes the pitch angle φ of the helicopter body 100 (see FIG. 5), the helicopter body 100. Roll angle ω (see FIG. 5) and helicopter body 100 yaw angle κ (see FIG. 5).

  When the target coordinates (latitude, longitude) are input from the keyboard 7, the coordinate designation method automatic tracking function is started (71 in FIG. 3).

  The coordinate input function 15 acquires the target coordinates (latitude and longitude) input from the keyboard 7 and passes them to the camera direction calculation function 16.

  The camera direction calculation function 16 calculates a new camera direction based on the inputs of the camera posture capture function 13 and the own device posture input function 14 in order to direct the direction of the camera 1 to the designated target coordinates. The camera control function 17 is automatically operated so as to face the direction.

  The camera control function 17 outputs a signal to the camera 1, and the camera 1 points the camera in the target direction derived by the camera direction calculation function 16. The camera direction calculation function 16 recalculates the camera direction as soon as input signals from the camera posture capture function 13 and the own device posture input function 14 arrive, and transmits control information to the camera control function 17 to It is possible to continue to the direction of the re-derived target.

  As the helicopter 100 approaches the target, it is possible to specify the target on the video. When the video of the target is designated on the video displayed on the viewer 5 using the pointing device 6, information on the coordinate position of the target on the viewer 5 is passed to the video position specifying function 18. The video position specifying function 18 passes the passed information to the image analysis function 19. This triggers the automatic image analysis method tracking function (72 in FIG. 3). Thereafter, the image analysis function 19 receives the video from the video capture function 11 and analyzes the video to grasp the place on the screen of the previously designated target, and the camera information calculation function 16 To pass. The camera direction calculation function 16 receives the location information and transmits a control signal to the camera control function 17 so that the camera 1 faces the target each time.

  In some cases, automatic tracking of a target cannot be suddenly continued by the image analysis system automatic tracking function. Specifically, if the target becomes too small to be recognized on the video, or if it becomes too large and it takes a long time to analyze it, or sudden air movement If it disappears from the video before analysis due to changes in body posture, etc.

  In such a case, the image analysis function 19 passes a signal indicating that the analysis cannot be continued to the camera direction calculation function 16. Upon receiving the signal indicating that the analysis cannot be continued, the camera direction calculation function 16 automatically switches to the coordinate designation method automatic tracking function (73 in FIG. 3).

  The details of the coordinate specification method automatic tracking function are shown.

  FIG. 4 shows an example of a world coordinate system in which a helicopter that is an aerial moving body moves. Also, FIG. 5 illustrates the definition of the platform coordinate system of the aerial moving body and the concept of its pitch, roll, and yaw. The concept of the camera coordinate system and the pan angle and tilt angle are illustrated in FIG.

  The latitude and longitude information of the current position of the helicopter 100, which is an aerial moving body, is represented by (θ0, λ0, h0). The target coordinates specified by the coordinate specification method are (θ1, λ1, h1). The unit of angle after this is radians. In order to obtain the direction vector (Δx, Δy, Δh) from the current position to the target, it is necessary to convert from a latitude / longitude coordinate system to a plane rectangular coordinate system. There is a method described in http://vldb.Gsi.go.jp/sokuchi/surveycalc/algorithm/bl2xy/bl2xy.htm, but the following method is also possible approximately. .

△ x = a (λ1-λ0) cos (θ1) (1 -a) where a is the radius of the earth.
△ y = a (θ1-θ0) (1-b)
△ z = h1-h0 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ (1-c)

であり、ワールド座標で示した前述の方向ベクトル(△x, △y, △h)のプラットフォーム座標(u,v,w)への変換は、逆回転をかければよいので、以下の式で可能である。

また、カメラのパン回転、チルト回転の回転行列は、

となる。カメラの基準方向を、ヘリ進行方向C0とすると、パン回転およびチルト回転によるカメラ方向ベクトルC1は、

であらわされる。このC1は（３）式の（u,v,w）に等しいため、これを用いて、方向ベクトル（u,v,w）にカメラを向けるためのパン角度、チルト角度は、

で基本的に計算可能である。
カメラ方向演算機能１６は、指定された目標物の位置座標と、自機姿勢入力機能１４から送られた現在位置情報およびジャイロ４から送られてきたヘリ機体姿勢情報を基にして、このα、βを算出し、カメラ制御機能１７に逐次送信することで制御を行う。 There are three types of rotation matrix for converting from world coordinates to platform coordinates: roll rotation, pitch rotation, and yaw rotation.

The above-mentioned direction vector (△ x, △ y, △ h) shown in world coordinates can be converted to platform coordinates (u, v, w) by reverse rotation. It is.

The rotation matrix of camera pan rotation and tilt rotation is

It becomes. If the reference direction of the camera is the helicopter traveling direction C0, the camera direction vector C1 by pan rotation and tilt rotation is

It is expressed. Since this C1 is equal to (u, v, w) in equation (3), the pan angle and tilt angle for directing the camera to the direction vector (u, v, w) are

Basically it can be calculated.
Based on the position coordinates of the designated target, the current position information sent from the own machine attitude input function 14 and the helicopter machine attitude information sent from the gyro 4, the camera direction calculation function 16 Control is performed by calculating β and sequentially transmitting it to the camera control function 17.

  Describes the image analysis method automatic tracking function.

  In order to realize the start function of the image analysis method and the stop function due to analysis failure, the image analysis function 19 is configured as the following two sub-functions. See FIG.

  In FIG. 7, the image analysis processing 21 receives the video position input information 27 from the video position specifying function 18 and starts image analysis. The image analysis process 21 has a function of identifying the position on the image of the specified object using the input video 23. Specific image analysis processing is not described in this specification.

  The position information of the object identified from the video 23 and the camera angle information 25 are transmitted to the camera direction calculation function 16 and used for camera control.

  The image analysis process 21 sets a timer value 24 for the timeout process 22 before starting the process by the input of the video 23.

  The time-out process 22 transmits a time-out message 26 to the image analysis process 21 when the set time has come.

  The image analysis processing 21 that has received the time-out message 26 transmits an analysis failure message 25 to the camera direction calculation function 16 instead of the camera angle information.

  Upon receiving the analysis failure message, the camera direction calculation function 16 stops the image analysis method automatic tracking function and switches to the coordinate designation method automatic tracking function.

  As described above, the first embodiment has a function of automatically switching between the coordinate designation method and the image analysis method, and realizes smooth shooting on the viewer and the target object when the aerial moving body suddenly changes its direction. Improvement of the function to maintain tracking as much as possible can be realized.

  As described above, the first embodiment is conceptually an automatic tracking imaging device from an aerial moving body having a tracking function of a coordinate designation method and a tracking function of an image analysis method, It has a video position display function that displays the captured video on the viewer, and an image analysis function that operates when a tracking target is specified on the video displayed on the viewer. In this case, automatic tracking shooting is performed by the tracking function of the coordinate specification method, and when a tracking target is specified on the video displayed on the viewer, automatic tracking shooting is performed by the tracking function of the image analysis method, and the image analysis is performed. If automatic tracking is not possible due to image analysis using the image analysis function under automatic tracking shooting using the tracking function of the method, automatic tracking shooting is performed by switching to the tracking function of the coordinate designation method. That.

  Further, from a different viewpoint, as described above, in the first embodiment, the own attitude calculation function unit for calculating the attitude of the own apparatus based on the information from the GPS and the gyro, and the video from the camera are captured for each frame. Image capture function unit that converts to an image, camera posture capture function unit that captures camera shooting direction (pan / tilt / zoom) information, camera control function that controls the camera shooting direction freely, and shooting tracking A coordinate specification function unit for specifying the position of the target object with coordinate values, a video position display function unit for displaying the image captured by the video capture function on the display, and a pointing device on the video displayed on the display Analyzing images captured by the video capture function based on the video location specification function for directly specifying the tracking target in detail and the tracking target specified by the video location specification function Image analysis function unit, camera posture information from camera posture capture function unit, own device posture information from own device posture calculation function unit, target object coordinate value from coordinate input function unit, and image analysis from image analysis function unit A camera direction calculation function unit for calculating the camera shooting direction from the result and passing it to the camera control function unit. When the target position coordinates are specified by the coordinate specification function unit, it is automatically If automatic tracking by coordinate specification is started, and if it is specified by a pointing device on the video, it automatically switches to automatic tracking by image analysis, and if automatic tracking by image analysis becomes impossible This is an automatic tracking imaging device having a function of automatically switching to automatic tracking by coordinate designation.

Embodiment 2. FIG.
The second embodiment of the present invention will be described below with reference to FIGS. FIG. 8 is a diagram showing another example of the configuration of the automatic tracking device system, FIG. 9 is a diagram illustrating the position of the aerial moving body and the position of the target object as a schematic diagram, and FIG. 10 is a specific coordinate recalculation function. It is a figure which illustrates processing contents with a flow chart. In the second embodiment, when automatic tracking is performed by image analysis, an object coordinate designation is performed by a coordinate position recalculation function that recalculates the coordinates of the object from the result of the image analysis, the own apparatus attitude information, and the camera attitude information. It has a function to improve accuracy.

  In the method of the first embodiment described above, the position coordinates of the target when the image analysis method is failed and the coordinate specification method is switched to are the values initially set by the coordinate input function 15. However, in practice, there are many cases where there is an error between the position of the real target and the specified position coordinates. Therefore, in the second embodiment, the automatic tracking device 2 is provided with a function for automatically correcting the target position coordinates based on the result of image analysis.

  That is, as illustrated in FIG. 8, a coordinate recalculation function 31 is added to the automatic tracking device system configuration diagram illustrated in FIG. 2 described above. The function of the coordinate recalculation function 31 is as follows.

FIG. 9 illustrates a schematic diagram of the position of the air moving body and the position of the target. The aerial moving body 100 is performing tracking shooting by the camera 2 assuming that the position of the target 42 is T0 when in the position S0, and the camera optical axis vector at that time is indicated by 43.
When the aerial moving body 100 moves from the position S0 to the position S1, a deviation between the target 42 and the camera optical axis vector 43 occurs. Therefore, the pointing device 6 corrects the position of the target 42 from T0 to T1. At this time, the camera optical axis vector is represented by 44. The coordinate recalculation function 31 includes a new target position T1, a line segment 45 indicating the shortest distance between the camera optical axis vector 43 at the position S0 and the camera optical axis vector 44 at the position S1, and the camera at the position S1. Let it be an intersection with the optical axis vector 44.

The world coordinates of S0 are (0, 0, 0), and the world coordinates of T0 are (Tx, Ty, Tz). Therefore, the vector of the camera optical axis vector 43 at the position S0 is represented by (dx, dy, dz) = (Tx, Ty, Tz).
Further, the world coordinates S1 after the airborne moving body 100 has moved to the position S1 is (Sx, Sy, Sz).
Further, the vector in the world coordinates of the camera optical axis vector 44 at the position S1 after the movement is C = (Cx, Cy, Cz).

The straight line equation indicating the camera optical axis vector 43 at the position S0 is

The linear equation showing the camera optical axis vector 44 at position S1 is

であらわされる。線分４５と位置S0でのカメラ光軸ベクトル４３が直交することから、

同じく、線分４５と位置S1でのカメラ光軸ベクトル４４が直交することから、

Therefore, the line segment 45 is

It is expressed. Since the line 45 and the camera optical axis vector 43 at the position S0 are orthogonal,

Similarly, since the line segment 45 and the camera optical axis vector 44 at the position S1 are orthogonal to each other,

When these two equations are used as simultaneous equations, the coordinates of the target position T1 can be obtained from the value of q.
In particular,

  Specific processing contents of the coordinate recalculation function 31 are illustrated in a flowchart in FIG.

  In step ST51, the coordinate recalculation function 31 acquires the position information and attitude information of the aerial moving body from the own apparatus attitude input function 14 via the camera direction calculation function 16.

  Next, in step ST <b> 52, camera posture information is acquired from the camera posture capturing function 13 via the camera direction calculation function 16.

Next, in step ST53, the value of the current position coordinate is calculated from the previous value.
The latitude, longitude, and altitude (θ0, λ0, h0) of the position S0 at the time of the previous calculation have been acquired, and that position is the origin in the world coordinate system. Knowing the latitude, longitude, and altitude (θ1, λ1, h1) of the current position, the world coordinates (Sx, Sy, Sz) of the position S1 are (1-a), (1-b), (1-c) From the equation
Sx = △ x = a (λ1-λ0) cos (θ1) where a is the radius of the earth.
Sy = △ y = a (θ1-θ0)
Sz = △ z = h1-h0
It can be calculated with

  Next, in step ST54, a camera optical axis vector C = (Cx, Cy, Cz) is obtained from the current camera angle.

であらわされる。 When the reference direction of the camera is the helicopter traveling direction C0, the vector C0 in the reference direction is represented by (1, 0, 0). If the camera pan angle and tilt angle when moved to position S1 are α and β, respectively, the rotation matrix for calculating the camera direction is

It is expressed.

となる。これをワールド座標に変換して、

となる。 As a result, the desired optical axis vector is

It becomes. Convert this to world coordinates,

It becomes.

  Next, in step ST55, q is calculated. From the result of step ST53 and the result of step ST54, calculation is performed using the above equation (11).

  Next, in step ST56, the coordinates (x, y, z) of the position T1 of the target are obtained from the value of q and the equation (9).

Next, in step ST57, the direction of the target position T1 is clarified. The shortest distance between the straight line (camera optical axis vector (when position is assumed at position S0)) 43 and the straight line (camera optical axis vector (after position correction at position S1)) 44 is theoretically the reverse direction of the camera. Because there is a possibility, it is necessary to exclude it. Here, if the value of q is positive, it is determined that the direction is the same as the direction of the camera, and if the value is negative, it can be determined that the direction is reverse. If it is in the opposite direction, the coordinates of the target position T1 are discarded and recalculated from the new position. If so, the process proceeds to step ST58.
In step ST58, the value of the target position T1 is passed to the camera direction calculation function 16 using T1 as the coordinates of the new target.

  In step ST59, the values of the position S1 of the aerial moving body 100 and the position T1 of the target are held for the next calculation.

  In step ST60, it is confirmed whether or not the processing of the image analysis method is stopped. If it is stopped, this function is also stopped. If not stopped, return to the beginning and perform coordinate recalculation.

  By realizing this function, it is possible to correct the target position coordinates in combination with the image analysis method. Since position correction is performed as a result of automatic tracking by image analysis, tracking with higher accuracy is possible without making the user aware of it.

Embodiment 3 FIG.
The third embodiment of the present invention will be described below. In the third embodiment, even when the position coordinates of the target are not known in advance, it is possible to perform tracking shooting by image analysis of the target by specifying the target displayed on the viewer with a pointing device In addition, during the automatic tracking process by image analysis, it has a function of recalculating the coordinates of the object from the result of the image analysis, the own attitude information, and the camera information.

It is assumed that the object (target object) on the image displayed on the viewer 5 is designated by the viewer 5 and the pointing device 6 from the initial position S0 without designating the position coordinates in advance.
Tracking by the screen analysis method is started (74 in FIG. 3).
At this time, the position coordinates are undecided.

  When the aerial moving body 100 moves to another position S1, when the object (target object) on the image displayed on the viewer 5 is designated again by the viewer 5 and the pointing device 6, the position coordinates of the target object 42 are changed. It is possible to calculate by the following method.

としても良い。
これにより、(11)式は、

となる。
図１０のフローチャートのステップＳＴ５５において、(11)式の変わりに(13)式を用いることで目標物位置T1の座標の指定が可能となる。 In equation (7), since the value of (tx, ty, tz) can be regarded as the unit vector C ′ = (C′x, C′y, C′z) of the camera optical axis at the position S0,

It is also good.
As a result, equation (11) becomes

It becomes.
In step ST55 of the flowchart of FIG. 10, the coordinates of the target position T1 can be specified by using the expression (13) instead of the expression (11).

  Thereafter, the correction can be performed sequentially by executing the process according to the process of FIG.

Embodiment 4 FIG.
A fourth embodiment of the present invention will be described below with reference to FIG. FIG. 11 is a flowchart illustrating another specific processing content of the coordinate recalculation function.

  The fourth embodiment has a function of automatically recognizing the operation when the user consciously manually controls the camera during automatic tracking, and temporarily stopping the automatic tracking process. The camera direction calculation function 16 carries out.

  In FIG. 11, steps ST51 to ST59 are the same as steps ST51 to ST59 in FIG.

  In step ST61, camera angle information is acquired again.

  In step ST62, it is confirmed whether the camera angle information acquired again in step ST61 is the same as the direction specified in step ST58. As a result of the confirmation in step ST62, if they are not the same, it is determined that the camera is manually operated, and the automatic tracking process is stopped. If the confirmation results in step ST62 are the same, stop is determined in step ST60.

  In addition, in each figure, the same code | symbol shows the same or an equivalent part.

It is a figure which shows Embodiment 1 of this invention, and is a figure which shows roughly the example of the relationship between an automatic tracking apparatus and its peripheral device. BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows Embodiment 1 of this invention, and is an example which illustrates typically the internal function of an automatic tracking apparatus, and the correlation of each function. It is a figure which shows Embodiment 1 of this invention, and is a figure which illustrates typically the whole state transition. It is a figure which shows Embodiment 1 of this invention, and is a figure which illustrates the world coordinate system in which an air mobile body moves around. It is a figure which shows Embodiment 1 of this invention, and is a figure which illustrates the definition of the platform coordinate system of an aerial mobile body, and the concept of the pitch, roll, and yaw. It is a figure which shows Embodiment 1 of this invention, and is a figure which illustrates the view of a camera coordinate system, a pan angle, and a tilt angle. It is a figure which shows Embodiment 1 of this invention and is a figure which illustrates an image analysis function. It is a figure which shows Embodiment 2 of this invention, and is a figure which shows the other example of a structure of an automatic tracking device system. It is a figure which shows Embodiment 2 of this invention, and is a figure which illustrates the position of an air moving body and the position of a target object as a schematic diagram. It is a figure which shows Embodiment 3 of this invention, and is a figure which illustrates the specific processing content of a coordinate recalculation function with a flowchart. It is a figure which shows Embodiment 4 of this invention, and is a figure which illustrates the other specific process content of a coordinate recalculation function with a flowchart.

Explanation of symbols

1 camera,
2 automatic tracking device,
3 GPS antenna,
4 Gyro,
5 Viewer (video display device),
6 pointing devices,
7 Keyboard,
11 Video capture function,
12 Image position display function,
13 Camera posture capture function,
14 Own attitude input function,
15 Coordinate input function,
16 Camera direction calculation function,
17 Camera control function,
18 Video location specification function,
19 Image analysis function,
21 Image analysis processing,
22 Timeout processing,
23 video,
24 timer value,
25 Camera angle information,
26 time out message,
27 Video position input information,
31 Coordinate recalculation function,
42 Target,
43 Camera optical axis vector (when position is assumed at position S0),
44 camera optical axis vector (after position correction at position S1), 45 line segments,
100 Airborne moving object.

Claims (5)

An automatic tracking imaging device from an aerial moving object having a tracking function of a coordinate designation method and a tracking function of an image analysis method,
A video position display function for displaying the video captured from the camera on the viewer, and an image analysis function that operates when a tracking target is designated on the video displayed on the viewer;
When the position coordinates of the target are specified, automatic tracking shooting is performed by the tracking function of the coordinate specification method,
When a tracking target is specified on the video displayed on the viewer, automatic tracking shooting is performed by the tracking function of the image analysis method,
When automatic tracking is impossible by image analysis by the image analysis function under automatic tracking shooting by the tracking function of the image analysis method, the automatic tracking shooting is performed by switching to the tracking function of the coordinate designation method. Auto-tracking imaging device from a moving body. Its own attitude calculation function that calculates the attitude of its own aircraft based on information from the GPS and gyro,
A video capture function to capture video from the camera,
A camera posture capture function that captures information about the shooting direction of the camera;
A camera control function for controlling the shooting direction of the camera;
A coordinate designating function for designating the position of the shooting tracking target object by a coordinate value;
A video position display function that displays images captured with the video capture function on the viewer;
A video position specifying function for designating a tracking object on the video displayed on the viewer;
Based on the tracking target specified by the video position specifying function, an image analysis function for analyzing the image captured by the video capture function,
From the camera posture information from the camera posture capture function, the own device posture information from the own device posture calculation function, the object coordinate value from the coordinate input function, and the image analysis result from the image analysis function, A camera direction calculation function that calculates the camera shooting direction and passes it to the camera control function;
An automatic tracking imaging device from an aerial moving object having
When the position coordinates of the target are designated by the coordinate designation function, automatic tracking shooting is automatically performed by coordinate designation,
When specified on the video, it automatically switches to automatic tracking shooting by image analysis, and when automatic tracking shooting by image analysis becomes impossible, it automatically switches to automatic tracking shooting by coordinate specification. An automatic tracking imaging device from a moving object in the air.   The automatic tracking imaging device from the aerial moving body according to claim 1 or 2, wherein the coordinates of the object are obtained from the result of the image analysis, the own apparatus attitude information, and the camera attitude information under the automatic tracking imaging by the image analysis. An automatic tracking imaging device from an aerial moving object characterized by improving accuracy of object coordinate designation by a recalculation coordinate position recalculation function.   In the automatic tracking imaging device from the aerial moving body according to claim 3, even when the position coordinate of the target is not known in advance by designating the object displayed on the viewer with a pointing device, An automatic tracking imaging device from an aerial moving body, characterized by enabling tracking imaging by image analysis of an object.   In the automatic tracking imaging | photography apparatus from the aerial moving body as described in any one of Claims 1-4, when a camera is manually controlled under automatic tracking imaging | photography, the operation | movement is recognized automatically, and an automatic tracking process An automatic tracking photographing device from an aerial moving body characterized by temporarily stopping the movement.

Images