JP6892134B2 - Measurement system, measurement method and measurement program - Google Patents

Description

The present invention relates to a measurement system, a measurement method, and a measurement program for performing various measurements such as a tsunami based on images taken by a plurality of cameras.

Existing tsunami measurement (observation) methods can be roughly classified into two types: tsunami observation methods that have reached the land and tsunami observation methods that have not reached the land.

(1) Observation method of tsunami that reached the land For observing the height of the tsunami that reached the land, in addition to the tide stations set up by the Meteorological Agency, tsunami observations such as tsunami observers and giant tsunami observers Equipment is used. The Meteorological Agency constantly observes the tide level using a tide station and a tsunami observer, and publishes the observed data as tide observation data (preliminary figures) and tide observation data. However, since all of these observation systems are installed on the coast, although it is possible to observe the tsunami that has reached the land, it is not possible to observe the tsunami in the distance (several tens of kilometers to over 50 km) before it arrives, and the arrival time is estimated. I can't.

(2) Observation method of tsunami before arrival at land For observation of tsunami before arrival at land, a submarine earthquake / tsunami observation network, GPS wave meter, and recently proposed tsunami radar are used.

The Submarine Earthquake and Tsunami Observation Network is a large-scale real-time earthquake and tsunami observation network promoted and maintained by the National Research Institute for Earth Science and Disaster Prevention as a Japan Submarine Earthquake and Tsunami Observation Network Development Project. In the underground earthquake and tsunami observation network, an observation device that integrates a seismograph and a tsunami meter is connected with an optical seafloor cable and installed on the seafloor off the Pacific Ocean in eastern Japan to acquire observation data in real time for 24 hours continuously. To do. As of January 2018, observation devices have been installed at 150 locations along the coast of Japan, and the total cable length is approximately 5,700 km. The underground earthquake and tsunami observation network is expected to directly detect subduction-zone earthquakes and tsunamis immediately after, and contribute to disaster prevention measures such as damage reduction and evacuation behavior by transmitting information quickly and accurately.
However, the installation of submarine systems and land stations requires enormous construction work, and the consent of local governments and fishery officials is required. In addition, a large amount of money is required for the maintenance and management of equipment. Therefore, it is difficult to spread it in the sea areas of the whole country.

The GPS wave meter system is a marine observation device that uses GPS satellites to measure vertical fluctuations of a buoy (GPS wave meter) floating offshore and directly observes sea level fluctuations such as waves and tides. As of January 2017, 18 GPS wave meters have been installed off the coast of Japan.
The GPS wave meter is installed to obtain the offshore wave information required for port facilities, but it can also be used for tsunami disaster prevention measures because it is possible to observe the vertical movement of the sea surface due to the tsunami when an earthquake occurs. Is also highly expected. Normally, GPS wave meters are installed about 20 km offshore, so it is possible to observe tsunamis several tens of kilometers away and estimate the arrival time.
However, GPS wave meters are expensive equipment, and when they were first installed in 2005, they exceeded 300 million yen per unit and are still in the hundreds of millions. In addition, GPS wave meters can be installed only at specific spots, making wide-area observation difficult.

Tsunami radar is a system that uses radar measurement technology to measure distant sea levels and monitor the occurrence of tsunamis. In these methods, the tsunami component is extracted from the sea surface velocity observed by the radar, and the wave height and arrival time of the tsunami are estimated.
However, with these methods, it is difficult to extract the tsunami component from the flow velocity on the sea surface, and it is particularly difficult to measure the tsunami, which has a long period, at an early stage.

Japanese Unexamined Patent Publication No. 2013-40898 Japanese Unexamined Patent Publication No. 2018-4259 Japanese Unexamined Patent Publication No. 2014-160044 Japanese Unexamined Patent Publication No. 2004-191268 JP-A-2017-150917 Japanese Unexamined Patent Publication No. 2016-85206

By the way, image measurement technology has been studied for 30 years, and has been put into practical use in recent years due to advances in hardware technology such as digital camera technology and computer manufacturing technology, and development of software technology such as algorithms and artificial intelligence. Currently, image measurement technology has been put to practical use in quality control fields such as print quality control, product quality control, and appraisal of jewelry and art objects. It is also widely used in the fields of face recognition and object recognition.

The present invention is a measurement system, measurement method, and measurement in which the same shooting target area is simultaneously photographed by a plurality of cameras installed at different points, and various measurements such as a tsunami are performed based on the images captured by the plurality of cameras. The purpose is to provide a program.

The measurement system of the present invention is a measurement system using a photographing means for simultaneously capturing an image including the same shooting target area by a plurality of cameras installed at different points, and includes machine control based on angle feedback and image shooting. It is a two-stage control means that combines machine control with image measurement control based on image processing and image measurement control, and the machine control is a shooting target in the shooting target area by machine control based on angle feedback using an angle sensor. The line of sight of each camera is roughly adjusted so that a part of the image can be taken by each camera, and the image measurement control takes a picture with each camera, feeds back the image taken from each camera, and changes the line of sight of each camera. It is a measurement system that includes a two-stage control means that performs precise control.

Further, in the measurement method of the present invention, an image including the same shooting target area is simultaneously taken by a plurality of cameras installed at different points, and mechanical control based on angle feedback using an angle sensor is performed to obtain a shooting target area. Approximately adjust the line of sight of each camera so that a part of a certain shooting target can be taken by each camera, take a picture with each camera, feed back the image taken from each camera, and precisely adjust the line of sight of each camera. It is characterized by performing control.

According to these inventions, images including the same shooting target area are simultaneously taken by a plurality of cameras installed at different points, and first, they are in the shooting target area by mechanical control based on angle feedback using an angle sensor. The line of sight of each camera is roughly adjusted so that a part of the shooting target can be taken by each camera, and the lack of machine control, that is, the error of machine control, is supplemented by image measurement control based on image shooting and image processing. To.

Further, the measurement program of the present invention is a measurement program using a photographing means for simultaneously capturing an image including the same shooting target area by a plurality of cameras installed at different points, and is a machine control based on angle feedback and an image. It is a two-stage control means that combines machine control with image measurement control based on shooting and image processing and image measurement control, and the machine control is in the shooting target area by machine control based on angle feedback using an angle sensor. The line of sight of each camera is roughly adjusted so that a part of the shooting target can be taken by each camera, and the image measurement control takes a picture with each camera, feeds back the image taken from each camera, and of each camera. The computer functions as a two-stage control means for precise control of the line of sight. The computer that executes the measurement program of the present invention functions in the same manner as the above-mentioned measuring device of the present invention.

According to the present invention, a two-stage control that combines mechanical control based on angle feedback, image taking, and image measurement control based on image processing and image measurement control is used to control a part lacking mechanical control, that is, mechanical control. The error is supplemented by image measurement control based on image shooting and image processing, and the part that image measurement control does not reach is realized by machine control, so multiple cameras installed at different points can simultaneously capture the same shooting target area. It is possible to track the same shooting target with high accuracy when shooting an image including the image and performing various measurements of the shooting target such as a tsunami.

It is a hardware block diagram of the sea level measurement system in embodiment of this invention. It is explanatory drawing which shows the installation state of the camera set of the sea level measurement system of FIG. It is explanatory drawing which shows the coordinate relation of 3D image measurement by stereo view. It is a block diagram which shows the structure of the sea level measurement system of FIG. It is a flow chart which shows the flow of the sea level measurement by the sea level measurement system of FIG. It is an image diagram of an atmospheric model. It is a flow chart of processing by the rain influence mitigation means. It is an image diagram of wave extraction. It is a flow chart which shows the flow of the wave correspondence processing using two images. It is an image diagram of a wave feature vector. It is an image diagram of the height calculation of the sea surface using the images taken by two cameras on the left and right. It is a flow chart which shows the flow of the sea level height calculation using the image taken by two cameras on the left and right. It is a figure which shows an example of a columnar target. It is a flow chart which shows the flow of calibration by the long-distance camera system calibration means. It is a flow chart which shows the flow of determination of presence or absence of a tsunami. It is an image diagram of the three-dimensional image measurement of the sea level at each time. It is an image diagram of the machine control of the line of sight of the camera. It is an image diagram of precision control by image measurement control of two cameras. It is a flow chart which shows the flow of precision control by image measurement control when having two cameras.

Hereinafter, the sea level measurement system according to the embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a hardware configuration diagram of the sea level measurement system according to the embodiment of the present invention, FIG. 2 is an explanatory diagram showing an installation state of a camera set of the sea level measurement system of FIG. 1, and FIG. An explanatory diagram showing the coordinate relationship, FIG. 4 is a block diagram showing the configuration of the sea level measurement system of FIG.

As shown in FIG. 1, the sea level measurement system as the measurement system in the present embodiment is connected to the server 1 as a computer, the client computers 2A and 2B connected to the server 1, and the client computers 2A and 2B, respectively. It is composed of a first camera set 3A and a second camera set 3B. The first camera set 3A and the second camera set 3B are the cameras 4A and 4B including the camera body and the lens, and the camera fixing adjustment units 5A and 5B such as a swivel on which the cameras 4A and 4B are fixed so that the posture can be adjusted. It is composed of and.

As shown in FIG. 2, the first camera set 3A and the second camera set 3B are installed at positions of baseline length B and installation altitude H at different points C1 and C2 on the land and ground near the sea. Strictly speaking, as shown in FIG. 3, the distance between the center of each camera lens of the first camera set 3A and the second camera set 3B installed at the points C1 and C2 is defined as the baseline length B. Further, in the following description, the extending direction of the baseline length B is the X direction, the height direction is the Y direction, and the directions orthogonal to the X direction and the Y direction are the Z directions.

As shown in FIG. 4, the server 1 includes a photographing means 10 for simultaneously capturing an image including the same sea surface area which is the same shooting target area by a plurality of cameras 4A and 4B installed at different points, and a plurality of cameras 4A. The wave extraction means 11 that extracts waves from the images simultaneously taken by the waves 4A and 4B and the same images from the images simultaneously taken by the plurality of cameras 4A and 4B for each wave extracted by the wave extraction means 11. A wave mapping means 12 that finds and associates waves, and a three-dimensional information calculation means that calculates the height of the sea surface and the height of waves by three-dimensionally analyzing each wave of the image associated by the wave mapping means 12. 13 and 14 for determining the presence or absence of sea surface abnormality such as a tsunami based on the height of the sea surface and the height of the wave calculated by the three-dimensional information calculation means 13.

Further, the sea level measurement system in the present embodiment has a long-distance camera system calibration means 15 for improving the accuracy of three-dimensional information calculation of the sea surface, and a tsunami traveling speed when the occurrence of a tsunami is measured as a sea level abnormality. A tsunami traveling speed calculating means 16 for calculating, a tsunami scale estimating means 17 for estimating the scale of the tsunami, a tsunami arrival time estimating means 18 for estimating the arrival time of the tsunami, a tsunami information transmitting means 19, and a storage means 20. Including. Further, as will be described in detail later, the sea level measurement system in the present embodiment includes the fog effect reducing means 21, the rain effect reducing means 22, and the two-stage control means 23. The server 1 functions as each of the above means 10 to 23 by executing the sea level measurement program.

FIG. 5 is a flow chart showing a flow of tsunami measurement by the sea level measurement system of FIG.
In order to realize image measurement of a tsunami as a sea level abnormality, first, two or more cameras 4A and 4B are installed at different points (S101).
Subsequently, a plurality of cameras 4A and 4B simultaneously capture an image including the same sea surface region (S102). Further, an infrared camera, fog influence reducing means 21, and rain influence reducing means 22 are used for shooting in a bad environment such as nighttime, rain, or fog, and the details will be described later (S201).
After that, the waves required for tsunami measurement are extracted from the images taken by the cameras 4A and 4B by using the wave extraction means 11, respectively (S103).
After extracting the waves for each image, the waves are associated with each other using the wave associating means 12, and the waves extracted from the images of the cameras 4A and 4B are associated with each other to obtain the same wave group (the same wave group is obtained). S104).

The three-dimensional information of the sea surface is calculated by using the three-dimensional information calculation means 13 of the sea surface for each group of the associated waves. First, the three-dimensional world coordinates (X, Y, Z) of each detected wave are calculated, and based on this, the three-dimensional information of the sea surface, that is, the height of the sea surface and the height of the wave are calculated (S105).
In addition, in order to ensure the accuracy of the three-dimensional information calculation, the camera system is calibrated by using the long-distance camera system calibration means 15 (S202).
Based on the calculated sea level height and wave height, the presence / absence of a tsunami is determined by using the means for determining the presence / absence of a tsunami (S106, S107).
When it is determined that a tsunami has occurred, the tsunami scale is estimated (S108) and the arrival time is estimated by using the tsunami traveling speed calculation means 16, the tsunami scale estimation means 17, and the tsunami arrival time estimation means 18 (S108). S109).
Finally, the measured tsunami information is transmitted to the necessary place and the necessary person by the tsunami information transmitting means 19 (S110).

Next, each means will be described in detail.

[Shooting means 10]
The photographing means 10 simultaneously captures an image including the same sea surface region by a plurality of cameras 4A and 4B installed at different points. As described above, the first camera set 3A including the camera 4A and the second camera set 3B including the camera 4B are installed at different points C1 and C2 on the land and ground near the sea as shown in FIG. An image of a long-distance sea surface in a radius of about 5 to 20 km is acquired. In order to realize the three-dimensional image measurement of the distant sea surface with a radius of 5 to 20 km, in the present embodiment, the three-dimensional image measurement technique by stereo vision is used. The distance between the cameras 4A and 4B, that is, the baseline length B between the cameras 4A and 4B is 20 meters or more, and the sea level of the installation location of the cameras 4A and 4B, that is, the installation altitude H of the cameras 4A and 4B is 20 meters or more.

In order to minimize the damage caused by the tsunami, etc. to the 1st camera set 3A and the 2nd camera set 3B, to minimize the disturbance during shooting, and to realize the photography as far away as possible, the 1st camera set 3A And the second camera set 3B should be installed on the beach, the closer it is to the sea, the better. In this embodiment, two sets of the first camera set 3A and the second camera set 3B are used, but it is possible to use three or more sets.

It is desirable to use high-sensitivity cameras with a sensitivity of 8 bits or more for the cameras 4A and 4B so that the measurement accuracy can be improved and the images required for measurement can be taken even in a bad environment. Further, it is desirable to use a lens having a telephoto function with a large aperture or a zoom lens having a telephoto function for photographing a long-distance sea surface.

The client computers 2A and 2B have a function of adjusting the parameters of the cameras 4A and 4B to control shooting and a function of capturing images taken by the cameras 4A and 4B. The server 1 has a communication function with the client computers 2A and 2B, controls the client computers 2A and 2B, sends commands necessary for taking a picture to the client computers 2A and 2B, and sends a photographed image from the client computers 2A and 2B. It has a function to acquire.

In the sea level measurement system of the present embodiment, it is necessary to adjust the line of sight of each of the cameras 4A and 4B in order to capture different sea levels. Therefore, the camera fixing adjustment units 5A and 5B can rotate in the pan direction (horizontal direction) and the tilt direction (vertical direction). By adjusting the pan direction and tilt direction angles of the camera fixed adjustment units 5A and 5B, the shooting direction angles of the cameras 4A and 4B installed in the camera fixed adjustment units 5A and 5B can be adjusted up and down and left and right, and a plurality of cameras 4A , 4B can be made to photograph the same sea surface.

The rotation angles in the pan direction and the tilt direction are controlled by the instructions of the client computers 2A and 2B. The specific values of the rotation angles in the pan direction and the tilt direction differ depending on the position of the measurement sea area, and are transmitted from the server 1 to the client computers 2A and 2B that control the camera sets 3A and 3B. Further, synchronous shooting of the cameras 4A and 4B is realized by simultaneously shooting a plurality of cameras 4A and 4B by an electric external synchronization signal or a software synchronization command from the server 1. The client computers 2A and 2B control parameters such as the exposure time of the camera body of the cameras 4A and 4B, and control such as lens zoom control and focusing. The images captured by the cameras 4A and 4B are first captured by the client computers 2A and 2B connected to the cameras 4A and 4B, then sent to the server 1 by the command of the server 1 to perform image processing on the server 1.

It is necessary to photograph the sea for 24 hours to measure sea level abnormalities such as tsunami. Therefore, in the sea level measurement system of the present embodiment, infrared cameras such as a far-infrared camera are mainly used as the cameras 4A and 4B in the daytime and mainly in the nighttime.

[Fog effect mitigation means 21]
Further, the server 1 includes a fog influence reducing means 21 for reducing the influence of fog and the like. The fog influence mitigation means 21 is based on an atmospheric model, and has a transmittance distribution of the atmosphere on the sea surface and sunlight linearly approximated to the vertical length of the image of the sea taken by the cameras 4A and 4B, that is, the y-coordinate of the image. From the intensity component of the above, adverse effects such as fog of images taken by the cameras 4A and 4B are reduced.

ただし、Ｉはカメラに入った光量すなわち霧などの環境影響を受けた画像であり、Ａは大気光の強度成分、Ｊは対象物の反射成分、ｔは対象物からカメラまでの間の大気の透過率である。Ｊは霧などの環境影響がない場合の画像であり、霧影響軽減手段２１により霧などの除去処理の際に求めるべき目標成分である。 FIG. 6 shows an image of the atmospheric model. Airglow exists in the atmospheric environment, and its intensity is represented by A. The amount of light entering the camera includes two components, an airglow component that directly enters and a light component that is reflected from the object. The atmospheric model can be expressed by Eq. (1).

However, I is an image of the amount of light entering the camera, that is, an image affected by the environment such as fog, A is an intensity component of airglow, J is a reflection component of an object, and t is an image of the atmosphere between the object and the camera. Transmittance. J is an image when there is no environmental influence such as fog, and is a target component to be obtained when removing fog or the like by the fog influence reducing means 21.

ただし、βは大気拡散係数、ｄは対象物からカメラまでの距離である。 The transmittance t can be expressed by the equation (2).

However, β is the atmospheric dispersion coefficient, and d is the distance from the object to the camera.

Equation (3) can be obtained from equation (1).

That is, if the intensity component A of airglow and the transmittance t of the atmosphere can be estimated, the influence of fog can be removed by the fog influence reducing means 21, and an image having only a clean reflection component J can be obtained.

In Reference 1 (Robby T. Tan, Visibility in bad weather from a single image, IEEE Conference on Computer Vision and Pattern Recognition, pp.1 -8, 2008), Tan proposes a method based on local area contrast maximization. doing. The proposed method is that t is a constant in the local region, and t in this region can be estimated based on maximizing the contrast in the local region on the premise of J <A. This method uses only contrast and can be supersaturated.

Reference 2 (R. Fattal. Single Image Dehazing [J]. ACM SIGGRAPH '08, pp.1 -9, 2008) proposes a fog removal method based on independent component analysis. Since the mode of the reflection component J of the object and the atmospheric transmittance are statistically independent in the local region, the reflectance can be estimated by the independent component analysis method. Since the setting of independence of this method is based on the color information of the image, it is not suitable for a photograph of the sea where the color information does not change much.

ただし、ｔ₀は大気透過率の初期値、ｋは比例係数、ｙはカメラより撮影された海の画像のｙ座標である。これにより、計算時間を大幅に短縮することができ、迅速に津波情報の算出が可能になる。 In the present embodiment, since the conventional method is used for estimating the airglow intensity component A, detailed description thereof will be omitted. The atmospheric transmittance t is calculated by a calculation method that approximates the vertical length of the captured sea image, that is, the y-coordinate of the image in a linear relationship, as shown in the following equation (4), without performing complicated calculation. suggest.

However, t ₀ is the initial value of atmospheric transmittance, k is the proportional coefficient, and y is the y coordinate of the sea image taken by the camera. As a result, the calculation time can be significantly shortened, and tsunami information can be calculated quickly.

[Rain effect mitigation means 22]
Further, the server 1 includes a rain influence reducing means 22 for reducing the influence of rain. FIG. 7 is a flow chart of processing by the rain influence reducing means 22. As shown in FIG. 7, the rain influence reducing means 22 obtains the maximum value image and the minimum value image of the color intensity of the pixels by using a plurality of continuously shot images, and provides a guide filter using the minimum value image as a guide. Reduce the effects of rain by applying and processing the maximum value image.

ただし、（ｉ，ｊ）は画素座標で、ｎ＝５である。 First, five consecutive images are taken at regular time intervals (S301).
Next, for each pixel of the five input image I _n (n = 1,2,3,4,5), the maximum value of the color intensity of the pixel from the equation (5), the pixel from the equation (6) Find the minimum value of the color intensity of. The minimum value of _{each pixel is used to generate the minimum value image I min} , and the maximum value of each pixel is used _{to generate the maximum value image I max} (S302).

However, (i, j) is the pixel coordinates, and n = 5.

ただし、μ_kとσ_kはそれぞれ小領域ω_kにおける最小値画像Ｉ_maxの画素の色強度の平均と分散であり、｜ω｜は小領域ω_kの画素の数であり、

は小領域ω_kにおける入力画像の色強度の平均値である。 Subsequently, with respect to the pixel of interest k (i, j), in the small region ω _k centered on k (i, j), the minimum value image I _min is used as a guide image, and the intermediate image I _f is obtained as follows ( S303).

However, μ _k and σ _k are the average and variance of the color intensities of the pixels of the minimum value image I _max in the small region ω _k, respectively, and | ω | is the number of pixels in the _{small region ω k.}

Is the average value of the color intensity of the input image in the small area ω _k.

ただし、Ａは調整係数である。 Finally, based on the input image I _n and the intermediate image I _f , the effect of rain is removed by processing the maximum value image by applying a guide filter as shown below, and the output image with enhanced wave appearance. (S304).

However, A is an adjustment coefficient.

[Wave extraction means 11]
The wave extraction means 11 extracts waves from images taken at the same time by a plurality of cameras 4A and 4B, respectively. It is difficult to apply a general pattern extraction method to the extraction of waves from captured images in the sea area, particularly the extraction of waves from long-distance captured images of 5 to 20 km, which is the target of the sea level measurement system in the present embodiment. The causes include uncertainties in wave shape, differences in wave characteristics in a plurality of images taken from a plurality of cameras 4A and 4B, and changes in the shooting environment due to all-weather shooting. In addition, high-speed detection is required.

FIG. 8 is an image diagram of wave extraction. As shown in FIG. 8, the captured image is first divided into several adjacent rectangular or square subregions. For the four corners of the upper left, upper right, lower left, and lower right of each small area, select a local area with the same area as the small area around the corner, and the threshold value for binarizing the image in this local area. (Small area threshold) T is obtained.

The threshold value T (i, j) for binarizing the attention point P (i, j), which is the image coordinates (i, j), is such that the attention point P (i, j) is set as in the equation (11). Based on the small area thresholds T of the four corners of the upper left, upper right, lower left, and lower right of the existing small area, it is calculated by multiplying _{the weighting factors k 1 to k 4 depending on the absolute distances to the four corners.}

That is, each point of interest is binarized using different weighting coefficients depending on the distance between the point of interest and the four angles. As a result, the binarization result of each pixel in the small area of interest can be smoothly connected to the binarization result of the pixels in the adjacent small areas of up, down, left, and right.

As described above, the component extracted by the binarization method (dynamic threshold calculation algorithm) based on the dynamic threshold in which the threshold for binarizing the point of interest dynamically changes is used as a wave in the image.

[Wave mapping means 12]
The wave associating means 12 finds and associates the same wave with each wave extracted by the wave extracting means 11 from the respective images simultaneously captured by the plurality of cameras 4A and 4B. FIG. 9 is a flow chart showing a flow of wave associative processing using two images. A similar method can be used when there are three or more images.

First, from multiple images taken by multiple cameras installed at different points, the region where the corresponding wave can exist, that is, the association candidate sea area, is obtained by analyzing the horizontal and vertical coordinate relationships of the epipolar geometry and the two-dimensional image. (S401).

Next, a wave extraction means is applied to the association candidate region of each image to extract m waves from the left image and n waves from the right image (S402). Further, as shown in FIG. 10, a two-dimensional feature vector is generated in which parameters such as the barycentric coordinates of the extracted wave image, the number of pixels, the perimeter, the circularity, and the length in each direction are the features of the corresponding wave. To do. As an example, the two-dimensional feature vector V can be defined as follows.

However, G is the coordinate of the center of gravity of the wave and is represented by G (x ₀ , y ₀ ). S is a parameter representing the magnitude of the wave, and is the number of pixels included in the extracted wave. L is the perimeter of the wave and C is the circularity of the wave. T is a parameter representing the length of the wave in the vertical direction and the horizontal direction, and is represented by T (h ₁ , h ₂ , w ₁ , w ₂ ).

なお、Ｐ₁（ｘ₁，ｙ₁）は、波を囲む長方形の左上の角の画素座標、Ｐ₂（ｘ₂，ｙ₂）は右下の角の画素座標である。

Note that P ₁ (x ₁ , y ₁ ) is the pixel coordinates of the upper left corner of the rectangle surrounding the wave, and P ₂ (x ₂ , y ₂ ) is the pixel coordinates of the lower right corner.

Next, the wave feature vector VL of the left image and the wave feature vector VR of the right image are created (S403).
_{For the m waves extracted from the left image, the characteristic parameter VL i} of the i-th (i = 1, 2, ..., M) wave is obtained according to the following equation, and the vector VL of the wave of the left image is generated.

_{For the n waves extracted from the right image, the characteristic parameter VR j} of the jth (j = 1, 2, ..., N) wave is obtained according to the following equation, and the vector VR of the wave in the left image is generated.

Next, for all the waves in the same region of each image, the candidate waves are identified from the epipolar constraint, the similarity of the two-dimensional feature vector is obtained, and the candidate wave list of the corresponding wave of each wave is obtained in descending order of similarity. To create. Specifically, for the i-th wave (i = 1, 2, ..., M) extracted from the left image, k waves near the epipolar line are extracted from the right image, and the left image in the right image Let k candidate waves of the corresponding wave of the i-th wave (S404, S405).

The similarity of the two-dimensional feature vector is calculated for k candidate waves, and a candidate wave list of the corresponding wave for the i-th wave extracted from the left image is created in descending order of similarity (S406). The order of the candidate waves is determined so that the wave with the highest degree of similarity is the first candidate wave and the wave with the second highest degree of similarity is the second candidate wave.

Based on the image coordinates of the center of gravity of the wave of interest and the upper wave of the candidate wave list, that is, the wave with high similarity, the three-dimensional world coordinates (X _note , Y _note , Z _note ) of the attention wave are obtained by the method of three-dimensional image measurement. The three-dimensional world coordinates (X _weather , Y _weather , Z _weather ) of the corresponding candidate wave are obtained (S407), and it is determined whether or not the difference is smaller than a predetermined threshold (S408).
If the difference is smaller than the predetermined threshold value, the corresponding wave is regarded as the corresponding wave of the attention wave (S409), and if not, the next wave is verified (S410, S411).

[Three-dimensional information calculation means 13]
The three-dimensional information calculation means 13 calculates the height of the sea surface and the height of the waves by three-dimensionally analyzing each wave of the image associated with the wave associating means 12.

There are waves on the surface of the sea, and the shape and size of the waves change from time to time. Therefore, when determining the height of the sea surface based on the three-dimensional measurement results of the waves on the sea surface, it is difficult to determine which part of which wave should be the height of the sea surface.
On the other hand, it is not necessary to obtain the height of a specific spot on the sea surface for tsunami measurement, and the height of the sea surface in a certain area may be calculated to determine the presence or absence of a tsunami.

Therefore, in the sea level measurement system of the present embodiment, the height of the sea level in a small area is calculated from the captured image. FIG. 11 is an image diagram of sea surface height calculation using images taken by two left and right cameras, and FIG. 12 is a flow diagram showing a flow of sea surface height calculation using images taken by two left and right cameras. is there.

As shown in FIG. 12, first, the same area is continuously photographed by the left and right cameras 4A and 4B, and a plurality of continuous time-series images are acquired (S501). For example, one image is taken every 1/30 second for 5 seconds, 150 images are taken from each of the left and right cameras, and a total of 300 images are acquired.

Next, waves are extracted from each image and associated with each other (S502).
Then, for all the associated waves, the three-dimensional world coordinates (X, Y, Z) of the feature points such as the center of gravity, the center, the upper part, the lower part, the left end, and the right end are obtained (S503). With respect to the camera, X indicates the horizontal direction, Y indicates the vertical direction, and Z indicates the depth direction.

Based on the calculated three-dimensional world coordinates (X, Y, Z) of each wave, the wave is classified into some small regions (S504). For example, as shown in FIG. 11, the waves are classified into three subregions.
For a particular subregion, for all associated waves in all time-series images, as in equation (24), the vertical direction of the three-dimensional world coordinates of the wave's center of gravity and other feature points, or sea level. _{The average value Y A} of the Y coordinate values in the direction perpendicular to _{is obtained, and the height H s} of the sea level in the relevant small area is calculated by multiplying the average value Y A (S505).

ただし、ｋ_sは計算のための比例係数、Ｈ_s0は海面の高さを計算する際における三次元世界座標を調整するための係数である。

However, the proportional coefficient for k _s is calculated, H _s0 is a factor for adjusting the three-dimensional world coordinates at the time of calculating the height of the sea surface.

ただし、ｋ_Vは計算のための比例係数、Ｈ_V0は波の高さを計算する際における世界座標を調整するための係数である。 Finally, for a specific small region, the variance value V (Y) of the Y coordinate values of all the associated waves of all the time series images is obtained and multiplied by a coefficient as in Eq. (25). Then, the wave height H _V of the relevant small region is calculated (S506).

However, k _V is a proportional coefficient for calculation, and H _{V 0} is a coefficient for adjusting the world coordinates when calculating the wave height.

[Distance camera system calibration means 15]
The calibration of the camera system involves internal parameters such as the pixel arrangement of the image sensors of the cameras 4A and 4B and the focal length of the lens, the distance between the two cameras 4A and 4B, that is, the baseline length, and the position and orientation of the cameras 4A and 4B. It is to obtain external parameters such as rotation angle. Many camera system calibration methods have been proposed, and currently, in particular, Zhang's method (Zhang. Z, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 22, No. 11, 2000, pp. 1330-1334.) Has been evaluated and implemented in OpenCV.

However, long-distance photography is required for tsunami measurement, and calibration of a long-distance three-dimensional image measurement system requires a large-sized calibration object, which requires objects from different visions. There are still problems such as the need to shoot.

Therefore, the long-distance camera system calibration means 15 in the present embodiment separates inside and outside using different targets by fusing an internal parameter calibration method using a plate-shaped target and an external parameter calibration method using a columnar target. Two-stage calibration is used.

In the long-distance camera system calibration means 15 of the present embodiment, two or more cylindrical targets or other elongated targets having a constant width are used as targets for external parameter calibration. FIG. 13 is an example of a cylindrical target.

The reason for selecting a columnar object as the object to be calibrated is that, as shown in FIG. 13 (A), the distance from the measurement point used for image measurement to the center line of the columnar target changes even when viewed from different viewpoints. This is because it does not. For example, when the target is observed from the line of sight 1, the measurement point 1 is observed on the image, and the distance from the measurement point 1 to the center line of the cylinder is R. Even when observing from the line of sight 2, the distance from the observation point 2 to the center line of the cylinder is the same R as when observing from the line of sight 1. That is, in the columnar target, a calibration error does not occur due to a change in the observation point.

Considering that it is a long-distance calibration, the length of an elongated target such as a cylinder is set to 1 meter or more. Further, in order to make the target easy to recognize, it is assumed that the surface of the target has two or more colors such as white and red and other colors that can be clearly distinguished.

A feature for calibrating a point on the boundary line of the different colors such as the feature point candidate 1 and the feature point candidate 3 shown in FIG. 13B, or a point or the center of gravity in the center of the color such as the feature point candidate 2. Let it be a point.

Further, the size of the region of different colors of the target such as L1, L2, and L3 shown in FIG. 13C is used as a known parameter in the calibration.

FIG. 14 is a flow chart showing the flow of calibration by the long-distance camera system calibration means 15.

First, a photograph of the calibration target is taken by a plurality of cameras 4A and 4B (S601).
Next, a plurality of feature points required for calibration are extracted from each image (S602).
After that, the extracted feature points are associated with each other (S603).
Finally, the parameters of the camera are calculated using the associated feature points and the known parameters such as L1, L2, and L3 described above (S604).

[Means for determining the presence or absence of sea level abnormality 14]
The sea level abnormality determination means 14 determines the presence or absence of a tsunami based on the sea level height and the wave height calculated by the three-dimensional information calculation means 13. FIG. 15 is a flow chart showing a flow of determining the presence or absence of a tsunami.

As shown in FIG. 15, first, in order to determine the presence or absence of a tsunami, the sea level in some specific areas calculated based on the three-dimensional image measurement technology and the tide level in normal times when no tsunami occurs. Are compared (S701). Based on the results of the comparison, these specific regions are classified into three regions, such as a tsunami generation candidate region, a region where a tsunami may occur, or a region where a tsunami does not occur.

If the difference between the measured sea level of a plurality of specific areas and the tide level in normal times when no tsunami occurs is larger than the specified threshold value, these areas are classified as tsunami generation candidate areas. If the difference between the sea level height of the above multiple specific areas and the normal tide level without tsunami is larger than the specified threshold and smaller than the specified threshold, there is a possibility of tsunami occurrence in these areas. Classify into areas. If the difference between the sea level height of the above-mentioned multiple specific areas and the tide level in normal times when no tsunami occurs is smaller than the specified threshold value, these areas are classified as areas where no tsunami occurs.

For the tsunami generation candidate area or the area where tsunami may occur, a plurality of small areas are added as measurement areas around the area (S702), and the measurement result of the sea level is added to these areas. Compare with normal tide level (S703). Based on the comparison result between the sea level height of these regions and the tide level in normal times, it is determined whether or not a tsunami has occurred using the phage estimation theory (S704), and the relevant specific region has a tsunami or no tsunami. Judge in the area. It is possible to determine sea level abnormalities such as high waves, storm surges and rapid tides other than tsunamis by the same procedure.

[Tsunami traveling speed calculation means 16, tsunami scale estimation means 17, tsunami arrival time estimation means 18]
The tsunami traveling speed calculation means 16 uses the tsunami traveling speed calculated based on the conventional tsunami traveling speed calculation method, corrects the tsunami traveling speed calculated based on the three-dimensional image measurement, and makes the tsunami traveling speed more accurate. To calculate.

FIG. 16 is an image diagram of three-dimensional image measurement of the height of the sea surface at each time.
When the occurrence of a tsunami is measured as a sea level abnormality, the height of the sea surface in the tsunami generation area and its surrounding area is measured at any time, and the maximum area of the sea level is detected at any time. In addition, the posture control of the cameras 4A and 4B tracks the region where the sea level closest to the cameras 4A and 4B is the maximum, that is, the tsunami front, and the distance to the tsunami front, that is, the Z value of the three-dimensional measurement result. To get.

As shown in FIG. 16, four-dimensional information (X, Y, Z, t (measurement time)) of the sea level change by three-dimensional image measurement of the sea level at each time such as time 1 and time 2. _{Is obtained, and the tsunami traveling speed V m} calculated based on the altitude change of the sea level of the tsunami front, the traveling direction and the traveling speed, that is, the image measurement is calculated.

ただし、ｈは計測領域の海の水深であり、ｇは重力加速度である。 Further, the formula (26), calculates the moving speed V _C of the tsunami on the basis of the conventional calculation method.

However, h is the depth of the sea in the measurement region, and g is the gravitational acceleration.

ただし、ｋ_mは津波速度補正のための係数である。 _{Using the tsunami traveling speed V C} calculated based on the conventional calculation method, the tsunami traveling speed V _m calculated by image measurement is corrected to calculate a more accurate tsunami traveling speed V.

However, k _m is a coefficient for the tsunami speed correction.

The tsunami scale estimation means 17 evaluates the scale of the tsunami based on the height of the tsunami. For example, the tsunami scale estimation means 17 evaluates in six stages of -1,0,1,2,3,4 according to the height of the tsunami.

The tsunami arrival time estimation means 18 calculates the time when the tsunami arrives at the coast from the position of the tsunami front, the height of the tsunami, and the tsunami traveling speed measured by the three-dimensional image measurement.

[Memory means 20]
First, the observer makes a measurement plan such as when and which sea area to measure. The server 1 determines the posture of each camera according to this measurement plan, calculates the rotation angle of each camera, and transmits the rotation angle to each of the client computers 2A and 2B. The storage means 20 stores captured images of the cameras 4A and 4B acquired from the client computers 2A and 2B. The server 1 calculates and outputs the presence or absence of a sea level abnormality such as a tsunami by processing the images stored in these storage means 20.

[Two-stage control means 23]
Further, in the sea surface measurement system of the present embodiment, the photographing means 10 includes a machine and an image two-stage control means 23 that combine mechanical control based on angle feedback and image measurement control based on image shooting and image processing. ..

The two-stage control means 23 first performs mechanical control based on angle feedback using a low-resolution angle sensor, and outlines the line of sight of each camera 4A, 4B so that the plurality of cameras 4A, 4B face approximately the same sea area. Take control. The goal of the rough control is to roughly adjust the line of sight of each camera 4A, 4B so that a part of the wave on the measurement target sea surface can be photographed by each camera 4A, 4B.

After the rough control, photographs are taken by the cameras 4A and 4B, the images taken by the cameras 4A and 4B are fed back, and the line of sight of the cameras 4A and 4B is precisely controlled by the image measurement control. The goal of precision control is to enable the cameras 4A and 4B to capture the waves on the sea surface as much as possible even if there are mechanical control errors or camera vibrations.

FIG. 17 is an image diagram of mechanical control of the line of sight of the camera. The mechanical control is a rough control of the first stage, and is a control for ensuring that the lines of sight of the two cameras 4A and 4B are directed to the same sea area. In the rough control, the accuracy of the line-of-sight control is not required, and if it can be ensured that two or more cameras 4A and 4B face the same sea area, that is, the same wave is applied to the images taken by the two cameras 4A and 4B. It would be nice if some could be detected. Therefore, an angle sensor having a low resolution of about 0.1 ° can be used for rough control.

FIG. 18 is an image diagram of precision control by image measurement control of two cameras. A similar control method can be applied when the number of cameras is three or more. The image measurement control is a second-stage precision control, in which the angles and images of the cameras 4A and 4B are finely adjusted by image feedback to ensure that the same target is captured by the two cameras 4A and 4B.

In order to shoot the same target from the two left and right cameras 4A and 4B, it is necessary to send the target line-of-sight angles to the left and right cameras 4A and 4B, respectively. One of the plurality of cameras is used as the main camera and the other is used as the sub camera. In the present embodiment, the left camera 4A is used as the main camera, the right camera 4B is used as the sub camera, and the line of sight of the left camera 4A, that is, the main camera. The line-of-sight angle of the right camera 4B, that is, the sub camera is automatically calculated according to the angle.

The key point of image measurement control is to feed back the images taken from the left and right cameras 4A and 4B, compare them with the target shot image, and based on the difference, further pan and tilt the camera fixed adjustment units 5A and 5B. Adjust the angle of. As a result, it is possible to correct an error caused by vibration of the cameras 4A and 4B, that is, an error such as the disturbance shown in FIG.

FIG. 19 is a flow chart showing a flow of precision control by image measurement control when two cameras are held. First, in the server 1, the line-of-sight angle for adjusting the postures of the cameras 4A and 4B, that is, the rotation angle in the pan and tilt directions is calculated from the shooting target area. After that, using the theory of triangulation, the line-of-sight angle of the right camera 4B, that is, the rotation angle in the pan and tilt directions is calculated based on the spatial relationship between the cameras 4A and 4B and the shooting target area, and the respective cameras 4A and 4B are controlled. It is sent to the client computers 2A and 2B (S801).

The client computer 2A connected to the left camera 4A sets the pan and tilt rotation angles of the left camera 4A received from the server 1 as the target value of the line-of-sight angle, and the actual line-of-sight angle of the left camera 4A acquired from the angle sensor and this line-of-sight. Machine control is performed by comparing the angle with the target value and adjusting the pan and tilt angles of the camera fixing adjustment unit 5A so as to minimize the difference (S802). Machine control is also performed on the right camera 4B in the same manner (S803).

After that, photographs are taken by the left camera 4A and the right camera 4B, respectively, and waves are extracted from the left and right captured images, respectively (S804, S805).
Next, from the waves extracted from the left and right images, the area that exists in both the left and right images, that is, the common area, is obtained, and one wave located at the center of the common area is extracted and used as the target wave for camera posture adjustment. (S806).

It is desirable that the target wave is located at the center of both the left and right images. The target wave is the attitude control target of the left and right cameras 4A and 4B, that is, the shooting target. For the left camera 4A, it is determined whether or not the target wave is within a certain range around the center of the image (S807), and if it is within the range, the attitude adjustment of the left camera 4A is completed, and if it is not within the range. Performs the above-mentioned mechanical control to adjust the pan and tilt angles (S809).

Next, a photograph is taken after the machine control, a target wave is extracted from the captured image, and it is examined whether or not image adjustment is necessary (S811, S813). If the target wave is in a certain area in the center of the image, there is no need to adjust the image, otherwise the image is translated within a certain range in the horizontal and vertical directions to adjust the image ( S815).

By the above image adjustment, it is checked whether the target wave is adjusted to the center of the left image, and if it is located in the center region of the image, the adjustment is finished. If not, it returns to the mechanical control of S809 and repeats the control (S809). ~ S815). That is, if the target wave does not reach a position near the center of the image no matter how much the image is translated, it means that the controllable range of the image measurement control is narrow and the control target cannot be achieved. In this case, that is, the part that the image measurement control does not reach is realized by the machine control. Adjust the posture of the camera 4B in the same way for the right camera 4B.

Finally, it is checked whether the two left and right cameras 4A and 4B have completed the adjustment, and if both are completed, the control of the cameras is terminated, and if not, the operation is waited until the end (S819, S820).

That is, in the above image measurement control, angle feedback by an angle sensor that acquires the pan and tilt angles of the camera fixing adjustment units 5A and 5B in which the cameras 4A and 4B are fixed so that the posture can be adjusted, and photography of the cameras 4A and 4B are taken. It has image feedback that extracts measurement objects such as waves from the camera and identifies the postures of the cameras 4A and 4B. The angle feedback is internal feedback, the image feedback is external feedback, and the image feedback has a double feedback structure that surrounds the angle feedback. First, the machine control is performed, and the part where the machine control is insufficient, that is, the error of the machine control is supplemented by the image measurement control, and the part which the image measurement control does not reach is realized by the machine control.

In the above embodiment, a sea level measurement system that measures the presence or absence of sea surface abnormalities such as a tsunami has been described as an example, but even if the shooting target is other than the sea surface, it is the same by a plurality of cameras installed at different points at the same time. It can be applied to a measurement system that captures an image including the imaging target area and performs various measurements of the imaging target, and has mechanical control based on angle feedback and image measurement control based on image imaging and image processing. It is possible to track the same shooting target with high accuracy by the two-stage control that combines the control.

The measurement system, measurement method, and measurement program of the present invention are applied to measure the presence or absence of a tsunami, measure the height of a tsunami when a tsunami occurs, measure the traveling speed of a tsunami, estimate the arrival time of a tsunami, and the like. it can. In addition to tsunamis, it can also be applied to measure the height, position, and arrival time of waves generated by typhoons as sea level abnormalities. In addition, it can be applied to the measurement of sea level abnormalities such as high waves, storm surges and rapid tides, and to the monitoring of coasts, and it can also be applied to measurements other than the sea level.

1 Server 2A, 2B Client computer 3A, 3B Camera set 4A, 4B Camera 5A, 5B Camera fixed adjustment unit 10 Imaging means 11 Wave extraction means 12 Wave mapping means 13 Three-dimensional information calculation means 14 Sea level abnormality presence / absence judgment means 15 Long distance Camera system calibration means 16 Tsunami traveling speed calculation means 17 Tsunami scale estimation means 18 Tsunami arrival time estimation means 19 Tsunami information transmission means 20 Storage means 21 Fog effect mitigation means 22 Rain effect mitigation means 23 Two-stage control means

Claims (5)

It is a measurement system that uses a shooting means that simultaneously shoots an image including the same shooting target area with multiple cameras installed at different points.
It is a two-stage control means that combines machine control based on angle feedback, image capture, and image measurement control based on image processing, and image measurement control.
The mechanical control roughly adjusts the line of sight of each camera so that each camera can shoot a part of the shooting target in the shooting target area by the mechanical control based on the angle feedback using the angle sensor.
A measurement system including a two-stage control means in which the image measurement control takes a picture with each camera, feeds back an image taken from each camera, and precisely controls the line of sight of each camera. The image measurement control is
When one of the plurality of cameras is used as the main camera and the other is used as the secondary camera, the line-of-sight angle of the main camera is first calculated from the shooting target area and used as the target value of the line-of-sight angle of the main camera.
The claim is characterized in that the line-of-sight angle of the sub-camera is calculated by the principle of triangulation based on the installation position of the main camera and the sub-camera and the shooting target, and is set as the target value of the line-of-sight angle of the sub-camera. 1 The measurement system described. The image measurement control is
Angle feedback by an angle sensor that acquires the pan and tilt angles of the camera fixed adjustment unit where the camera is fixed so that the camera can be adjusted in posture, and image feedback that extracts the measurement object from the camera's photography and identifies the camera's posture. The measurement system according to claim 1 or 2, wherein the angle feedback is an internal feedback, the image feedback is an external feedback, and the image feedback has a double feedback structure surrounding the angle feedback. Taking images that include the same shooting target area at the same time with multiple cameras installed at different points,
By mechanical control based on angle feedback using an angle sensor, the line of sight of each camera can be roughly adjusted so that a part of the shooting target in the shooting target area can be shot by each camera.
A measurement method that includes taking a picture with each camera, feeding back the image taken from each camera, and precisely controlling the line of sight of each camera. It is a measurement program that uses a shooting means to shoot an image including the same shooting target area at the same time by multiple cameras installed at different points.
It is a two-stage control means that combines machine control based on angle feedback, image capture, and image measurement control based on image processing, and image measurement control.
The mechanical control roughly adjusts the line of sight of each camera so that each camera can shoot a part of the shooting target in the shooting target area by the mechanical control based on the angle feedback using the angle sensor.
A measurement program in which the image measurement control functions as a two-stage control means for taking a picture with each camera, feeding back the image taken from each camera, and precisely controlling the line of sight of each camera.

Images