JP6178663B2 - Data complementing device, data complementing method, and data complementing program - Google Patents

Description

  The present invention relates to a data complementing device, a data complementing method, and a data complementing program that complement data.

  Since GPS (Global Positioning System) has been made public, GPS receiving terminals are mounted on mobile bodies such as ships, airplanes, and automobiles, and an environment in which it is easy to obtain trajectory data of the mobile body has been established. For example, there is a ship on which an automatic identification system (Automatic Identification System, AIS) is mounted for the purpose of safe navigation of the ship. A GPS receiving terminal is mounted on or connected to the AIS. AIS periodically distributes ship track data (hereinafter referred to as AIS data) such as position and speed measured by GPS to nearby ports and ships.

  In addition, since satellite images with high resolution can be obtained at a relatively low price, information such as size and position is extracted from the captured moving body, and the track data of the moving body is combined to obtain mutual information. Technologies that support referrals have been developed. For example, Non-Patent Document 1 discloses a SAR (Synthetic Aperture Radar: Synthetic) by extracting ship information such as position, size, and speed from a SAR satellite image and comparing the ship coordinates, size, and speed in AIS data. A method for mapping a ship image on an Aperture Radar image and AIS data is disclosed.

  In this method, the position of the shooting time is estimated based on the speed and the direction of the ship in one AIS data acquired after shooting of the SAR satellite, and the AIS data whose size or speed difference is less than or equal to the threshold value using the proximity method. And the ship image correspond to each other. As a result, the ship image and the AIS data on the satellite image can be referred to each other, and the missing AIS data, the AIS data without the corresponding ship image, and the like can be detected.

S.K.Chaturvedi, “Ship Recognition by Integration of SAR and AIS”, Journal of Navigation, Volume 65, Issue 02, pp. 323-337, April 2012.

  However, in the above-described prior art, if an manually input value such as an International Maritime Organization (IMO) number (ship identification number) or a size is incorrect, or time-series AIS data in one ship is missing, AIS An error may occur in the correspondence between the data and the ship image.

  For example, if there is an error in the size value, it will deviate from the size obtained from the ship image on the SAR image, and cannot be associated correctly. In addition, a ship with an incorrect IMO number is a ship that should normally display a warning as an abnormal ship, such as a suspicious ship such as a pirate disguised as a normal ship. However, if there is an error in the correspondence between the AIS data and the ship image, there is a problem that a false alarm is issued.

  In addition, if the AIS data is forgotten to be turned on or if the AIS is broken, or if the AIS data is thinned out to reduce storage costs and communication costs in the storage that collects the AIS data, the AIS data may be lost. There are many. In such a case, the coordinate value of the ship must be estimated from the AIS data deviated from the satellite image capturing time. However, when an operation such as speed adjustment or turning is performed, it is difficult to estimate the shooting time of the satellite image. In particular, in a sea area where GPS errors or ships are concentrated, the ship image is not necessarily near the coordinate value indicated by the AIS data, and it becomes difficult to associate the AIS data with the ship image.

  An object of the present invention is to improve the reliability of a correspondence relationship between time-series observation data of a moving object and time-series image data of the moving object.

A data complementing device, a data complementing method, and a data complementing program, which are one aspect of the invention disclosed in the present application, are obtained by a processor at a first observation time before and after the imaging time of area image data obtained by imaging an area from a first database. When the first observation data including the first feature data specifying the moving body and the behavior thereof is acquired, it is determined whether or not the value of the first feature data is abnormal. Filtering processing for setting abnormal information indicating abnormal in the first feature data, an image of a second moving body moving in the region on the region image data from the second database, and the first The moving object image data including the second moving object and the second feature data specifying the behavior of the moving object is acquired, and the abnormality information is set by the filtering process. The identity of the first moving body and the second moving body is determined based on the first observation data including the first feature data that has not been obtained and the moving body image data. When it is determined that there is identity, first information indicating that there is a correspondence relationship between the mobile object image data and the first feature data is set, and when it is determined that there is no identity, When the abnormality information is set in one feature data, second information indicating that the moving object image data is not associated is set in the first feature data, and the first feature data corresponds The abnormality information is set by the mapping process for setting the third information indicating that it cannot be attached to the moving body image data and the filtering process, and the second information is set by the mapping process Based on the first feature data and the moving body image data in which the third information is set, the identity of the first moving body and the second moving body is determined, and the identity A first complement process that complements the first feature data so as to be the same as the second feature data when it is determined that there is identity or when it is determined that there is identity in the mapping process; At least the first complement processing of generating second observation data based on the second feature data when it is determined that there is no identity or when it is determined that there is no identity in the mapping processing A complementing process for executing a complementing process and an output process for outputting the mapping process or a processing result by the complementing process are executed, and in at least one of the mapping process and the complementing process, In the identity determination, the processor determines the position, speed, and preceding observation time of the first moving body at a preceding observation time that is an observation time before the photographing time in the first feature data. An azimuth indicating a moving direction from the position of the first moving body to the position of the first moving body at a subsequent observation time that is an observation time after the photographing time in the first feature data Based on the first estimated position of the first moving body at the photographing time, and based on the position, speed, and azimuth of the first moving body at the subsequent observation time Calculating a second estimated position of the first moving body at the photographing time, and when both the first estimated position and the second estimated position are positions in the area image data, Confirming the identity of the first moving body and the second moving body Calculating a corresponding score indicating a on the basis of the calculated corresponding score, and judging the identity between the second mobile body and the first moving body.

  According to the representative embodiment of the present invention, it is possible to increase the accuracy of the correspondence between the time-series observation data of the moving object and the time-series image of the moving object. Problems, configurations, and effects other than those described above will become apparent from the description of the following embodiments.

It is explanatory drawing which shows an example of matching with AIS data and ship image data. It is explanatory drawing which shows the data supplement example 1. FIG. It is explanatory drawing which shows the data supplement example 2. FIG. It is explanatory drawing which shows the system configuration example of a communication system. It is a block diagram which shows the structural example of a data complementation apparatus. It is explanatory drawing which shows the example of a data structure of satellite image data. It is explanatory drawing which shows the example of a data structure of ship image data. It is explanatory drawing which shows the data structure example of AIS data. It is explanatory drawing which shows the example of a data structure of the time series data shown in FIG. It is explanatory drawing which shows the example of a data structure of ship corresponding | compatible data. It is a flowchart which shows the example of a detailed process sequence by a data complementation apparatus. It is a flowchart which shows the example of a detailed process sequence of the filtering process (step S1101) by the filtering process part shown in FIG. 12 is a flowchart 1 illustrating a detailed processing procedure example of mapping processing (step S1106) by the mapping processing unit illustrated in FIG. 11. 12 is a flowchart 2 illustrating a detailed processing procedure example of mapping processing (step S1106) by the mapping processing unit illustrated in FIG. 11. It is explanatory drawing which shows the relationship between an estimated route and the maximum search distance. It is a flowchart which shows the example of a detailed process sequence of the time shift process (step S1306) shown in FIG. It is a flowchart which shows the detailed process sequence example of the maximum search distance calculation process (step S1401) shown in FIG. It is a flowchart which shows the detailed example of a process sequence of the corresponding | compatible score calculation process (step S1405) shown in FIG. It is explanatory drawing which shows the relationship between the direction of ship and an azimuth. 12 is a flowchart 1 illustrating a detailed processing procedure example of remapping processing (step S1107) by the complement processing unit illustrated in FIG. 11. 12 is a flowchart 2 illustrating a detailed processing procedure example of remapping processing (step S1107) by the complement processing unit illustrated in FIG. 11. It is explanatory drawing which shows an abnormality and a defect | deletion complementation example. It is explanatory drawing which shows the example of the output screen of the complementation result of the AIS data output by the data display process part of FIG. It is explanatory drawing which shows Example 1 of the output screen of AIS data and the ship image output by the data display process part of FIG. It is explanatory drawing which shows the example 2 of the output screen of AIS data and the ship image output by the data display process part of FIG. It is explanatory drawing which shows the example of the data display drawing output by the data display process part of FIG.

  Hereinafter, exemplary embodiments of a mobile body information complementing device according to the present invention will be described with reference to the accompanying drawings. In the following drawings, parts denoted by the same reference numerals represent the same items, and the basic configuration and operation are the same. In addition, in embodiment of this invention, the apparatus, method, etc. which are used are examples, and of course, this invention is not limited to these.

  Furthermore, as will be described later, the embodiment of the present invention may be implemented by software running on a general-purpose computer, or may be implemented by dedicated hardware or a combination of software and hardware. Further, the present invention uses AIS data as an example, and is also a target for complementing other mobile information.

<Association between AIS data and ship image data>
FIG. 1 is an explanatory diagram showing an example of association between AIS data and ship image data. In FIG. 1, the screen which mapped AIS data and ship image data is shown. The AIS data and the satellite image data from which ship image data is extracted are data obtained by different routes and asynchronously. The AIS data a1 includes the ship ID of the ship, the size of the ship, and the speed of the ship. The ship image data A1 includes a ship image ID, a ship size, and a ship speed. The association between the AIS data a1 and the ship image data A1 is performed by comparing the two data. For example, if the difference in size and the difference in speed are within an allowable range, the AIS data a1 and the ship image data A1 are associated with each other as the same ship.

  In the example of FIG. 1, the ship image data A1 is data acquired at 16:25:05 on November 11, 2012, and the AIS data a1 is 16:00 on the same day (t0 in FIG. 1), 16 : 20 (t1 in FIG. 1), data acquired at 16:40 (t2 in FIG. 1). t0 to t2 are time series data included in the AIS data a1, and include the speed of the ship, the acquisition time, and the position at the acquisition time. By associating the AIS data a1 and the ship image data A1 as in the present embodiment, it is possible to confirm the time-series navigation route of the ship on the satellite image.

<Example of data completion>
FIG. 2 is an explanatory diagram showing a data supplement example 1. As shown in FIG. Data supplement example 1 is an example in which AIS data a2 and ship image data A2 are associated by complementing AIS data a2 when there is an abnormality in AIS data a2.

  (A) In FIG. 2, among the sizes (0 m, 23 m) of the AIS data a2, the length of the ship is “0 m” due to an erroneous input, which is shorter than the shortest length of the ship. Accordingly, the AIS data a2 cannot be associated with the ship image data A2. In this way, the AIS data a2 including an abnormal value is referred to as abnormal data.

  (B) In this embodiment, for the abnormal data, the data complementing device obtains a correspondence score that becomes an index value indicating whether the data is suitable for associating with the ship image data A2 by data other than the abnormal value. When it is determined from the correspondence score that the AIS data a2 is data suitable for associating with the ship image data A2, the data complementing apparatus has the ship length “101 m” in the ship image data and the AIS data a2 The data is complemented by updating the length of the ship “0 m”. Thereby, like FIG. 1, AIS data a2 and ship image data A2 can be matched now.

  FIG. 3 is an explanatory diagram illustrating a data supplement example 2. As illustrated in FIG. Data complementation example 2 is an example in which AIS data a3 and ship image data A3 are associated by complementing AIS data a3 when AIS data is missing.

  (A) When there is a deficiency in AIS data or time-series data in AIS data, AIS data and ship image data A3 cannot be associated with each other.

  (B) In this embodiment, when there is a deficiency in the AIS data associated with the ship image data A3, the data complementing apparatus uses the ship image data A3 to generate AIS data a3 corresponding to the ship image data A3. Supplement the AIS data. Thereby, like FIG. 1, it becomes possible to associate the AIS data a3 with the ship image data A3.

<System configuration example>
FIG. 4 is an explanatory diagram illustrating a system configuration example of the communication system. In the communication system 400, three types of artificial satellites 401 to 403 orbiting the earth are used. The first artificial satellite is an artificial satellite that captures a satellite image and transmits it to the database server 407. The second artificial satellite 402 is an artificial satellite that transmits and receives AIS data. The third artificial satellite 403 is an artificial satellite that transmits a GPS signal including time information and orbit information of the third artificial satellite 403 itself to the AIS 405.

  In addition, the communication system 400 includes an AIS 405, a base station 406, a database server 407, and a data complementing device 408. As described above, the AIS 405 is mounted on the ship 404, and a GPS receiving terminal is mounted or connected thereto. The AIS 405 receives a GPS signal from the third artificial satellite 403, measures AIS data such as the position and speed of the ship on which the AIS 405 is mounted, and periodically base stations 406 and ships 404 in neighboring ports. Or wirelessly distributed to the second artificial satellite 402. When AIS data is distributed to the second artificial satellite 402, the AIS data is distributed from the second artificial satellite 402 to the base station 406.

  The base station 406 is installed, for example, in a harbor, receives AIS data from the AIS 405 of the ship 404 and the second artificial satellite 402, and transfers it to the database server 407. Further, the base station 406 receives the satellite image data captured by the first artificial satellite 401 from the first artificial satellite 401 and transfers it to the database server 407. The satellite image data may be received by another receiving device (not shown) other than the base station 406 and transferred to the database server 407.

  The database server 407 accumulates AIS data and satellite image data. In this example, for the sake of convenience, the same database server 407 accumulates AIS data and satellite image data. However, the database server 407 that accumulates AIS data and the database server 407 that accumulates satellite image data are different. It may be a server. In any case, AIS data and satellite image data may be given to the data complementing device 408.

  The data complementing device 408 maps the ship image data and the AIS data on the satellite image in the sea area, and complements the abnormality (FIG. 2) or loss (FIG. 3) of the AIS data. Specifically, the data complementing device 408 determines the estimated range of the position of the ship 404 at the imaging time of the satellite image data using the two AIS data at the observation time immediately before and immediately after the imaging time of the satellite image data. Then, the attributes of the AIS data and the attributes that can be extracted from the ship image data are compared. Then, the data complementing device 408 collates the ship image reflected in the satellite image with the AIS data, detects the above-described abnormality or defect, and complements the AIS data. Each function of the data complementing apparatus 408 will be described with reference to FIG.

<Configuration Example of Data Complementing Device 408>
FIG. 5 is a block diagram illustrating a configuration example of the data complementing device 408. In FIG. 5, the data complementing device 408 is shown as a single computer. However, the data complementing device 408 is configured as a computer system in which each component is remotely arranged and connected via a network. Anyway.

  The data complementing apparatus 408 includes a satellite image DB 500, a ship image DB 501, an AIS data DB 502, a ship correspondence DB 503, a master DB 505, a central processing unit 505, an input device 506, a display device 507, and a data memory 508. A program memory 509.

  The satellite image DB 500 stores satellite image data acquired by the first artificial satellite 401. The ship image DB 501 stores ship image data extracted from satellite image data. The AIS data DB 502 stores regularly collected AIS data. The ship correspondence DB 503 stores ship correspondence data indicating a correspondence relationship between AIS data and ship image data. The master DB 504 stores the registered IMO number and the size range of the reference ship 404. These DBs 500 to 505 realize their functions by a storage device such as a nonvolatile memory or a hard disk. Further, at least one of these DBs 500 to 505 may be configured to be held by the database server 407.

  The central processing unit 505 includes a filtering processing unit 511, a mapping processing unit 512, a complementing processing unit 513, and a data display processing unit 514. In the case of this form example, it is configured by a computer, and the filtering processing unit 511, the mapping processing unit 512, the complementing processing unit 513, and the data display processing unit 514 are all realized as part of the functions of the program executed on the computer. Is done. Note that these programs are stored in the program memory 509. Details of the filtering processing unit 511, the mapping processing unit 512, the complement processing unit 513, and the data display processing unit 514 will be described later.

  The input device 506 inputs data to the central processing unit 505, and the display device 507 displays data from the central processing unit 505. The data memory 508 stores satellite image data 521, ship image data 522, AIS data 523, and ship correspondence data 524. The program memory 509 stores an OS (Operating System), a program that realizes a filtering processing unit 511, a mapping processing unit 512, a complementing processing unit 513, and a data display processing unit 514.

<Data structure example>
Next, an example of the data structure of data stored in the various DBs 500 to 505 and the data memory 508 shown in FIG. 2 will be described.

  FIG. 6 is an explanatory diagram showing an example of the data structure of the satellite image data 521. The satellite image data 521 is stored in the satellite image DB 500, read by the central processing unit 505, and written in the data memory 508. Then, after the update, it is stored again in the satellite image DB 500.

  The satellite image data 521 includes a satellite image ID 600, a photographing time 601, a minimum longitude 602, a minimum latitude 603, a maximum longitude 604, a maximum latitude 605, and pixel data 606, for example, in the form of an array. Is retained.

  The satellite image ID 600 holds a value uniquely shaken in the satellite image data 521. At the photographing time 601, the date when the satellite image was photographed is held as a value. In the minimum longitude 602, the minimum latitude 603, the maximum longitude 604, and the maximum latitude 605, coordinate values indicating the range of the satellite image in the geographical coordinate system on the celestial surface are held as values. The pixel data 606 holds RGB color model values in a two-dimensional array according to the size of the satellite image. Pixel data 606 is displayed as a satellite image on the display device.

  FIG. 7 is an explanatory diagram illustrating a data structure example of the ship image data 522. The ship image data 522 is stored in the ship image DB 501, read by the central processing unit 505, and written in the data memory 508. And after update, it is stored in ship image DB501 again. The ship image data 522 is data extracted from satellite image data by existing technology.

  Ship image data 522 includes satellite image ID 700, ship image ID 701, longitude 702, latitude 703, speed 704, direction 705, length 706, width 707, pixel data 708, and data state 709. The information is held in the form of an array, for example.

  The ship image ID 701 holds a value uniquely shaken with respect to the ship image data 522. In the longitude 702 and the latitude 703, coordinate values of the ship 404 in the geographical coordinate system on the celestial surface are held as values. The speed 704 holds the speed of the ship 404 extracted from the ship image data as a value. In the direction 705, the orientation information of the ship 404 extracted from the ship image data is held as a value. The length 706 and the width 707 hold the size of the ship 404 extracted from the ship image data as values. In the pixel data 708, RGB color model values are held in a two-dimensional array according to the size of the ship image detected from the satellite image data. Pixel data 708 is displayed as a ship image on the display device.

  In the data state 709, identification information indicating whether or not AIS data has been transmitted from the AIS 405 of the ship 404 displayed as the ship image is held as a value. Specifically, for example, when there is AIS data corresponding to the ship image data by mapping processing of AIS data and ship image data, “normal” is stored, and there is no corresponding AIS data. Stores “missing”. In the case of default, “NULL” is held in the data state 709.

  FIG. 8 is an explanatory diagram showing an example of the data structure of the AIS data 523. The AIS data 523 is stored in the AIS data DB 502, read by the central processing unit 505, and written in the data memory 508. After the update, the data is stored again in the AIS data DB 502.

  The AIS data 523 includes a ship ID 800, an IMO number 801, a ship name 802, a length 803, a width 804, pixel data 805, time series data 806, and a data state 807. This information is retained in the form. The IMO number 801, the ship name 802, the length 803, the width 804, the pixel data 805, and the time series data 806 are data that are manually input.

  The ship ID 800 holds a value uniquely assigned to the AIS data 523. The IMO number 801 holds a ship 404 identification number assigned to a large ship by the International Maritime Organization as a value. The ship name 802 holds the name of the ship registered in the IMO as a value. The length 803 and the width 804 hold the size of the ship 404 registered in the IMO as values. In the pixel data 805, RGB color model values are held on a two-dimensional array in accordance with the size of the ship 404 photograph. Pixel data 805 is displayed on the display device as a ship image.

  The time series data 806 holds information such as the position and speed of the ship 404 for each time. Details of the time-series data 806 will be described later with reference to FIG. In the data state 807, identification information indicating whether there is an erroneous input in the IMO number or size of the AIS data is held as a value. When the IMO number 801 in the AIS data is abnormal, “IMO number abnormal” is stored, and when the length 803 and the width 804 are abnormal, “size abnormal” is stored. If there is a ship image corresponding to the AIS data by the mapping process between the AIS data and the ship image, “normal” is stored in the data state 807. In the case of default, “NULL” is held in the data state 807.

  FIG. 9 is an explanatory diagram showing an example of the data structure of the time series data shown in FIG. The time series data 806 includes a time series ID 901, a reception time 902, a longitude 903, a latitude 904, a speed 905, and a data state 906, and these pieces of information are held in an array format, for example. The time series ID 901 holds a value uniquely assigned to the time series data 806. The reception time 902 holds the time when the time series data is received as a value. The longitude 903 and the latitude 904 hold the coordinate information of the ship 404 when time series data is transmitted as values. The speed 905 holds the speed of the ship 404 when the time series data is transmitted as a value. The data state 906 holds, as a value, identification information indicating whether time-series data is received at regular intervals. If the next time-series data is not received at regular intervals, that is, if there is a partial defect in the time series, “partial defect” is registered in the data state 906 of the time-series data before the defect is present. Is done. In the default case, NULL is held in the data state 906.

  FIG. 10 is an explanatory diagram showing an example of the data structure of the ship correspondence data 524. The ship correspondence data 524 is information indicating a correspondence relationship between AIS data and ship image data. The ship correspondence data 524 is generated on the data memory 508 and stored in the ship correspondence DB 503.

  The ship correspondence data 524 includes a ship ID 1000, a ship image ID 1001, a size correspondence score 1002, a speed correspondence score 1003, a ship type correspondence score 1004, a direction correspondence score 1005, and a longitude 1006 after shifting the preceding time series data. A latitude 1007 after the shift of the preceding time series data, a longitude 1008 after the shift of the subsequent time series data, a latitude 1009 after the shift of the subsequent time series data, a maximum search distance 1010, and a correspondence establishment flag 1011 The information is held in the form of an array, for example.

  The size-corresponding score 1002 stores a score representing the degree of size difference when the size of the vessel 404 is compared between the vessel image data and the AIS data. The speed correspondence score 1003 holds a score representing the degree of speed difference when the speed of the ship 404 is compared between the ship image data and the AIS data. The ship type correspondence score 1004 holds a score representing the degree of pixel difference when the pixel data of the ship 404 is compared between the ship image data and the AIS data.

  The direction-corresponding score 1005 holds a score representing the degree of direction difference when the direction of the ship 404 is compared between the ship image data and the AIS data. Also, normalized values are input to the size correspondence score 1002, the speed correspondence score 1003, the ship type correspondence score 1004, and the direction correspondence score 1005, and the total range of these is, for example, [0, 1]. .

  For the longitude 1006 after the shift of the preceding time series data and the latitude 1007 after the shift of the preceding time series data, the time series data (preceding time series data) before the imaging time of the satellite image is taken as the value. The longitude and latitude at the estimated shooting time when the time is shifted to the time are held. The preceding time-series data may be data observed at any time point as long as it is before the satellite image capturing time, but the longitude and latitude observed immediately before the satellite image capturing time are preferable.

  Similarly, in the longitude 1008 after the shift of the subsequent time series data and the latitude 1009 after the shift of the subsequent time series data, the time series data (following time) immediately after the preceding time series data after the imaging time of the satellite image are used as values. The longitude and latitude at the estimated shooting time when the time series is shifted to the shooting time of the satellite image is held. The subsequent time series data may be data observed at any time as long as it is after the shooting time of the satellite image, but the longitude and latitude observed immediately after the shooting time of the satellite image are preferable.

  The maximum search distance 1010 holds a distance for setting a range when searching for a corresponding ship image from the position of the ship 404 indicated by the AIS data. In the correspondence relationship establishment flag 1011, information indicating whether or not a correspondence relationship is established between the AIS data specified by the ship ID 1000 and the ship image data specified by the ship image ID 1001 is set as a value. . Here, the default is “0” (not established), and “1” when the correspondence is established. When the correspondence score in the AIS data identified by the vessel ID 1000 and the vessel image data identified by the vessel image ID 1001 having the largest correspondence score is equal to or greater than the threshold, the correspondence establishment flag is set to “1”.

<Processing contents of each processing unit 511-514>
Next, the filtering processing unit 511, the mapping processing unit 512, the complement processing unit 513, and the data display processing unit 514 illustrated in FIG. 2 will be described.

  The filtering processing unit 511 executes a filtering process for filtering in advance the AIS data group in the AIS data DB 502 for abnormality or partial missing AIS data of manually input data such as IMO number and size. Specifically, for example, the filtering processing unit 511 writes the AIS data in the AIS data DB 502 to the data memory 508, and determines whether or not each AIS data is abnormal. When there is an abnormality in the IMO number, “IMO number abnormality” is set in the data state of the AIS data, and when there is an abnormality in the size of the ship 404, “size abnormality” is set in the data state of the AIS data. This makes it possible to specify which AIS data has been erroneously input.

  In addition, the filtering processing unit 511 sets “partially missing” in the data state of the time-series data in the AIS data, if applicable. For example, when time series data is not received at regular intervals, the time series data is missing. Therefore, the filtering processing unit 511 sets “partially missing” in the data state of the time-series data immediately before the loss. Thereby, it is possible to specify at which point in which time-series data group there is a defect.

  The mapping processing unit 512 performs a mapping process with the ship image data on the AIS data determined to be abnormal and partially missing by the filtering processing unit 511, and converts the AIS data into three types of normal data, abnormal data, and missing data. Classify. Normal data is AIS data in which corresponding ship image data exists. The abnormal data is AIS data in which a corresponding ship image does not exist or is input incorrectly such as an IMO number or size. The missing data is AIS data in which time-series data is lost with respect to ship image data for which a correspondence relationship cannot be formed, or partial missing AIS data in which time-series data is not received at a fixed time interval. If it is normal data, as shown in FIG. 1, since it is matched with ship image data, it is possible to refer to information mutually between AIS data and satellite image data.

  The complement processing unit 513 executes the mapping process again for the ship image data that has no correspondence with the abnormal data or the missing data. Specifically, for example, the complement processing unit 513 performs the mapping process again using the abnormal data classified by the filtering processing unit 511 and the mapping processing unit 512 and the ship image data in which AIS data is lost, and the ship image Complement the abnormal and missing AIS data based on the data.

  If the IMO number cannot be extracted from the ship image data, the complement processing unit 513 removes the AIS data “IMO number abnormality” again at the time of the mapping process. Further, when calculating the correspondence score between the AIS data and the ship image data, the correspondence standard is determined by the abnormality of the AIS data. For example, in the AIS data with abnormal size, the complement processing unit 513 first calculates the correspondence score by excluding the size as the correspondence criterion and arbitrarily combining the remaining correspondence criteria. As a result, the abnormality / missing data can be complemented, and the mutual reference accuracy between the ship image data and the AIS data on the satellite image is improved.

  The data display processing unit 514 displays a satellite image reflecting the mapping result by the complement processing unit 513 on the display device. For example, the data display processing unit 514 displays a combination of ship image data and AIS data having a correspondence relationship by referring to the ship correspondence data 524. In this embodiment, the mapping result is displayed by the data display processing unit 514. However, the mapping result may be output, transmitted, or stored in an external device.

<Example of complementary processing procedure>
FIG. 11 is a flowchart illustrating an example of a detailed processing procedure performed by the data complementing apparatus 408. The data complementing device 408 performs the filtering process described above by the filtering processing unit 511 (step S801), and stores one or more satellite image groups of the satellite image DB 500 in the data memory 508 (step S1102). The filtering process (step S1101) avoids the error of manually input data such as IMO number and size, or the partial missing AIS data that was not received at regular intervals, erroneously linked with unrelated ship image data. It is a process to do. Details of the filtering process (step S1101) will be described with reference to FIG.

  Next, the data complementing device 408 acquires one satellite image p from the satellite image group on the data memory 508 (step S1103). Thereafter, the data complementing device 408 identifies the ship image from the satellite image p, reads the ship image data 522 including the identified ship image from the ship image DB, and stores it in the data memory 508 (step S1104). Further, the data complementing device 408 includes an AIS including time series data immediately before and after the shooting time of the satellite image p from the AIS data DB 502 storing the AIS data group that has been subjected to the filtering process (step S1101). The data 523 is read and stored in the data memory 508 (step S1105). Then, the data complementing apparatus 408 performs mapping processing between the AIS data 523 and the ship image data 522 on the satellite image p by the mapping processing unit 512 (step S1106).

  In the mapping process (step S1106), the AIS data that is not abnormal or partially missing by the filtering process (step S1101) is mapped with each ship image reflected in one satellite image, and the AIS data is classified into normal data or abnormal data. In this process, the ship image data is classified into normal data or missing data. Details of the mapping process (step S1106) will be described later in detail with reference to FIGS.

  Next, the data complementing device 408 executes the mapping process again by the complementing processing unit 513 (step S1107). Specifically, for example, the complement processing unit 513 obtains an estimated position of the ship 404 at the time of capturing the satellite image based on the abnormal data, performs a mapping process with the missing data, and complements the data. The remapping process (step S1107) will be described later with reference to FIG.

  Next, the data complementing device 408 writes the data in the data memory 508 to the DB (step S1108), and checks whether all satellite images in the satellite image data 521 have been processed (step S1109). If all satellite images have not been processed in step S1109 (step S1109: NO), the process returns to step S1103 to process the remaining satellite images. In step S1109, when all the satellite images have been processed (step S1109: Yes), the data complementing device 409 executes data display processing by the data display processing unit 514 (step S1110). In the data display process (step S1110), the data display processing unit 514 displays the AIS data and the ship image data in association with each other. An example of display contents of the data display process (step S1110) will be described later in detail with reference to FIGS.

<Example of filtering process>
FIG. 12 is a flowchart illustrating a detailed processing procedure example of the filtering process (step S1101) by the filtering processing unit 511 illustrated in FIG. The filtering processing unit 511 reads AIS data from the AIS data DB 502 and stores it in the data memory 508 (step S1201). Next, the filtering processing unit 511 acquires one unacquired AIS data AIS 405 from the data memory 508 (step S1202). The acquired AIS 405 is AIS data to be filtered.

  Next, the filtering processing unit 511 refers to the IMO number in the master DB 504 and determines whether the IMO number of the AIS 405 is abnormal (step S1203). Specifically, for example, the filtering processing unit 511 determines whether or not the IMO number of the AIS 405 matches the registered IMO number in the master DB 504.

  In the case of mismatch, the IMO number of the AIS 405 does not match the registered IMO number and does not exist in the master DB 504. Therefore, it is determined as abnormal. If it is determined that there is an abnormality (step S1203: Yes), the filtering processing unit 511 registers “IMO number abnormality” in the data state of the AIS 405 (step S1204), and proceeds to step S1211.

  On the other hand, when the IMO number of the AIS 405 matches the registered IMO number, the IMO number of the AIS 405 exists in the master DB 504. Therefore, it is determined as normal. When it is determined to be normal (step S1203: No), the filtering processing unit 511 refers to the information indicating the size range in the master DB 504 and determines whether the ship size of the AIS 405 is abnormal (step S1205). ). The ship size is the width and length of the ship 404.

  The master DB 504 stores information indicating a size range that serves as a reference for each of the width and the length. The filtering processing unit 511 determines that the size is abnormal when at least one of the width and the length is not within the size range, and determines that the size is normal when both the width and the length are within the size range.

  Therefore, if the filtering processing unit 511 determines that the size is abnormal (step S1205: Yes), it registers “size abnormal” in the data state of the AIS 405, and proceeds to step S1207. On the other hand, when it is determined that the size is not abnormal (step S1205: No), the process proceeds to step S1207.

  In step S1207, two consecutive time series data that have not been acquired from the time series data group of AIS405 are acquired (step S1207). Here, it is assumed that two continuous time series data to be processed are t1 and t2. t1 is the preceding time series data, and t2 is the succeeding time series data observed immediately after t1.

  The filtering processing unit 511 compares the reception time at t1 with the reception time at t2, and determines whether or not the difference in reception time is greater than or equal to a threshold value (step S1209). In other words, the filtering processing unit 511 determines whether or not time-series data is missing between t1 and t2 by checking whether t1 and t2 are received at regular time intervals.

  When the time difference between the reception times is equal to or greater than the threshold (step S1209: Yes), the filtering processing unit 511 registers “partial loss” in the data state of t1 (step S1209). On the other hand, if it is not greater than or equal to the threshold (step S1209: No), the process proceeds to step S1210. In step S1210, the filtering processing unit 511 determines whether there are two consecutive time-series data that are not acquired in the AIS 405 (step S1210).

  If there are two unacquired continuous time-series data (step S1210: Yes), the process returns to step S1207, and the filtering processing unit 511 acquires two consecutive time-series that have not been acquired from the time-series data group of the AIS405. Data is acquired (step S1207). On the other hand, when there is no unacquired continuous two time series data (step S1207: No), the process proceeds to step S1211.

  In step S1211, the filtering processing unit 511 determines whether all AIS data on the data memory 508 has been processed (step S1211). When there is unprocessed AIS data (step S1211: No), the process returns to step S1202, and the filtering processing unit 511 acquires one unacquired AIS data AIS405 from the data memory 508 (step S1202).

  On the other hand, when all the data has been processed (step S1211: Yes), the filtering processing unit 511 writes the AIS data group on the data memory 508 back to the AIS data DB 502 (step S1212). Therefore, the AIS data group in the AIS data DB 502 is the content after filtering by the filtering process (step S1101). Thereby, the filtering process (step S1101) is terminated, and the process proceeds to step S1102.

  As described above, the filtering processing unit 511 performs filtering in advance before the mapping process (step S1106) between the AIS data and the ship image data by the filtering process (step S1101). As a result, it is possible to avoid the abnormality of the manually input data such as the IMO number and the size, and the partial missing AIS data that has not been received at a fixed interval erroneously making a correspondence with the unrelated ship image data. Note that the filtering processing unit 511 may be in an external device outside the data complementing device 408. In that case, the data complementing device 408 may acquire the filtering result from the external device.

<Example of mapping process>
FIGS. 13 and 14 are flowcharts illustrating a detailed processing procedure example of the mapping process (step S1106) by the mapping processing unit 512 illustrated in FIG. The mapping processor 512 first acquires one AIS data AIS 405 from the AIS data group stored in the data memory 508 in step S1105 of FIG. 11 (step S1301).

  Next, the mapping processor 512 determines whether or not the data state of the AIS 405 acquired in step S1301 is abnormal (step S1302). Specifically, for example, when “IMO number abnormality” or “size abnormality” is registered in the data state 807 of the AIS 405, the mapping processing unit 512 determines that the AIS 405 is abnormal.

  When it is determined that there is an abnormality (step S1302: Yes), the process proceeds to step S1308. On the other hand, when it is determined that there is no abnormality (step S1302: No), time-series data received from the AIS 405 immediately before the shooting time of the satellite image p stored in the data memory 508 in step S1104 of FIG. , T1) and the time-series data received immediately after (hereinafter referred to as t2) are acquired (step S1303).

  Then, in order to avoid the coordinate value and the speed of the partially missing AIS data from being significantly different from the shooting time data and accidentally making a correspondence with an unrelated ship image, the mapping processing unit 512 performs step S1303. It is determined whether or not the data state of the acquired time series data t1 is “partially missing” (step S1304). When it is a partial defect | deletion (step S1304: Yes), it transfers to step S1308. On the other hand, if it is not a partial defect (step S1304: No), the mapping processing unit 512 determines whether the coordinate value of the ship 404 in at least one of the time series data t1 and the time series data t2 is outside the range of the satellite image p. It is determined whether or not (step S1305).

  If it is out of range (step S1305: YES), the process proceeds to step S1308. On the other hand, when it is within the range (step S1305: No), the mapping processing unit 512 executes time shift processing (step S1306). In the time shift process (step S1306), the reception times of the time series data t1 and t2 are shifted to the imaging time of the satellite image p, and the time series data t1 ′ and t2 indicating the estimated coordinate values of the ship 404 at the imaging time. This is a process for obtaining ′. Details of the time shift process (step S1306) will be described later with reference to FIG.

  After the time shift process (step S1306), the mapping processing unit 512 determines whether or not the estimated coordinate value of the ship 404 in at least one of the time series data t1 ′ or t2 ′ is outside the range of the satellite image p. Judgment is made (step S1307). If it is within the range (step S1307: No), steps S1401 to S1409 in FIG. 14 are executed, and the process proceeds to step S1308. Steps S1401 to S1409 in FIG. 14 are processes for constructing a correspondence relationship between the AIS 405 and the ship image data. On the other hand, if it is out of the range (step S1305: Yes), the mapping processing unit 512 proceeds to step S1308 without executing steps S1401 to S1409 of FIG.

  In step S1308, the mapping processing unit 512 determines whether steps S1301 to S1307 have been executed for all AIS data on the data memory 508. When there is unprocessed AIS data (step S1308: No), the process returns to step S1301. On the other hand, if there is no unprocessed AIS data (step S1308: Yes), the mapping processing unit 512 registers “missing” in the data state of the ship image data determined to have no correspondence with the AIS 405 (step S1309). “Abnormal” is registered in the data state of the AIS 405 determined to be incompatible with the ship image data (step S1310). Thereby, the mapping process (step S1106) is terminated, and the process proceeds to step S1107 in FIG.

  As described above, in the mapping process (step S1106), the AIS data that is not abnormal or partially missing by the filtering process (step S1101) is mapped with each ship image data reflected in one satellite image, and the AIS data is converted into normal data. Or it can classify | categorize into abnormal data and can classify ship image data into normal data or missing data.

  Next, FIG. 14 will be described. FIG. 14 shows a case where it is determined in step S1307 in FIG. 13 that the estimated coordinate value of the ship 404 in at least one of the time-series data t1 ′ or t2 ′ is within the range of the satellite image p (step S1307: The process of No) is shown.

  Step S1307: In the case of No, the mapping processing unit 512 executes a maximum search distance calculation process (step S1401). The maximum search distance calculation process (step S1401) is a process for calculating the maximum search distance (hereinafter referred to as D) in the AIS data. The maximum search distance D is a distance from the straight channel L of the ship 404 connecting the estimated coordinate values of the ship 404 in the time series data t1 ′ and t2 ′. Since the coordinate value of the ship 404 in the ship image data is not necessarily in the vicinity of the coordinate value of the ship 404 of the AIS 405, the maximum search distance D from the straight route L is obtained, and the ship image within the range of the maximum search distance D is obtained. However, it is assumed that the correspondence relationship with the AIS 405 is related. Here, the relationship between the time series data t1, t2, t1 ′, t2 ′, the estimated route L, and the maximum search distance D is illustrated.

  FIG. 15 is an explanatory diagram showing the relationship between the estimated route L and the maximum search distance D. The region R is a region where the ship 404 can navigate between the coordinate value of the time series data t1 ′ and the time series data t2 ′. Details of the maximum search distance calculation process (step S1401) in FIG. 14 will be described later with reference to FIG.

  In FIG. 14, after the maximum search distance calculation process (step S1401), the mapping processing unit 512 includes the ship image data group including the ship image on the satellite image p stored in the data memory 508 in step S1104 of FIG. From this, one ship image data (hereinafter referred to as “ship”) is acquired (step S1402).

  Next, the mapping processor 512 calculates the shortest distance dist from the coordinate value of the ship to the estimated route L (step S1403). The shortest distance dist is the coordinate value of t1 ′ (longitude x1 ′, latitude y1 ′) and the coordinate value of t2 ′ (longitude x2 ′, longitude) in the geographic coordinate system on the celestial surface (longitude x, latitude y). This is the distance to the line segment (straight route L) of latitude y2 ′). The dist is calculated by the following equation (1). In addition, r of Formula (1) is the equator radius of the earth.

  Then, the mapping processing unit 512 determines whether or not the calculated dist is less than or equal to the maximum search distance D (step S1404). That is, it is determined whether or not the coordinate value of the ship exists in the region R. When dist is not less than or equal to the maximum search distance D (step S1404: No), the process proceeds to step S1406. On the other hand, when dist is equal to or smaller than the maximum search distance D (step S1405: Yes), the mapping processing unit 512 executes a corresponding score calculation process (step S1405), and proceeds to step S1406.

  The corresponding score calculation process (step S1405) is a process for calculating a corresponding score. The correspondence score is an index value indicating the probability of identity between the ship specified by the AIS data and the ship specified by the ship image data on the basis of the size, speed, ship type, and direction of the ship 404. , Whether the AIS data and the ship image data are in a correspondence relationship. In this example, it is assumed that the higher the corresponding score, the more identical the ship specified by the AIS data and the ship specified by the ship image data are, and the AIS data and the ship image data are in a correspondence relationship. Details of the corresponding score calculation process (step S1405) will be described later with reference to FIG.

  After calculating the corresponding score for “ship”, the mapping processing unit 512 determines whether or not all ship image data on the data memory 508 has been processed (step S1406). When not processing (step S1406: No), it returns to step S1402, and the mapping process part 512 acquires ship again (step S1402). On the other hand, when processed (step S1406: Yes), the mapping process part 512 acquires the ship image data (henceforth A) with the largest corresponding | compatible score from the ship image data group from which the corresponding score was calculated (step A). S1407).

  Then, the mapping processing unit 512 determines whether or not the correspondence score between ais and A is greater than or equal to a threshold value (step S1408). If it is equal to or greater than the threshold (step S1408: Yes), it is assumed that ais and A are data relating to the same ship, the mapping processing unit 512 registers “normal” in each data state of ais and A, and ais and A Is set to “1” (step S1409), and the process proceeds to step S1308 in FIG. On the other hand, if it is not equal to or greater than the threshold (step S1408: No), the process proceeds to step S1308 in FIG. Thus, ais and A are associated with each other from the data states “normal” of ais and A and the ship correspondence data in which ais and A are associated.

  In this way, by executing the mapping process (step S1106), the AIS data is classified into normal data or abnormal data for the AIS data that is not abnormal or partially lost by the filtering process (step S1101), and the ship image data is Can be classified as normal data or missing data.

<Example of time shift processing procedure>
FIG. 16 is a flowchart showing a detailed processing procedure example of the time shift processing (step S1306) shown in FIG. As described above, the time shift process (step S1306) shifts the reception times of the time series data t1 and t2 to the imaging time of the satellite image p as shown in FIG. This is processing for obtaining time series data t1 ′ and t2 ′ indicating estimated coordinate values.

  In the time shift processing (step S1306), the mapping processing unit 512 first uses the following equation (2) to calculate the coordinate values (longitude x1, latitude y1) of the time series data t1 received immediately before the shooting time 601 in the AIS data AIS405. ) To the coordinate value (longitude x2, latitude y2) of the time series data t2 received immediately after the shooting time 601 is calculated (step S1301). In equation (2), r is the equator radius of the earth. The linear distance d is shown in FIG.

  Next, the mapping processing unit 512 calculates an azimuth angle θ formed by a straight line from the coordinate value of t1 to the coordinate value of t2 using the following formula (3) (step S1302). The azimuth angle θ is shown in FIG. The azimuth angle θ is used in the following formula (4).

  Then, the mapping processing unit 512 uses the difference between the reception time 902 of t1 and the shooting time 601 and the speed of the ship 404 at t1, and the ship 404 between the reception time 902 of t1 and the shooting time 601 at the speed. The movement distance s1 in the case of moving is calculated (step S1303). The mapping processing unit 512 determines whether or not the movement distance s1 is equal to or less than the linear distance d (step S1304). When the movement distance s1 is equal to or less than the linear distance d (step S1304: Yes), time series data t1 ′ is obtained using the following equation (4) (step S1305), and the process proceeds to step S1307. Specifically, an estimated coordinate value (longitude x1 ′, latitude y1 ′) at the photographing time of the satellite image when the reception time of t1 is shifted to the photographing time is obtained. In Equation (4), r is the equator radius of the earth.

  On the other hand, when the moving distance s1 is greater than the straight line distance d in step S1304 (step S1304: No), the mapping processing unit 512 substitutes t1 for t1 ′ (step S1306), and proceeds to step S1307. That is, by setting t1 ′ and t1 to the same coordinate value, the movement by the movement distance s1 is prevented.

  In step S1307, the mapping processing unit 512 uses the difference between the reception time 410 at t2 and the imaging time 601 and the speed of the ship 404 at t2, and from the imaging time 601 to the reception time 410 at t2 at the speed. The movement distance s2 when the ship 404 has moved is calculated (step S1307). The mapping processing unit 512 determines whether or not the movement distance s2 is equal to or less than the linear distance d (step S1308). When the movement distance s2 is equal to or less than the linear distance d (step S1308: Yes), time series data t2 ′ is obtained using the following equation (5) (step S1309), and the time shift process (step S1306) is terminated. Specifically, an estimated coordinate value (longitude x2 ′, latitude y2 ′) at the imaging time of the satellite image when the reception time of t2 is shifted to the imaging time is obtained. In equation (5), r is the equator radius of the earth.

  On the other hand, in step S1308, when the moving distance s2 is greater than the linear distance d (step S1308: No), the mapping processing unit 512 substitutes t2 for t2 ′ (step S1310), and performs a time shift process (step S1306). finish. That is, by setting t2 ′ and t2 to the same coordinate value, the movement by the movement distance s2 is prevented.

  Thereby, the position of the ship 404 in the AIS data at the time of capturing the satellite image can be estimated. In particular, using the AIS data immediately before and after the shooting time makes it easy to estimate the position of the ship 404 at the shooting time even when there is an operation such as rapid acceleration or turning.

<Example of maximum search distance calculation processing procedure>
FIG. 17 is a flowchart showing a detailed processing procedure example of the maximum search distance calculation processing (step S1401) shown in FIG. The mapping processing unit 512 first receives the coordinate value (longitude x1, latitude y1) of the time series data t1 received immediately before the imaging time 601 in the AIS data AIS405 using the above formula (2) immediately after the imaging time 601. The straight line distance d to the coordinate value (longitude x2, latitude y2) of the time series data t2 is calculated (step S1401).

  Next, the mapping processing unit 512 uses the difference between the reception time 902 of t1 and the shooting time 601 and the speed of the ship 404 at t1, and the ship between the reception time 902 of t1 and the shooting time 601 at the speed. The movement distance s1 when 404 moves is calculated (step S1402). Further, the mapping processing unit 512 uses the difference between the reception time 410 at t2 and the imaging time 601 and the speed of the ship 404 at t2, and the ship 404 between the imaging time 601 and the reception time 410 at t2 at that speed. The movement distance s2 is calculated in the case of the movement (step S1403). Then, the mapping processing unit 512 calculates the maximum search distance D in the AIS 405 using the following formula (6) (step 1404). Thereby, the maximum search distance D is calculated, and the maximum search distance calculation process (step S1401) is ended.

<Corresponding score calculation processing procedure example>
FIG. 18 is a flowchart showing a detailed processing procedure example of the corresponding score calculation processing (step S1405) shown in FIG. The mapping processing unit 512 sets α, β, γ, and δ in advance as scores that can be set in the size correspondence score, speed correspondence score, ship type correspondence score, and direction correspondence score, respectively. The score, the ship type correspondence score, and the direction correspondence score are set to initialization (= 0) (step S1801). Α, β, γ, and δ are normalized scores, and the sum is 1. Further, α, β, γ, and δ may be scores calculated with weights in addition to fixed scores.

  Next, the mapping processing unit 512 determines whether the difference between the length of the ship 404 in the ship and the length 803 in the AIS 405 and the difference between the width of the ship 404 in the ship and the width 804 in the AIS 405 are within an error range. It is determined whether or not (step S1802). If any difference is within the error range (step S1802: Yes), the mapping processing unit 512 sets α to the size correspondence score (step S1803), and proceeds to step S1804. On the other hand, if at least one of the differences is outside the error range (step S1802: No), the size correspondence score remains the initial value, and the process proceeds to step S1804.

  In step S1804, the mapping processing unit 512 determines whether or not the speed difference between the speed of the ship 404 at ship and the average speed of the speed of the ship 404 at t1 and the speed of the ship 404 at t2 is within an error range. Is determined (step S1804). If it is within the error range (step S1804: Yes), the mapping processing unit 512 sets β in the speed correspondence score (step S1805), and proceeds to step S1806. On the other hand, if it is out of the error range (step S1804: No), the speed correspondence score remains the initial value, and the process proceeds to step S1806.

  Next, in step S1806, the mapping processing unit 512 determines whether or not the difference between the ship ship type and the AIS405 ship type is within an error range (step S1806). Specifically, the mapping processing unit 512 compares the ship pixel data 708 and the AIS 405 pixel data 805 to determine whether or not the pixel error is within the error range. If it is within the error range (step S1806: Yes), the mapping processing unit 512 sets γ for the ship type corresponding score (step S1807), and proceeds to step S1808. On the other hand, when it is out of the error range (step S1806: No), the ship type correspondence score remains the initial value, and the process proceeds to step S1808.

  After that, the mapping processing unit 512 calculates the azimuth angle θ1 of the ship coordinate value (longitude x, latitude y) from the coordinate value t1 (longitude x1, latitude y1) using the following formula (7) (step) 1508).

  Next, the mapping processing unit 512 calculates the azimuth angle θ2 of the coordinate value of t2 (longitude x2, latitude y2) from the coordinate value of ship (longitude x, latitude y) using the following equation (8) (step) 1509).

  Then, the mapping processing unit 512 determines whether or not the difference between the ship direction 705 and the azimuth angles θ1 and θ2 is within an error range (step S1810). Specifically, the mapping processing unit 512 determines whether or not the difference between the shipping direction 705 and the azimuth angle θ1 is within an allowable range, and the difference between the shipping direction 705 and the azimuth angle θ2 is within the allowable range. It is determined whether or not.

  FIG. 19 is an explanatory diagram showing the relationship between the ship direction 705 and the azimuth angles θ1 and θ2. Returning to FIG. 18, when any one of the differences is within the error range (step S1810: Yes), the mapping processing unit 512 sets δ to the direction correspondence score between AIS405 and ship (step S1811), The process moves to S1812. On the other hand, if any difference is out of the error range (step S1810: No), the direction-corresponding score remains the initial value, and the process proceeds to step S1812.

  In step S1812, the mapping processing unit 512 determines the vessel 404ID of AIS405, the vessel image ID of ship, the size correspondence score, the velocity correspondence score, the vessel type correspondence score, the direction correspondence score, the coordinate values of t1 ′ and t2 ′, and the maximum search. The distance D is stored in the data memory 508 as shown in FIG. 10 (step S1812).

  Thereafter, the mapping processing unit 512 calculates a total score of the size correspondence score, the speed correspondence score, the ship type correspondence score, and the direction correspondence score between the AIS 405 and the ship (step 1513). This total score becomes the corresponding score output by the corresponding score calculation process (step S1405). Thereby, the corresponding score calculation process (step S1405) is terminated. The corresponding score is calculated for each ship image data, and the maximum corresponding score is selected in step S1407.

<Example of remapping process>
20 and 21 are flowcharts showing a detailed processing procedure example of the remapping processing (step S1107) by the complement processing unit 513 shown in FIG. First, the complement processing unit 513 acquires one AIS data AIS 405 from the AIS data group stored in the data memory 508 in step S1105 of FIG. 11 and subjected to mapping processing (step S1106) (step S2001). ).

  Next, the complement processing unit 513 determines whether or not the data state of the AIS 405 acquired in step S2001 is abnormal (step S2002). Specifically, for example, when “IMO number abnormality” or “size abnormality” is registered in the data state 807 of the AIS 405, the complement processing unit 513 determines that the AIS 405 is abnormal.

  When it is determined that there is no abnormality (step S2002: No), the process proceeds to step S2007. On the other hand, if it is determined that there is an abnormality (step S2002: Yes), the complement processing unit 513 immediately before the shooting time of the satellite image p stored in the data memory 508 in step S1104 of FIG. Received time-series data (hereinafter referred to as t1) and time-series data received immediately thereafter (hereinafter referred to as t2) are acquired (step S2003).

  Then, the complement processing unit 513 determines whether or not the coordinate value of the ship 404 in at least one of the time series data t1 and the time series data t2 is outside the range of the satellite image p (step S2004). If it is out of range (step S2004: Yes), the process proceeds to step S2007. On the other hand, when it is within the range (step S2004: No), the complement processing unit 513 executes time shift processing (step S2005). The time shift process (step S2005) is the same process as the time shift process (step S1306) shown in FIG.

  After the time shift process (step S2005), the complement processing unit 513 determines whether or not the estimated coordinate value of the ship 404 in at least one of the time series data t1 ′ or t2 ′ is outside the range of the satellite image p. Judgment is made (step S2006). If it is within the range (step S2006: No), steps S2101 to S2110 in FIG. 21 are executed, and the process proceeds to step S1408. Steps S2101 to S2110 in FIG. 21 are processes for complementing the correspondence between the AIS 405 and the ship image data. On the other hand, if it is out of range (step S2006: Yes), the complement processing unit 513 proceeds to step S2007 without executing steps S2101 to S2110 of FIG.

  In step S2007, the complement processing unit 513 determines whether steps S2001 to S2006 have been executed for all AIS data on the data memory 508. If all of the AIS data on the data memory 508 has not been processed (step S2007: No), the process returns to step S2001. On the other hand, when all the processing has been performed (step S2007: Yes), the complementing processing unit 513 supplements the remaining ship image data and generates new AIS data (step S2008).

  Specifically, for example, as illustrated in FIG. 3, the complement processing unit 513 generates AIS data from the remaining ship image data. The complement processing unit 513 updates the data state of the remaining ship image data from “missing” to “normal” and sets the data state of the newly generated AIS data to “normal”. Then, the complementary processing unit 513 sets a correspondence establishment flag in the ship correspondence data 524 between the remaining ship image data and the newly generated AIS data to “1”. Thus, the remapping process (step S1107) is terminated. A specific example of complement will be described later.

  Next, FIG. 21 will be described. FIG. 21 shows a case where it is determined in step S2006 of FIG. 20 that the estimated coordinate value of the ship 404 in at least one of the time-series data t1 ′ or t2 ′ is within the range of the satellite image p (step S2006: The process of No) is shown.

  Step S2006: In the case of No, the complement processing unit 513 executes a maximum search distance calculation process (step S2101). The maximum search distance calculation process (step S2101) is the same process as the maximum search distance calculation process (step S1401), and thus description thereof is omitted.

  After the maximum search distance calculation process (step S2101), the complement processing unit 513 stores the ship image data including the ship image on the satellite image p stored in the data memory 508 after the mapping process (step S1106). One ship image data (hereinafter referred to as “ship”) is acquired (step S2102).

  Next, the complement processing unit 513 determines whether or not the acquired data state of the ship is “missing” (step S2103). If it is not “missing” (step S2103: No), since “ship” is not a complement target, the process proceeds to step S2107. On the other hand, if it is “missing” (step S2103: Yes), since the corresponding AIS data is missing, the complement processing unit 513, like step S1403, the shortest distance dist from the coordinate value of the ship to the estimated route L. Is calculated (step S2104).

  Then, the complement processing unit 513 determines whether or not the calculated dist is equal to or less than the maximum search distance D (step S2105). That is, it is determined whether or not the coordinate value of the ship exists in the region R. If dist is not less than or equal to the maximum search distance D (step S2105: No), the process proceeds to step S2107. On the other hand, when dist is equal to or smaller than the maximum search distance D (step S2107: Yes), the complement processing unit 513 executes a corresponding score calculation process (step S2106), and proceeds to step S2107. Since the corresponding score calculation process (step S2106) is the same process as the corresponding score calculation process (step S1405), description thereof is omitted.

  After calculating the corresponding score for the ship, the complement processing unit 513 determines whether all ship image data on the data memory 508 has been processed (step S1406). If not processed (step S2107: No), the process returns to step S2102, and the complementing processing unit 513 acquires the shape again (step S2102). On the other hand, when processed (step S2107: Yes), the complement processing unit 513 acquires the ship image data (hereinafter referred to as A) having the maximum corresponding score from the ship image data group for which the corresponding score is calculated (step A). S2108).

  Then, the complement processing unit 513 determines whether or not the correspondence score between ais and A is greater than or equal to a threshold value (step S2109). If it is equal to or greater than the threshold (step S2109: Yes), the complement processing unit 513 supplements ais that is abnormal AIS data using A (step S2110), and proceeds to step S2007 in FIG. Specifically, for example, the complement processing unit 513 updates the data state of ais from “abnormal” to “normal” and updates the data state of A from “missing” to “normal”. Then, the complement processing unit 513 sets a correspondence establishment flag in the ship correspondence data 524 between ais and A to “1”.

  On the other hand, if it is not equal to or greater than the threshold (step S2109: No), the process proceeds to step S2007 in FIG. Thus, by executing the remapping process (step S1107), the abnormal AIS data can be complemented and converted to normal AIS data, and the missing ship image data can be complemented to obtain the missing ship image. It can be converted into normal AIS data that can be associated with data.

<Abnormal / Deficit Compensation Example>
FIG. 22 is an explanatory diagram of an example of abnormality / deletion complementation. In FIG. 22, in order to simplify the description, only the length is used as the size. (A) shows the correspondence between ship image data and AIS data after the mapping process (step S1106). In addition, the code | symbol of ship image data and AIS data shall be each ID. Since the ship image data A1 and the AIS data a1 have the same length and speed, there is a correspondence relationship. Accordingly, in the ship correspondence data 524 of FIG. 7, the ship image ID: A1 and the ship 404ID: a1 are registered in association with each other. The length (0 m) of the AIS data a2 is erroneous input data. Further, the ship image data A2 to A5 are “missing” because there is no AIS data that can be associated.

  (B) shows an abnormality complementing process in the complementing process (step S1107). The abnormality complement process is a process indicated by steps S2101 to S2110 in FIG. Assume that the ship image data A2 is data of the highest correspondence score among the ship image data A2 to A5 with respect to the AIS data a2. Comparing the ship image data A2 and the AIS data a2, since the length of the ship 404 in the AIS data a2 is abnormal, the complementary processing unit 513 determines the length (101 m) of the ship 404 in the ship image data A2 as the AIS data. Complement to a2. Thereby, the complement processing unit 513 can associate the ship image data A2 with the AIS data a2.

  (C) shows the ship image data A3 to A5 not complemented in (B). The ship image data A3 to A5 do not have AIS data that can establish a correspondence relationship. Accordingly, the complement processing unit 513 also applies the ship image data A3 to A5 to the AIS data in step S2008. Specifically, for example, the ship image data A3 to A5 are given ship 404ID: a3 to a5 so that they can be handled as AIS data a3 to a5. Thereby, the complement process part 513 can match ship image data A3-A5 and AIS data a3-a5.

<Example of display screen>
FIG. 23 is an explanatory diagram showing an example of an output screen of the complementary result of AIS data output by the data display processing unit 514 of FIG. The screen in FIG. 23 displays the AIS data and the AIS data results after complementing the ship image on the satellite image in one satellite image, and comprehensively confirms the AIS data for each of normal, abnormal, and defect. can do. In the ship ID 2300, the ID of the AIS data or the ID of the ship image on the complemented satellite image is displayed. When each ship ID is clicked, a screen showing the correspondence between AIS data and the ship image is displayed. Details will be described later with reference to FIGS. 24 and 25.

  In size 2301, the size in AIS data is generally displayed. The size of the corresponding ship image is displayed in the abnormal size AIS data. In the reception time 2302, the reception time in the AIS data is generally displayed. In addition, when complementing the ship image on the satellite image from which AIS data is missing, the imaging time of the satellite image is displayed. In the coordinate value 2303, the speed 2304, and the direction 2305, the coordinate values (903 and 904 in FIG. 9), the speed 905, and the direction of the calculated AIS data are displayed. In the missing AIS data, the position, speed, and direction of the ship image are displayed. A state 2306 is an item for identifying normality, abnormality, or loss of AIS data, and a data state 906 of the AIS data as a result calculated in the remapping process (step 1107) is displayed.

  24 and 25 are explanatory diagrams showing examples of AIS data and ship image output screens output by the data display processing unit 514 of FIG. Correspondence between normal, abnormal, and missing AIS data and a ship image is displayed on one satellite image. A check box 2400 for selecting a normal data type, a check box 2401 for selecting a missing data type, a check box 2402 for selecting an abnormal data type, a message box 2403 for displaying correspondence information for normal data, and a missing data A message box 2404 for displaying data type identification information, a message box 2405 for displaying data type identification information for abnormal data, and a text 2406 for displaying satellite image information are provided.

  The check boxes 2400 to 2402 are for the user to select which data, normal, abnormal, or missing, is displayed. For example, FIG. 25 shows an example in which only missing and abnormal are selected. The message boxes 2403 and 2404 are for displaying the attribute information of AIS data. For example, when the AIS data is normal, the attribute information such as the ship name and size of the message box 2403 is displayed. If the AIS data is partially missing, a display indicating a partially missing state is displayed as in a message box 2404. Such a display allows the user to check the state of the AIS data and the supplemented information on the satellite image.

  FIG. 26 is an explanatory diagram showing an example of a data display drawing output by the data display processing unit 514 of FIG. In one ship 404 that has transmitted AIS data, the corresponding ship image data A1 is searched and displayed. Points 2600 and 2601 indicating coordinate values after time shift in the time series of the AIS data a1 received immediately before and after the shooting time, an area 2602 for searching the corresponding ship image data A1, size, speed And a message box 2603 for displaying the corresponding scores of the AIS data a1 and the ship image data A1 for each correspondence standard such as ship type and direction.

  Points 2600 and 2601 are data for displaying the position of the ship 404 estimated from the time series of the AIS data a1 at the photographing time. For example, it can be confirmed whether the corresponding ship image data A1 is separated from the AIS data a1. The area 2602 is displayed with reference to the maximum search distance D for searching for the corresponding ship image data A1, and for example, it is possible to check the presence or absence of the ship image corresponding to the AIS data a1. The message box 2603 displays the size correspondence score, the speed correspondence score, the ship type correspondence score, and the direction correspondence score in the AIS data a1 and the ship image data A1, for example, the sizes of the AIS data a1 and the ship image data A1. Is not compatible, the size-corresponding score is 0, indicating that the size does not correspond. By such display, the user can check the correspondence standard and the search range between the AIS data a1 and the ship image data A1.

  In this embodiment, an example of data display at one time point is shown in FIG. 23, but in addition to this, data at a past time point can also be displayed together. That is, it is possible to add a display to that effect on AIS data that is normal at the current time but has been lost in the past. Thereby, the user can confirm more detailed information about the correspondence status between the AIS data and the ship image.

  As described above, according to the embodiment of the present invention, the data complementing apparatus filters abnormalities or partial defects in manual input by the operator as AIS data as support information for complementing abnormalities or defects in AIS data. Next, the data complementing device acquires AIS data at two points before and after the imaging time of the satellite image, estimates the position of the ship at the imaging time, determines the maximum search distance, and determines the length, speed, ship type, and direction. By performing scoring using these four criteria, the ship on the satellite image and the AIS data are mapped. Then, the data complementing device extracts normal data, complements the attribute value as abnormal data by re-mapping AIS data having abnormality or partial defect and the ship image having no correspondence after the mapping process, and twice Is complemented as missing data using the attribute information of the ship image that has no correspondence after the mapping process.

  In this way, the position of the ship is estimated at the shooting time based on the AIS data not received immediately before and after the shooting time of the satellite image data, and the maximum search distance for searching the corresponding ship image is calculated. Thus, it becomes possible to perform mapping between the ship image and AIS data with higher accuracy. By using such a mapping technique, it is possible to give attribute information to a moving object shown in a satellite image, and it is possible to refer to the satellite image and the moving object information mutually.

  Although the present invention has been described in detail with reference to the accompanying drawings, the present invention is not limited to such specific configurations, and various modifications and equivalents within the spirit of the appended claims Includes configuration.

500 Satellite image DB
501 Ship Image DB
502 AIS data DB
503 DB for ships
504 Master DB
511 Filtering processing unit 512 Mapping processing unit 513 Complement processing unit 514 Data display processing unit

Claims (12)

From the first database, first observation data including first feature data specifying the first moving body and its behavior at observation times before and after the imaging time of the area image data obtained by imaging the area is acquired, and the first A filtering processing unit that determines whether or not the value of the feature data of 1 is abnormal, and sets abnormal information indicating abnormality in the first feature data if abnormal,
From the second database, a moving object image data and a second feature data specifying the image and the second moving body and its behavior of the second movable body that moves the area on the area image data Based on the first observation data including the first feature data that is acquired and the abnormality information is not set by the filtering processing unit, and the moving body image data , the first moving body The first information indicating that there is a correspondence between the moving object image data and the first feature data when it is determined that there is an identity. When it is determined that there is no identity, or when the abnormality information is set in the first feature data, second information indicating that the mobile object image data is not associated is displayed in the first information. Features Set data, the mapping processing unit for setting a third information indicating that the first characteristic data is not associated with the moving object image data,
The first feature data in which the abnormality information is set by the filtering processing unit and the second information is set by the mapping processing unit , and the moving body image data in which the third information is set Based on the above, the identity of the first moving body and the second moving body is determined, and when it is determined that there is identity or when the mapping processing unit determines that there is identity, When it is determined that there is no identity with the first complement processing that complements the first feature data so as to be the same as the second feature data, or when it is determined that there is no identity in the mapping processing unit A complement processing unit that executes at least the first complement processing among the second complement processing that generates second observation data based on the second feature data;
Have a, and an output processing unit that outputs a result of processing by the mapping processing unit or the complementary processing unit,
At least one of the mapping processing unit and the complementing processing unit may determine the identity at the preceding observation time that is an observation time before the imaging time in the first feature data. The position and speed of one moving body, and the position of the first moving body at the preceding observation time and the subsequent observation time that is the observation time after the imaging time in the first feature data. A first estimated position of the first moving body at the photographing time is calculated based on an azimuth indicating a moving direction to the position of the first moving body, and the first estimated position at the subsequent observation time is calculated. Based on the position, speed, and azimuth of the moving body, a second estimated position of the first moving body at the photographing time is calculated, and the first estimated position and the second estimated position are calculated. Are both positions within the region image data. , Calculating a corresponding score indicating the probability of identity between the first moving body and the second moving body, and based on the calculated corresponding score, the first moving body and the second moving body A data complementing device characterized by determining identity with a body . The data complementing device according to claim 1,
The filtering processing unit determines whether or not it is abnormal depending on whether or not the identification information for identifying the first moving body in the first feature data is registered identification information. data interpolation apparatus according to. The data complementing device according to claim 1,
The filtering processing unit determines whether or not it is abnormal depending on whether or not information indicating a size of the first moving body in the first feature data is within a reference range. Data complementer. The data complementing device according to claim 1,
The filtering processing unit determines whether or not the value of the first observation data is partially missing, and when the value is partially missing, abnormal information indicating that the value is partially missing A data complementing device characterized in that it is set to one feature data . The data complementing device according to claim 4,
The filtering processing unit is partially missing depending on whether or not an interval between a preceding observation time that is an observation time before the imaging time and a subsequent observation time that is an observation time after the imaging time is a predetermined interval or more. A data complementing device characterized in that it is determined whether or not . The data complementing device according to claim 1,
The at least one processing unit is based on a difference between the size of the first moving body in the first feature data and the size of the second moving body in the second feature data. And calculating the corresponding score . The data complementing device according to claim 1,
The at least one processing unit is based on a difference between a speed of the first moving body in the first feature data and a speed of the second moving body in the second feature data. And calculating the corresponding score . The data complementing device according to claim 1,
The at least one processing unit is based on a difference between the image of the first moving body in the first feature data and the image of the second moving body in the second feature data. And calculating the corresponding score . The data complementing device according to claim 1,
The at least one processing unit is configured such that the second moving data at the photographing time in the second characteristic data is determined from the position of the first moving body at the preceding observation time in the first characteristic data. The first azimuth angle indicating the moving direction to the position of the moving body, and the position of the second moving body at the photographing time in the second feature data, the first feature data in the first feature data. The data complementing device , wherein the corresponding score is calculated based on a difference from a second azimuth indicating a moving direction to the position of the first moving body at a subsequent observation time . The data complementing device according to claim 1,
The at least one processing unit includes:
The distance from the position of the first moving body at the preceding observation time to the position of the first moving body at the subsequent observation time, and the first moving body from the preceding observation time to the photographing time And a maximum search distance indicating a search range of the second moving body based on the moving distance of the first moving body from the photographing time to the subsequent observation time,
Calculating a separation distance of the second moving body from a moving path connecting the first estimated position and the second estimated position to the position of the second moving body at the photographing time;
The data complementing apparatus , wherein the correspondence score is calculated when the separation distance is equal to or less than the maximum search distance . A data complementing method by a data complementing device having a processor for executing a program and a memory for storing a program to be executed by the processor,
The processor is
From the first database, first observation data including first feature data specifying the first moving body and its behavior at observation times before and after the imaging time of the area image data obtained by imaging the area is acquired, and the first A filtering process for determining whether or not the value of the feature data of 1 is abnormal, and setting abnormal information indicating abnormal in the first characteristic data if abnormal,
Moving body image data including, from a second database, an image of a second moving body that moves within the area on the area image data, and second feature data that identifies the second moving body and its behavior; Based on the first observation data including the first feature data that is acquired and the abnormality information is not set by the filtering process, and the moving body image data, The identity with the second moving object is determined, and when it is determined that there is an identity, first information indicating that the moving object image data and the first feature data have a corresponding relationship is set. When it is determined that there is no identity, or when the abnormality information is set in the first feature data, second information indicating that the mobile object image data is not associated is displayed as the first feature. Day A mapping processing set, sets the third information indicating that the first characteristic data is not associated with the moving object image data to,
The first feature data in which the abnormality information is set by the filtering process and the second information is set by the mapping process, and the moving body image data in which the third information is set And determining the identity between the first moving body and the second moving body, and determining that there is an identity or when determining that there is an identity in the mapping process. The first complement processing for complementing the first feature data so as to be the same as the feature data, and the second complement when it is determined that there is no identity or when the mapping processing determines that there is no identity Of the second supplement processing for generating the second observation data based on the feature data, at least the supplement processing for executing the first supplement processing;
An output process for outputting a processing result by the mapping process or the complement process; and
In at least one of the mapping process and the complement process, the processor
In the determination of the identity, the position and speed of the first moving body at the preceding observation time that is the observation time before the photographing time in the first feature data, and the previous observation time. Based on the azimuth indicating the moving direction from the position of the first moving body to the position of the first moving body at the subsequent observation time that is the observation time after the photographing time in the first feature data. The first estimated position of the first moving body at the shooting time is calculated, and the shooting is performed based on the position, speed, and azimuth of the first moving body at the subsequent observation time. When the second estimated position of the first moving body at the time is calculated, and both of the first estimated position and the second estimated position are positions in the area image data, the first estimated position Correspondence measure indicating the certainty of identity between the moving body and the second moving body Calculating the A, on the basis of the calculated corresponding score, data interpolation method comprising determining the identity of the second mobile body and the first moving body. A data complementing program executed by a processor in a data complementing apparatus accessible to a first database and a second database,
In the processor,
From the first database, obtaining first observation data including first feature data for identifying a first moving body and its behavior at observation times before and after the imaging time of the area image data obtained by imaging the area; A filtering process for determining whether or not the value of the first feature data is abnormal, and setting abnormal information indicating abnormal in the first feature data when abnormal;
Moving body image data including an image of a second moving body that moves in the area on the area image data, and second feature data that specifies the second moving body and its behavior, from the second database. And the first moving body based on the first observation data including the first feature data in which the abnormality information is not set by the filtering process and the moving body image data. The first information indicating that there is a correspondence between the moving object image data and the first feature data when it is determined that there is an identity. When it is determined that there is no identity, or when the abnormality information is set in the first feature data, second information indicating that the mobile object image data is not associated is displayed in the first information. Feature A mapping processing is set to over data, sets the third information indicating that the first characteristic data is not associated with the moving object image data,
The first feature data in which the abnormality information is set by the filtering process and the second information is set by the mapping process, and the moving body image data in which the third information is set And determining the identity between the first moving body and the second moving body, and determining that there is an identity or when determining that there is an identity in the mapping process. The first complement processing for complementing the first feature data so as to be the same as the feature data, and the second complement when it is determined that there is no identity or when the mapping processing determines that there is no identity Of the second supplement processing for generating the second observation data based on the feature data, at least the supplement processing for executing the first supplement processing;
An output process for outputting a processing result by the mapping process or the complement process, and
In at least one of the mapping process and the complement process, the processor
  In the determination of the identity, the position and speed of the first moving body at the preceding observation time that is the observation time before the photographing time in the first feature data, and the previous observation time. Based on the azimuth indicating the moving direction from the position of the first moving body to the position of the first moving body at the subsequent observation time that is the observation time after the photographing time in the first feature data. The first estimated position of the first moving body at the shooting time is calculated, and the shooting is performed based on the position, speed, and azimuth of the first moving body at the subsequent observation time. When the second estimated position of the first moving body at the time is calculated, and both of the first estimated position and the second estimated position are positions in the area image data, the first estimated position Correspondence measure indicating the certainty of identity between the moving body and the second moving body Calculating the A, on the basis of the calculated corresponding score, data interpolation program characterized by executing the process of determining the identity of said first mobile body and the second moving body.