JP3496567B2 - Photographed image matching device, map updating device, and map creating device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a captured image matching apparatus capable of matching the captured image with higher imaging resolution accuracy, also relates to MAP update device.

[0002]

2. Description of the Related Art It has been well known that an aerial photograph is used to create a map. In the map creation, a photographed image photographed from the air is prepared, the change of the road or the building is examined using the photographed image, and the obtained change is reflected on the map. Furthermore, it has been proposed to use a satellite image taken by an artificial satellite instead of the aerial image. This kind of map creation is described, for example, in Japanese Patent Laid-Open Nos. 5-165402 and 6-230767. In addition, Japanese Patent Application No. 9-357611 filed by the applicant.
A suitable map making technique is also proposed in the issue.

In recent years, the use of maps has been expanding, and a typical one is a navigation device for vehicles. Various navigation devices having higher functions have been proposed, and for example, it has been proposed to inform the driver of the presence of a front curve and the lane to be traveled. Further, it has been proposed to apply a navigation technique to control the vehicle according to the road gradient and the curvature. Also, I
Various new functions related to TS (Intelligent Transport Systems) have been proposed.

The navigation device must be (i) up-to-date with respect to the map data in order to realize its high function,
And (ii) require high accuracy. In this respect, satellite photography is not restricted by flight plans and flight permission areas like aerial photography, and thus can provide images for map creation with high frequency. However, satellite imaging has the disadvantage of relatively low resolution. It is hoped that this disadvantage will be overcome and that the maps will always be up-to-date and highly accurate.

Here, when a map is created, especially when a map is updated, the amount of change is obtained by comparing the photographed image with past images and maps. Also, a plurality of captured images are connected to obtain a wide range of images. In these processes, matching (positioning) between captured images is required.

In the conventional ordinary matching, the correlation between two images is obtained from the difference in pixel value between the images by the method of least squares or the like, and the position where the two images best match is obtained.
Conventionally, matching is done only in resolution units. That is, when viewed at the pixel level corresponding to the resolution, a certain pixel in one image is always superposed on one of the pixels in the other image. If the resolution is sufficiently higher than that of the object to be imaged, sufficient accuracy can be obtained by such matching in resolution units.

[0007]

However, in the case of an aerial imaged image, particularly a satellite image, although the resolution has improved due to recent technological advances, the resolution is lower than that of the object to be imaged. In resolution-based matching, the displacement between two superposed captured images is also relatively large, which leads to a decrease in map accuracy. Therefore, it is desired to perform matching without deviation at a level smaller than the resolution.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an apparatus capable of matching captured images with higher accuracy than resolution.

Another object of the present invention is to provide an apparatus for applying the above method to update a map by image matching.

In order to improve the matching accuracy, it is effective to specify the shooting position with high accuracy. If high-accuracy position identification is possible, a high-precision map can be created not only when updating the map but also when recreating the entire map. In view of this point, another object of the present invention is to provide a map creation device capable of creating a highly accurate map.

[0011]

(1) In order to achieve the above object, a photographed image matching apparatus of the present invention photographs a ground from the air with a predetermined photographing resolution, and a reference mark having a known shape within a photographing range. Means for acquiring a captured image including
A hand that acquires a target image to be matched with the captured image
A step , means for acquiring as reference patterns a plurality of mark displacement images that are images of reference marks that are displaced by a distance smaller than the imaging resolution, and the reference marks reflected in the captured image are the plurality of marks. By comparing with the shift image, the mark position specifying means for specifying the position of the reference mark in the photographed image with higher accuracy than the photographing resolution, and the position of the specified reference mark,
An image matching unit that matches the captured image with the target image.

As described above, according to the present invention, a captured image including the reference mark is acquired. On the other hand, as the reference pattern, a plurality of mark shift images in which the standard marks are arranged with a slight shift are prepared. The shift amount is set smaller than the resolution. Then, the mark shift image and the reference mark image in the actual photographed image are compared, and the detailed position of the reference mark is specified from the comparison result. For example, the mark shift image closest to the reference mark image is selected, and the position of the mark shift image and the actual reference mark are considered to be the same. In this way, the position where the reference mark appears in the captured image is specified with higher accuracy than the resolution. The captured image is matched with the target image using the position of the specified reference mark. As a result, matching of captured images can be performed with higher accuracy than resolution.

The target image is the image of the other party matched with the photographed image. The target image may also be a captured image, but is not limited thereto. The target image may be an image showing the detailed shape of the subject in the shooting range. In the latter case, the matching is preferably performed after the target image is mosaiced to the imaging resolution level, which facilitates the matching.

Further, the image matching may be performed after subjecting the photographed image to an appropriate deformation process, and such a process is also included in the scope of the present invention. That is, the captured image at the time of matching may be one that has undergone appropriate image processing.

Further, the above-mentioned photographed image matching device is suitable for image processing for map creation. However, the scope of application of the present invention is not limited to map creation processing.

(2) One aspect of the present invention is a device for updating a map using an image of the ground taken from the air. According to the principle of the above-mentioned photographed image matching device , this device specifies the position of the reference mark in the photographed image with high accuracy equal to or higher than the resolution. Furthermore, the captured image is compared with the comparison target image at the same location based on the specified mark position. The comparison target image is, for example, a past captured image of the same location. The map is updated based on the changed portion of the captured image obtained as a result of the comparison. According to the present invention, since the old and new captured images are accurately matched, the changed portion can be accurately known, and the accuracy of map update can be improved.

The comparison target image may be an image showing a past detailed shape of the same portion as the photographed image. Specifically, this is a map or a map-level detailed image, for example, and is a skeleton map of an embodiment described later.
In this case, preferably, the comparative image is mosaiced to the photographing resolution level prior to the comparison with the photographed image. Maps and the like are convenient for update processing in that unnecessary information that is not related to map update is mostly removed. Moreover, by making a too detailed map into a mosaic and making the resolutions uniform, it is easy to find the correlation between the images, and thus the matching can be easily performed.

(3) In the present invention, the reference mark is not limited to the specially installed mark. That is, the reference mark may be selected from existing shapes existing on the ground surface within the shooting range. In this case, the existing shape is mosaiced to obtain an image corresponding to the resolution of the captured image. A plurality of mosaic images obtained by gradually shifting the existing shape are used as mark shift images. According to this aspect, since it is not necessary to newly install the reference mark, it is possible to save the installation cost and labor of the mark.

(4) In one aspect of the present invention, the reference mark reflects, emits or absorbs light in a predetermined wavelength range at a level different from that of the ground surface and installations thereon. For example, the reference mark reflects near infrared rays as light in a predetermined wavelength range. According to this aspect, it is easy to detect the reference mark from the captured image, and thus it is easy to create a map based on the position of the reference mark.

(5) In one aspect of the present invention, the reference mark is installed on the moving body. Photographing is performed with the moving body placed at a predetermined position. The moving body is, for example, a vehicle or a ship. According to this aspect, the reference mark does not have to be fixedly installed on the ground, so that the mark installation cost and labor can be saved. In particular, it is advantageous in that the reference mark does not have to be installed in an area where the frequency of map update is low. In addition, it is possible to avoid a situation where the reference mark is covered with trees.

(6) One embodiment of the present invention is a device for creating a map using an image of the ground taken from the air. According to the principle of the above-mentioned photographed image matching device , this device specifies the position of the reference mark in the photographed image with high accuracy equal to or higher than the resolution. Further, a map is created using the captured image based on the specified mark position. For example, the position correction of the image is performed by the position correction means based on the position of the reference mark, and a map is created using the position-corrected image. According to the present invention, since the position of the reference mark is accurately specified, the image processing based on the position of the reference mark is performed with high accuracy, and thus a highly accurate map can be created. Creating a map includes creating a new map that recreates the entire map and creating an updated map as described above.

[0022] (7) In one aspect of the present invention is a photographic image matching device again, taken at a predetermined photographing resolution, means for obtaining a photographic image including the reference mark of known shape within the imaging range, the imaging range A means for acquiring the detailed shape of at least a part of the subject, and an image obtained by mosaicing the detailed shape to the imaging resolution level,
Acquiring a plurality of reference mosaic images in which the detailed shape is displaced by a distance smaller than the imaging resolution
And means, by comparing said plurality of reference mosaic image and the captured image, means for performing matching of the detailed shape and the captured image. This mode also uses the principle of the present invention, but unlike the above mode, highly accurate matching can be performed without using a reference mark.

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings. First, the image matching device of the present invention will be described. Here, image matching is applied to the map update process.

On the left side of FIG. 1, two images of the same place taken from an artificial satellite are shown. The upper new image was taken recently and the lower old image was taken in the past. Consider creating an updated map using these two images.

In order to update the map, it is necessary to detect a changed portion of both images, and therefore image matching is performed, that is, the two images are aligned and overlapped. In the present invention, this matching is performed with high accuracy as follows.

As shown in FIG. 1, in the present invention, the reference mark X
An image including is acquired. The reference mark X is set in advance at a predetermined position, and the shooting range is set so as to include the reference mark X.

On the right side of FIG. 1, a preferable example of the shape of the reference mark X is shown. The reference mark X is composed of five mark elements, and each mark element is a square having a size corresponding to the imaging resolution. The imaging resolution is the length of the subject (ground surface) corresponding to one pixel of the image sensor of the artificial satellite.
For example, if the imaging resolution is 1 m, the length of one side of the mark element is also set to 1 m.

As shown in FIG. 1, four mark elements are arranged so as to surround one mark element, and these four mark elements are located at the four corners of the square s. The four surrounding mark elements are arranged vertically and horizontally with respect to the central mark element with a distance corresponding to half the resolution. Therefore, the mark elements at the four corners have a gap of twice the resolution with the adjacent mark elements.

Next, FIGS. 2 (a) to 2 (c) show photographed images of the reference marks when the positions of the marks are slightly shifted. Cell indicated by thin line (square)
Corresponds to one pixel of the original captured image. Since one side of each mark element is set equal to the resolution,
Equal to one side of the cell.

As shown in FIG. 2A, it is assumed that the central mark element of the reference mark completely overlaps one pixel.
At this time, each remaining mark element extends over four pixels. The central pixel has the original mark color. However, the surrounding pixels are not captured as shown in FIG. 2A, but are obtained as light-colored pixels whose overall darkness is 1/4.

In terms of pixel value, in the central pixel, the color of the mark is detected at a rate of 100% (1). Mark 1
In pixels containing only / 4, the color of the mark is 25% (1/4)
Is detected at a rate of. That is, 1 of the original mark color
A pixel value corresponding to / 4 is observed.

For example, the mark element is red (single color),
The surroundings are colorless. When the red color is represented by 8 bytes (256 levels), the central red pixel value is 255 and the surrounding is 63.

Next, in FIG. 2B, the reference mark is shown at a position slightly displaced to the right with a size less than the resolution.
The amount of mark deviation is half the resolution. At this time, the mark element at the center extends over two pixels on the left and right. The remaining mark elements extend over the upper and lower two pixels. These pixels have a pixel value corresponding to 1/2 of the mark color. When the reference mark is vertically displaced by 1/2 pixel,
A state in which (b) is rotated by 90 degrees appears. As is clear from the figure, the mark element is not divided into three pixels and captured.

Next, in FIG. 2 (c), the reference mark is shown in a position displaced to the upper right. The amount of deviation in the right direction and the amount of deviation in the upward direction are both half the resolution. In this case, each mark element at the four corners completely overlaps one pixel, and the pixel has a pixel value corresponding to 100% of the mark color. On the other hand, the mark element at the center is captured in four pixels at a rate of 1/4 each.

As described above, when viewed at the resolution level, the image of the reference mark changes depending on the positional relationship between the reference mark and the pixel. Utilizing this property, the present embodiment performs highly accurate image matching as follows.

As shown in FIG. 3, in this embodiment, images of nine standard marks are prepared as reference patterns. Each image corresponds to the mark shift image of the present invention. The reference mark of each image is arranged at a position displaced from the other images by a distance smaller than the imaging resolution. The amount of mark deviation is half the resolution. Specifically, an image corresponding to FIG. 2A and an image in which the reference mark is shifted by 1/2 of the resolution in the up, down, left, right, and diagonal directions are prepared, and a total of 9 images are prepared. These reference patterns can be created by calculation without actually photographing the mark.

The image of the reference mark shown in the photographed image is compared with the above-mentioned nine mark shift images. Then, the mark shift image closest to the actual mark image is selected by the least square method or the like. The position of the reference mark in the selected mark shift image is specified as the position of the mark in the captured image. As a result, the position of the reference mark is half the resolution.
It is specified with high accuracy. Mark position identification is shown in Fig. 1.
This is done for each of the new and old images. Based on the detected mark position, both images are superimposed so that the two marks match. As a result, matching of the entire image is also performed with a high accuracy of 1/2 the resolution.

More specifically, first, the approximate position (resolution level) of the reference mark in the photographed image is obtained.
The approximate position is obtained from the setting of the shooting range and the position coordinates of the reference mark. Then, the mark position information at a detailed level can be obtained using the pattern image of FIG. As is clear from FIG. 3, this information indicates the positional relationship between the mark and the pixel of the captured image, that is, the displacement information (amount and direction) between the two. Matching is performed using the above-mentioned general position and the shift information which is the detailed position.

The image matching is performed after enlarging the photographed image. Due to the enlargement process, each cell in FIGS. 2 and 3 is composed of a plurality of pixels, and one pixel of the captured image corresponds to a plurality of pixels of the image processing apparatus. The positions of two captured images are adjusted in units of pixels on the image processing apparatus. This makes it easy to partially overlap cells and cells (pixels and pixels of the original image). The enlargement process may be performed at any stage of matching. However, when enlarging before the processing of FIG. 3, pattern images of the same size are prepared.

"Mark position specification (mark matching)"
The pattern image comparison process of FIG. 3 described above is a matching process of only the mark portion. In the present invention, this processing is called mark matching. This is to distinguish it from matching of the entire image. In the mark matching, for example, a difference in pixel value is obtained between pixels at the same position in two mark images to be compared. The absolute values of the pixel value differences are summed,
The image with the smallest total value is selected. Alternatively, the squares of the pixel value differences may be summed and the image with the smallest sum may be selected (least squares method). Mark matching can also be called filter processing. That is, the photographed reference mark is input, the nine mark shift images in FIG. 3 are referred to, and one mark image (mark position) is output.

The mark matching of this embodiment can be modified, for example, as follows.

(I) In the present embodiment, the reference mark is displaced by 1/2 of the resolution for each reference image, which results in 1 of the resolution.
A precision of / 2 has been achieved. However, the accuracy of matching can be further improved by preparing a reference image in which the reference marks are shifted more finely and by smaller distances. However, more reference images are required, and the data processing amount and processing time also increase. Considering this point, the required accuracy may be determined in consideration of the relationship between the imaging resolution and the size of the subject, and the reference image corresponding to the required accuracy may be used.

(Ii) Further, in the above-mentioned mark matching,
One mark-shifted image closest to the reference mark of the captured image was selected. In another processing example, two images having a high degree of similarity may be selected. The degree of similarity is, for example, the sum of squares of differences in pixel values, and the smaller the sum is, the higher the degree of similarity is. Then, a weighting calculation using the degree of similarity is performed to specify the position of the reference mark in the captured image. If the two images have the same degree of similarity, it is determined that the reference mark appears in the middle of the two images. It is also possible to use more than two reference mark images (or all the reference mark images) and perform the weighting calculation based on the similarity between these images. By such processing, further improvement in matching accuracy can be expected.

(Iii) Further, referring to FIG. 3, four pairs of the same mark shift image can be seen. Therefore, if at least five mark shift images are prepared, the same processing as above can be realized.

"Matching Example" FIG. 4 shows the result of matching performed by the above method in comparison with the prior art. 10 to 14 are enlarged views of the images in FIG. In addition to the fiducial mark, the subject at the top is
It includes speed indications "50" and "60" drawn on the road. In reality, the fiducial marks and these numbers are far apart.

The reference mark of the comparative image 1 appears under the same conditions as in FIG. 2A, and the reference mark of the comparative image 2 appears under the same conditions as in FIG. 2C. Both images have a mosaic shape because the resolution of satellite image capturing is about 1 m. And
The images of the subjects “50” and “60” vary as illustrated depending on the positional relationship with the pixels.

In the prior art, matching is performed in pixel units. The numbers "50" and "60" of the two images are overlapped, for example, in an arrangement in which the sum of the pixel values by the least squares method is the smallest. On the other hand, in the present invention, the position of the reference mark is obtained with higher accuracy than the resolution. That is, the deviation between the mark and the pixel is obtained for each comparative image. Both images are superposed based on this shift information. As a result of such processing, as shown in FIG. 4, it is possible to perform matching with higher accuracy than before, and it is possible to accurately detect changes in old and new images.

Next, FIG. 5 shows a matching result of images obtained by photographing the road (uppermost stage). Also, FIGS.
9 shows each image in FIG. 5 in an enlarged manner. In shooting 1, the reference mark appears in the same position as in FIG.
Then, the reference mark appears in the same position as in FIG. The white lines at both ends and the center of the road are mosaic-shaped,
There are variations between the two images.

According to the prior art, since the matching is performed in pixel units, the two images are overlapped with a slight shift.
However, according to the present invention, it is possible to accurately superpose two images by using the reference mark shift information.

The image matching of the present embodiment may be performed after subjecting the original image to appropriate image processing. From the viewpoint of matching, the image before processing and the image after processing are the same captured image. Therefore, matching the processed images is naturally included in the scope of the present invention. The image processing referred to here is, for example, correction processing of brightness or the like, ortho image generation processing described later, and the like.

"Map Update Device" Next, a map update device using the above image matching method will be described. The application range of the image matching of the present invention is not limited to map creation. However, as described above, as the use of digital maps in vehicles has expanded in recent years, there is an increasing demand for improvement in map accuracy, and it is preferable to apply the present invention to map creation.

Referring to FIG. 6, in response to the request for photographing sent from the receiving station 3, the artificial satellite 1 photographs the ground surface from the sky and sends the photographed image data to the receiving station 3 on the ground. Then, the map creating apparatus 10 of the present invention acquires the image data from the receiving station 3. The image data may be sent by wireless or wired communication. Further, the image data may be stored in a recording medium, and this medium may be attached to the map updating device 10. Furthermore, the captured image may be printed out at the receiving station 3, and the image may be captured by the map updating device 10 using a scanner or the like. Further, the map updating device 10 may be integrated with the receiving station 3. That is, the captured image may be directly sent from the artificial satellite 1 to the map updating device 10.

FIG. 7 shows the structure of the map updating apparatus 10. Here, the newly photographed image and the past photographed image are compared, and the map is updated (maintenance) based on the comparison result.

The photographed image is input from the image input section 12,
It is stored in the image storage unit 14. The input unit 12 is one form of the captured image acquisition means of the present invention. The storage unit 14 holds captured images used when updating maps in the past. The mark shift image storage unit 16 stores the nine mark shift images shown in FIG.

The mark position specifying unit 18 performs mark matching using the image of the reference mark portion in the photographed image and the mark shift image read from the storage unit 16 to specify the detailed position of the reference mark. The detailed position is 1 / of the resolution
2 shows the positional deviation information of the reference mark with the accuracy of 2.

The map creation processing section 20 compares the latest photographed image with the past photographed image used at the time of the last update,
Extract the changed part. At this time, both images are matched based on the processing result of the position specifying unit 18. Then, the extracted changed portion is reflected on the existing map. The existing map is read from the map storage unit 22 and used. In the update process, partial map changes, additions, and deletions are performed.
If the shape of the road has changed, change the road shape on the map. If roads have been added or deleted, do the same on the map. Other than roads (facilities, etc.)
Is also the same.

The updated map is stored in the map storage unit 22 and output from the output unit 24. For example,
The map is output to a printer, a display, an external map storage device, or the like. Also, the map may be sent to the map user by communication.

Next, FIG. 8 shows image processing by the map creation processing unit in FIG. When the image captured by the satellite is input, first, the luminance and saturation are corrected (S1
0), the image distortion is removed (S12).

Further, the highly accurate position correction characteristic of the present invention is performed (S14). The position correction is performed by the position specifying unit 18
Is performed based on the position specification result of. In the position correction, the positional relationship between the coordinate axes (latitude and longitude) and the captured image is corrected.

Specifically, first, the photographing condition information added to the photographed image is referred to, and the photographing direction and the photographing range are obtained. Since the setting position of the reference mark is known, the reference mark can be found in the image. The position of the captured image is corrected so that the reference mark is located at the set coordinates.
At this stage, only position correction at a level lower than the resolution is performed. Next, the detailed position of the reference mark obtained by the position specifying unit 18 is referred to. This information is 1 / of the resolution
The figure shows the fine positional deviation amount of the reference mark at the level of 2. Therefore, the position of the captured image is shifted by the amount of the shift.

Next, an orthoimage is created from the position-corrected image (S16). The ortho image processing is a processing for obtaining an image of the ground surface viewed from directly above by performing image processing on a satellite photograph obtained by photographing the same area from a plurality of angles. In this processing, elevation information of each point is obtained by a technique such as triangulation, and an image viewed from directly above is generated using the elevation information. The ortho image can represent roads, buildings, terrain, etc. without distortion even in the periphery of the image. The elevation information obtained by this processing is preferably used for creating a three-dimensional map.

Next, the orthoimage is matched with the orthoimage at the time of the previous map update, and the changed portion is detected by comparing the two (S18). The past orthoimages are stored in the image storage unit 14. The position correction described above is also performed on the past ortho images. Therefore, if the two images after the position correction are superposed as they are, the highly accurate matching of the present invention is performed.

After that, the process of creating a map from a normal photographed image may be performed on the portion detected in S18. Here, a skeleton map is created from the ortho image (changed portion) (S20). The skeleton map contains data indicating the shapes of roads, terrain, buildings, facilities, etc. These data are extracted from the ortho image.
A skeleton map can be efficiently created by overlaying ortho images and existing maps for comparison.

Further, basic data is calculated from the skeleton map (S22). The basic data is road link information (link length, slope, radius of curvature of curve, etc.). The road of the skeleton map is divided into link groups, and information is added to each link. This basic data is preferably used in a navigation digital map composed of road links.

Further, an updated map is created from the skeleton map and basic data (S24). That is, the existing map is read and the map is rewritten based on the information obtained in S20 and S22. At this time, the attribute data is added to the map. The attribute data includes road type, route number, traffic regulation, place name, intersection name, facility name and the like.
For example, various types of information about new roads are added to the map. The attribute data is stored in the map storage unit 22 or another storage device. The completed map is stored in the map storage unit 22 and output from the output unit 24.

[Comparison of Photographed Image and Detailed Shape Image] In the above embodiment, the old and new photographed images were compared. Next, an embodiment in the case where the comparison target is a detailed shape of an imaged portion will be described. The comparison target is information on detailed past shapes of roads, buildings, and the like at the shooting location, and is specifically a detailed image at the map or map level. Here, a map will be described as a comparison target.

First, in this embodiment, the special special reference mark as shown in FIG. 1 is not used. The fiducial mark is selected from existing shapes on the ground surface. For example, buildings and roads,
The speed display in FIG. 4 is used as a fiducial mark. However, a shape that is basically unchanged is selected as the reference mark.

This existing shape is mosaiced to a resolution equivalent to the photographing resolution. At this time, a plurality of mosaic images are created, and in each image, the existing shape is 1 / of the imaging resolution.
They are arranged so as to be displaced by a distance corresponding to 2. In this way, the mark shift image of this embodiment is obtained. These images are used for mark matching in the same manner as the mark shift image of FIG.

Further, the entire map image to be compared is also mosaiced to a level equivalent to the photographing resolution prior to matching with the photographed image. This mosaic image is matched with the captured image by the same processing as that of the above-described embodiment.

FIG. 9 is a flowchart showing the processing when the comparison target is a skeleton map. As a difference from FIG. 8, in S18, the ortho image is matched with the mosaicked skeleton map. This skeleton map is a map that reflects the changed portion obtained at the previous map update.

In the present embodiment, the objects to be compared are maps and skeleton maps, and these are images from which unnecessary information unrelated to updating is removed. Therefore, the update process can be easily performed by using such a comparison target. Particularly in the present embodiment, the comparison target is mosaiced prior to the matching. Therefore, images having the same resolution may be matched, images may be compared in units of cells corresponding to the resolutions, the matching process is facilitated, and the presence / absence of a change is facilitated.

Further, according to the present embodiment, as will be described below, it is possible to obtain information that cannot be understood by matching captured images. That is, a subject having a size of resolution can be seen in the image depending on the positional relationship with the pixels of the image sensor,
I can't see it. If such a subject is visible only in the image of this time and not in the previous image, it is difficult to compare the two. Simple comparison is difficult even when there is a difference of resolution or less due to difference in imaging conditions. However, if the comparison target is a detailed image such as a map, the existence and position of the subject are known in advance. For example, the subject is a white line (broken line) on the road
, The existence of the white line and the place where the white line is drawn are understood. If you use this comparison image,
It is also possible to match an object with a size as large as the resolution.

Further, in this embodiment, the reference mark is
Selected from existing shapes in the map. Therefore, since it is not necessary to newly install the reference mark, the installation cost and labor of the mark can be saved.

In the case of matching the photographed images described above, the existing shape may be used as the reference mark, and the same advantage can be obtained. On the contrary, it goes without saying that the dedicated reference mark of FIG. 1 may be used when using a map or the like for comparison.

Further, if the principle of this embodiment is applied,
High-precision matching is possible without reference marks. For example, an example of matching images in speed display will be described with reference to FIG. 4 used in the above embodiment. The images of "50" and "60" shown at the top of FIG. 4 are mosaiced. At this time, a plurality of mosaic images are created by gradually shifting the images. The shift amount is a distance smaller than the resolution (for example, the same 1/2 as described above). The mosaic image and the captured image are matched. This matching process may be performed on a limited part of the captured image. An enormous amount of data processing can be avoided and matching errors can be suppressed to a small level.

"Wavelength of reference mark" It is conceivable to set red (R), green (G), blue (B) and near-infrared as appropriate photographing wavelength ranges of the satellite photograph. Among them, near infrared rays have a special wavelength, and the reference mark can be made conspicuous by using this wavelength.

Specifically, the reference mark is constructed so as to reflect near infrared rays at a higher level than the surrounding general ground surface and installations on it. The closer the reflectance is to 100%, the better. A portion of the photographed image in which the intensity of near infrared rays is equal to or higher than a predetermined threshold is detected. The threshold value is set between the near-infrared intensity when the reference mark is photographed and the maximum near-infrared intensity when the ordinary ground surface is photographed.

Further, the reference mark may be constructed so as to reflect only near infrared rays, that is, not to reflect visible light. In this case, the reference mark does not appear in the RGB image but only in the near-infrared image, so that the reference mark is easily detected.

The reference mark may emit or absorb near infrared rays. In the case of absorption, the surface of the reference mark is made of a near infrared absorbing material. The absorption rate is higher than the surrounding general ground surface, etc., and is preferably set close to 100%. You can easily find the fiducial mark by looking for a blank area in the near-infrared image.

It is also preferable to construct the reference mark so as to absorb only near infrared rays based on the same principle as the above-mentioned reflection mode. A part that appears in the RGB image but not in the near infrared image is detected.

The map updating apparatus of this embodiment may be the same as that shown in FIG. 7, except that a mark detecting unit for detecting a mark based on near infrared light is added.

As described above, the reference mark can be made conspicuous by providing the reference mark that reflects the light of the special wavelength, which facilitates the detection of the reference mark from the photographed image. It is needless to say that the reference mark may be similarly configured by using a wavelength other than near infrared rays.

"Installation of reference mark on moving body" It is also preferable to provide the reference mark on the moving body. The moving body is a vehicle, a ship, or the like. This moving body is arranged at a predetermined position. The position of the moving body is accurately measured using a surveying technique such as GPS surveying. In this state, the satellite image is taken.

In this embodiment, the reference mark does not have to be fixed to the ground surface (including the roof of a building). Therefore, the mark installation cost and labor can be reduced. In particular, there is an area where the map is not frequently updated, and it is advantageous in that the reference mark does not need to be installed in such an area. In addition, the reference mark is not covered with trees, etc. In this respect, the maintenance cost of the mark can be reduced.

[Map Creation Device] In the device of FIG. 7, the map was updated and maintained using the result of specifying the position of the reference mark. However, if the position of the reference mark can be specified with high accuracy, the map accuracy can be improved even when the entire map is newly recreated.

The map creating apparatus of this embodiment has the same configuration as that shown in FIG. 7, and creates a map by the same processing as that shown in FIG. However, the difference from the processing of FIG. 8 is that the matching and comparison processing of S18 is not performed. Therefore, the captured image is converted into an ortho image after position correction, a skeleton map is created for a predetermined range (or the entire range) of the ortho image, and a map is created from this skeleton map. Since the position correction is accurate, the accuracy of the final map is also improved.

[Modification] Another modification of the present embodiment will be described below.

(1) In this embodiment, the mark having the special shape shown in FIG. 1, that is, the mark composed of five square mark elements is used. However, the shape of the reference mark is not limited to this. A mark with a simpler shape may be adopted. For example, the reference mark may be composed of only one square having a size equal to the resolution. However, if the reference mark has a simple shape, an erroneous matching is more likely to occur when an object having a similar shape happens to exist. Considering this point, it is preferable to adopt a mark having a special shape as shown in this embodiment.

(2) In the above description, satellite imaging is assumed, and it is assumed that the angle of the image with respect to the ground surface (coordinate axis) is the same for the new image and the old image. However, the shooting directions of the two images are different, and the new image may be tilted with respect to the old image. In this case, it is considered that the influence of the inclination is almost eliminated by matching in a large range on the image. For example, the inclinations of both images are aligned based on the shape of a long road. Also, prepare a plurality of reference marks. By aligning the plurality of reference points so that the two images match each other, the inclinations of the images are aligned.

(3) Further, as shown in FIG. 1, in the present embodiment, the shooting ranges of the two images are the same, but the shooting timings are different. This is because the comparison processing of old and new images was assumed. However, the present invention is not limited to such processing. For example, two images in which the shooting ranges partially overlap are acquired. The reference mark is arranged in the overlapping portion. These images are overlaid and joined using fiducial marks. The present invention effectively works for matching in such a connection process.

[0091]

As described above, according to the present invention, it is possible to perform matching of picked-up images with a high accuracy equal to or higher than the resolution. Furthermore, with high-precision matching,
It is possible to make an updated map accurately.

In particular, when a satellite image is used, the frequency of photographing can be increased, but the resolution can be suppressed. However, the accuracy of the map can be improved by applying the present invention. As a result, it becomes possible to provide the latest and highly accurate map.

Further, according to the present invention, it is possible to perform highly accurate matching although the reference mark is smaller than the resolution. Therefore, it is advantageous in that it is not necessary to install a large mark even when the resolution is limited such as satellite image capturing.

Further, according to the present invention, by using the existing shape as the reference mark, it is not necessary to install a dedicated reference mark, and it is possible to reduce the mark installation cost and labor.

Further, according to the present invention, by providing the reference mark that reflects, emits or absorbs light having a special wavelength such as near infrared rays, the reference mark can be easily detected.

Further, according to the present invention, by installing the reference mark on the moving body, it is possible to reduce the mark installation cost and labor. It is not necessary to set the mark fixedly in the area where the map is hardly updated.

Furthermore, according to the present invention, an accurate map can be created even when a new map is recreated from a photographed image by utilizing the high-precision reference mark position specifying function.

[Brief description of drawings]

FIG. 1 is a diagram showing reference marks used in a captured image matching method according to an embodiment of the present invention.

FIG. 2 is a diagram showing an image when the positions of reference marks in the captured image are slightly different.

FIG. 3 is a diagram showing a reference pattern image used to specify the position of a reference mark in detail.

FIG. 4 is a diagram showing a matching result by the method of the present invention in comparison with a conventional matching method.

FIG. 5 is a diagram showing a matching result by the method of the present invention in comparison with a conventional matching method.

FIG. 6 is a diagram showing an overall configuration of a map update system of the present embodiment.

FIG. 7 is a block diagram showing the configuration of the map updating device in FIG.

8 is a flowchart showing an update map creation process of the apparatus of FIG.

9 is a flowchart showing a second example of update map creation processing of the apparatus of FIG.

FIG. 10 is an enlarged view of the uppermost subject in FIG.

FIG. 11 is an enlarged view showing a first photographed image on the left side in the middle of FIG.

12 is an enlarged view showing a second photographed image on the right side in the middle of FIG.

13 is an enlarged view showing an image matched by a conventional method on the lower left side of FIG.

FIG. 14 is an enlarged view showing an image matched by the method of the present invention on the lower right side of FIG.

FIG. 15 is an enlarged view of the uppermost subject in FIG.

16 is an enlarged view showing a first photographed image on the left side in the middle of FIG.

17 is an enlarged view showing a second captured image on the right side in the middle of FIG.

FIG. 18 is an enlarged view of an image matched by a conventional method on the lower left side of FIG.

19 is an enlarged view showing an image matched by the method of the present invention on the lower right side of FIG.

[Explanation of symbols]

1 satellite, 3 receiving stations, 10 map updating device, 12
Image input unit, 14 image storage unit, 16 mark shift image storage unit, 18 mark position specifying unit, 20 map creation processing unit, 22 map storage unit, 24 output unit.

Claims (13)

(57) [Claims] 1. A terrestrial aerial at a predetermined imaging resolution is photographed, means for obtaining a photographic image including the reference mark of known shape within the imaging range, the hand to obtain the target image to be the captured image matching
A step , means for acquiring, as a reference pattern, a plurality of mark displacement images, which are images of reference marks that are respectively displaced by a distance smaller than the imaging resolution, and the reference marks reflected in the captured image are the plurality of marks. A mark position specifying unit that specifies the position of the reference mark in the captured image with higher accuracy than the shooting resolution by comparing with the shift image, and the captured image based on the specified position of the reference mark. captured image matching apparatus characterized by comprising an image matching means for matching the. 2. The captured image matching device according to claim 1.
In location, the captured image matching apparatus, wherein the target image is also captured image including the reference mark. 3. The captured image matching device according to claim 1.
In the above arrangement , the target image is an image obtained by mosaicking an image showing a detailed shape of a subject in a shooting range up to the shooting resolution level, and the captured image matching apparatus . 4. The photographed image matching device according to claim 1.
The reference mark includes at least one square having a size corresponding to one pixel of the image pickup device, and the reference mark of each mark shift image corresponds to one pixel of the image pickup device as compared with other mark shift images. The photographed image matching apparatus is characterized in that the mark position specifying means obtains deviation information indicating a deviation between the reference mark and the pixel of the image pickup device . 5. A map updating apparatus for updating a map using an image obtained by photographing the ground from the air, and photographing for obtaining a photographed image photographed with a predetermined photographing resolution and including a reference mark of a known shape within a photographing range. An image acquisition unit, a mark image storage unit that stores a plurality of mark shift images that are images of reference marks that are arranged at a distance smaller than the shooting resolution, and a plurality of reference marks that are captured in the shot image. Mark position specifying means for specifying the position of the reference mark in the photographed image with higher accuracy than the photographing resolution, and the photographed image based on the specified position of the reference mark. Extracting means for comparing with a comparison target image at the same location and extracting a changed portion of the captured image from the comparison target image; Zui and map update apparatus characterized by comprising a updating means for updating the map. 6. The map updating apparatus according to claim 5, wherein the comparison target image is a past captured image of the same location as the captured image. 7. The map updating apparatus according to claim 5, wherein the comparison target image is an image showing a past detailed shape of the same location as the photographed image, and the photographing resolution is prior to the comparison with the photographed image. A map update device characterized by being mosaicized to a level. 8. The map updating apparatus according to claim 5, wherein the reference mark is selected from existing shapes existing on the ground surface within a shooting range, and the mark shift image shows the existing shapes up to the shooting resolution level. A map updating device, which is a mosaiced image. 9. The map updating apparatus according to claim 5, wherein the reference mark reflects, emits or absorbs light in a predetermined wavelength range at a level different from that of the ground surface and installations thereon. Map update device. 10. The map updating apparatus according to claim 9, wherein the predetermined wavelength range is near infrared rays. 11. The map updating apparatus according to claim 5, wherein the reference mark is installed on a moving body. 12. A map creating apparatus for creating a map using an image of the ground taken from the air, which is taken at a predetermined imaging resolution and obtains an image that includes a reference mark of a known shape within an imaging range. An image acquisition unit, a reference mark image storage unit that stores a plurality of mark shift images that are images of reference marks that are respectively displaced by a distance smaller than the photographing resolution, and the reference mark reflected in the photographed image. Mark position specifying means for specifying the position of the reference mark in the photographed image with higher accuracy than the photographing resolution by comparing with a plurality of mark shift images, and an image based on the position of the specified reference mark. position
A position correction unit that performs correction, and performs position correction of the captured image based on the position of the specified reference mark ,
A map creating apparatus, which creates a map using the position-corrected image . 13. A means for capturing a photographed image of a ground image from the air with a predetermined photographing resolution and including a reference mark having a known shape within the photographing range, and at least a part of a subject within the photographing range. means for obtaining the detailed shape, an image, each has mosaic the detailed shape to said imaging resolution levels, a plurality of reference is an image in which the detailed shape is staggered by a small distance from the imaging resolution means for obtaining a mosaic image, by comparing the plurality of the reference mosaic image and the captured image, to identify the location of the detailed shape, have been identified
A picked-up image matching device , comprising: means for matching the picked-up image with the detailed shape based on the position of the detailed shape.