JP6433015B2 - Navigation device and navigation method - Google Patents

Description

  The present invention relates to a navigation device and a navigation method.

Conventionally, a navigation system that performs route guidance, that is, navigation between a current position and a designated target position is used. (For example, refer to Patent Document 1).
When guiding to a facility based on the coordinates of the facility that is the target position and the coordinates of the current position, a general navigation system navigates by a route on the map displayed on the display screen.

JP 2003-4474 A

However, if the facility at the target position does not approach the vicinity of the facility on the map, it is difficult to understand the relative position between the target position and the current position. Further, it is represented as geographical and geometric information on the display screen as a map image.
Actually, the state of human space recognition in the brain is not a geometric figure like a map drawn as it is, but the direction from the position of itself or a distance interval to the destination. Recognize space. Therefore, since the route to the destination is not stored as geometric information of the map, the direction of the destination is viewed many times in order to re-recognize the direction and distance to the destination. And confirm the distance.

In the human brain, the path to the target position is spatially recognized based on a combination of a sense of direction and distance from the current position to the target position, not geometric information such as a map.
In other words, based on a place that has never been visited, and looking at a detailed map to get to the target position, the human brain is in the direction of the path to the target position when there is no sense of direction of the current position. It is difficult to perform spatial recognition. The state where there is no sense of direction indicates a state where the sense of direction cannot be recognized even when looking at the map, for example, when going up from the subway to the ground.
As described above, the representation form of an image map is described in a geometric expression, and it is difficult to induce a sense of direction and distance from the current position in the human brain, and spatial recognition in the human brain. It's hard to say that it's the best to do.

  The present invention has been made in view of such circumstances, and provides information that is closest to the information format used for spatial recognition in the human brain, unlike the geometric information of a map. It is an object of the present invention to provide a navigation device and a navigation method capable of performing the above.

The present invention has been made to solve the above-described problems, and the navigation apparatus of the present invention extracts direction information and distance information from the current position to the target position from the coordinates of the current position and the target position. Spatial recognition information for generating, from the relative position analysis unit and each of the direction information and the distance information, spatial recognition information that allows a human to recognize the direction sense and distance sense of the route from the current position to the target position in the brain. A three-dimensional image corresponding to the coordinates of the left image data and the right image data; a generation unit; a five-sense information conversion unit that converts the spatial recognition information into five-sense information that is notified to a human through the five senses; In order to reduce the user's operation when selecting a measurement location in the three-dimensional image forming unit and the three-dimensional image, in either the left image or the right image A single image measurement point selection unit that converts the selected measurement point into a selection point in the three-dimensional image, and the single image measurement point selection unit corresponds to the position coordinate selected by the user. An image is read out, projected as left image data and right image data, and the measurement points input to either one are converted into coordinates in the three-dimensional image, and the distance between the measurement points or the area surrounded by the measurement points The area is calculated.
In the navigation device of the present invention , the sense information is auditory information, visual information, or tactile information, the auditory information is a sound signal having a frequency and intensity that can be recognized in human hearing, and the visual information is in human vision. It is illumination light with a recognizable frequency and intensity, and the tactile information is vibration information that can be recognized in human tactile sense.
In the case of the sound signal, the navigation device of the present invention changes the intensity of the sound according to the correspondence of the direction of travel to the direction of the target position, changes the frequency according to the distance from the current position to the target position, and In the case of light, the intensity of light is changed according to the direction of the target position in the direction of travel, the frequency of the blinking cycle is changed according to the distance from the current position to the target position, and in the case of the tactile information, the direction of travel The intensity of vibration is changed according to the direction of the target position, and the frequency of vibration is changed according to the distance from the current position to the target position.

In the navigation device of the present invention, the spatial recognition information is spatial character information that is a character that recalls a sense of direction and distance in the human brain, and spatial code information that is a symbol that recalls a sense of direction and distance. It is characterized by that.

  The navigation device according to the present invention further includes a media distribution unit that displays the spatial recognition information superimposed on a two-dimensional image used to generate a three-dimensional image in the direction of its own line of sight or the generated three-dimensional image. It is characterized by that.

  In the navigation device of the present invention, the single image measurement point selection unit uses a three-dimensional image formed from each of the left image and the right image, and the same pixel point of the left image and the right image by image correlation. The coordinates selected in either the left image or the right image are converted into coordinate points of a three-dimensional image by estimating each operation for selecting any one point.

  The navigation device of the present invention forms a plurality of surrounding three-dimensional images for each predetermined distance interval by using a left image and a right image captured at a predetermined angle for each predetermined distance interval, and the three-dimensional image is formed on the road plane. A road plane image generation unit that projects onto corresponding two-dimensional coordinates and extracts the plane image of the road is further provided.

The navigation method of the present invention includes a relative position analysis unit that extracts direction information and distance information from the current position to the target position from the coordinates of the current position and the target position, and each of the direction information and the distance information. A spatial recognition information generating unit for generating spatial recognition information for allowing a human to recognize a sense of direction and distance of a route from the current position to the target position in the brain;
A five-sense information conversion unit that converts the spatial recognition information into five-sense information that is notified to the human through the five senses, and a three-dimensional image forming unit that generates a three-dimensional image corresponding to the coordinates of the left image data and the right image data. The three-dimensional image forming process to be formed and the single image measurement point selection unit selected in either the left image or the right image in order to reduce the user's operation when selecting the measurement point in the three-dimensional image A single image measurement location selection process for converting a measurement location into a selection location in the 3D image, wherein the single image measurement location selection unit reads the 3D image corresponding to the position coordinates selected by the user. , Projecting each as left image data and right image data, converting the measurement point input to one of them into coordinates in the three-dimensional image, And calculating the area of a region surrounded by the measurement point.
In the navigation method of the present invention, the five sense information is auditory information, visual information, or tactile information, the auditory information is a sound signal having a frequency and intensity that can be recognized in human hearing, and the visual information is in human vision. It is illumination light with a recognizable frequency and intensity, and the tactile information is vibration information that can be recognized in human tactile sense.

  According to the present invention, unlike the geometric information of a map, information (spatial recognition information) that is closest to the information format used in spatial recognition of direction sense and distance intervals in the human brain is provided. It is possible to provide a navigation device and a navigation method that can be used.

It is a schematic block diagram which shows the structural example of the navigation apparatus by the 1st Embodiment of this invention. FIG. 6 is a diagram in which spatial character information and spatial code information are superimposed on a display screen of left imaging data and right imaging data forming a stereo image (three-dimensional image). It is a flowchart which shows the operation example of the navigation by the navigation apparatus of the 1st Embodiment of this invention. It is a schematic block diagram which shows the structural example of the navigation apparatus by the 2nd Embodiment of this invention. It is a figure which shows each display image of the left image data imaged with the stereo camera, and right image data. It is a flowchart which shows the operation example of the process of the dimension measurement of the three-dimensional image in the three-dimensional coordinate system by the navigation apparatus of the 2nd Embodiment of this invention. It is a flowchart which shows the other operation example of the process of the dimension measurement of the three-dimensional image in the three-dimensional coordinate system by the navigation apparatus of the 2nd Embodiment of this invention. It is a schematic block diagram which shows the structural example of the navigation apparatus by the 3rd Embodiment of this invention. It is a figure explaining the process which produces | generates the plane image of the road which moved while image | photographing from the several 3D image formed from the left image data and right image data imaged in the several continuous position. It is a flowchart which shows the operation example of the production | generation process of a planar road image (planar image) from the three-dimensional image in the three-dimensional space of the world coordinate system by the navigation apparatus of the 3rd Embodiment of this invention.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration example of a navigation device according to the first embodiment of the present invention. 1 includes a position input unit (current position input unit) 101, an imaging data input unit 102, a relative position analysis unit 103, a spatial character information generation unit 104, a spatial code information generation unit 105, and five sense information conversion. Unit 106, media distribution unit 107, sound conversion unit 108, vibration conversion unit 109, illumination conversion unit 110, equipment database 111, and image information database 112.

The position input unit 101 inputs information about the current position of the user supplied from the user and the name of the facility as the target position selected by the user. Further, the position input unit 101 searches the facility database 111 for the location of the facility with the above name corresponding to the name of the facility, and sets the extracted location of the facility as the target location. In the facility database 111, for example, the name of the facility to be managed, identification information for uniquely identifying the facility, the type of facility, the description of the facility, and the coordinates of the facility (facility coordinates) are written and stored in advance as a set. The image information database 112 stores image data of a map image to be displayed on a display screen such as a mobile terminal or a server.
The current position information may be anything that can specify coordinates on the map, such as an address input by the user or position information of a GPS (Global Positioning System) carried by the user.

  The imaging data input unit 102 inputs right image data and left image data supplied from a user, which are used to form a three-dimensional image as a stereo image. Then, the imaging data input unit 102 writes and stores the supplied right image data and left image data in the image information database 112 in correspondence with the position coordinates indicating the coordinates of the imaging position.

  The relative position analysis unit 103 reads a map image of an area including the coordinates of the current position and the target position from the image information database 112. In addition, the relative position analysis unit 103 obtains direction information indicating the direction of the target position viewed from the current position and distance information indicating the distance from the current position to the target position from the relative coordinates of the current position and the target position.

  The space character information generation unit 104 generates a character indicating the direction from the direction information, a character indicating the type of the road from the coordinates, and a number and a unit indicating the distance from the distance information. Here, the spatial character information generation unit 104 stores, for example, character information whose direction can be spatially recognized in the human brain from the direction information, and a table in which the character information indicating the direction and direction of the coordinate from the coordinate is stored. Extract from Here, the character information indicating the direction (spatial character information) is a text character indicating east, west, south, north, left, right, up, down, front, back, up, down, and the like.

In addition, the spatial character information generation unit 104, for example, on the map, the route from the current position to the target position is any one of a road, a sidewalk, a public land (land owned by a government office), and a private land (land owned by the private sector). Is extracted from the map stored in the image information database 112.
Further, the space character information generation unit 104 generates character information of a numerical value indicating a distance and a unit from the distance information. The character information indicating the distance (spatial character information) is a text character indicating a unit such as a text character indicated by 0 to 9 indicating a numerical value, km or m. That is, the spatial character information generation unit 104 generates spatial character information such as character text representing a position, character text representing an orientation, character text representing a distance, or character text representing a position, orientation, and distance.

  The spatial code information generation unit 105 generates a code indicating a direction from the direction information and a code indicating a distance from the distance information. The code information (spatial code information) indicating the direction is an arrow, an inclination symbol, or the like. Code information (spatial code information) indicating the distance is indicated by a length symbol, that is, the length of a line segment, the length of an arrow, or the thickness of an arrow. If the arrows are long or thin, a long distance is indicated. On the other hand, if the arrows are short or thick, a short distance is indicated. That is, the space character information generation unit 104 generates space code information other than the character text representing the position, the character text indicating the orientation, the character text indicating the distance, or the character text indicating the position, orientation, and distance.

Each of the spatial character information generation unit 104 and the spatial code information generation unit 105 described above is spatial recognition information that enables the human brain to recognize a direction interval and a distance interval as expressions other than a map (the spatial character information and the spatial code described above). Information). As a result, the user is notified of space character information and space code information, which are space recognition information based on the positional relationship from the current position to the target position. That is, the spatial recognition information includes the direction sense and the distance interval with respect to the human brain, which leads to spatial recognition from the relative coordinates between the two points of the target position and the current position recognized as spatial localization in the human brain. It is information that can convey.
Moreover, since the space character information and the space code information used as the space recognition information can be transmitted to the user's mobile terminal by mail or an application, the space character information and the space code information are not limited to being shown as image information on the map. In addition, the space recognition information may be similarly transmitted not only to the mobile terminal but also to a computer such as a server of a management center that manages public facilities.

  The five sense information conversion unit 106 generates five sense information that allows a human to recognize the direction and distance based on the five senses based on the spatial character information and the spatial code information. In the present embodiment, the five senses indicate vision, hearing, and touch. Here, the five sense information represents, for example, a direction and a distance by a signal length, strength, blinking frequency, and the like.

The sound conversion unit 108 converts the signal generated as the five sense information into sound information (sound) of frequency and intensity as auditory information that can be recognized by human hearing. Here, for example, the sound conversion unit 108 changes the intensity of the sound corresponding to the direction of the target position based on the change of the current position notified from the user, and changes the sound intensity corresponding to the distance. Change the frequency.
As an example, the sound conversion unit 108 performs control to increase the sound intensity as the direction in which the user travels approaches the target position, and to increase the sound frequency as the user's current position approaches the target position. Configure as follows. As long as it is a rule that allows each user to perceive the direction sense and the distance interval in the brain, the sound intensity and frequency may be controlled to indicate the direction and distance.

The vibration converting unit 109 converts the signal generated as the five sense information into vibration information (vibration vibration, hereinafter simply referred to as vibration) of frequency and intensity as tactile information that can be recognized by human tactile sense. Here, for example, the vibration conversion unit 109 is notified from the user and changes the intensity of vibration corresponding to the direction of the target position based on the change of the current position, and vibration corresponding to the distance. Change the frequency.
As an example, the vibration conversion unit 109 increases the intensity (amplitude) of vibration as the direction in which the user proceeds approaches the direction of the target position, and increases the frequency of vibration as the current position of the user approaches the target position. Configure to perform control. As long as it is a rule that allows each user to perceive spatial sense of direction sense and distance interval in the user's brain, the intensity and frequency of vibration may be controlled to indicate the direction and distance.

The illumination conversion unit 110 converts the signal generated as the five sense information into optical information (illumination light) of frequency and intensity as visual information that can be recognized by human vision. Here, for example, the illumination conversion unit 110 changes the intensity of the illumination light in accordance with the direction of the target position in accordance with the direction of the target position from the change in the current position notified from the user, and corresponds to the distance. Change the frequency of the illumination light.
As an example, the illumination conversion unit 110 increases the intensity (illuminance) of the illumination light as the direction in which the user travels approaches the direction of the target position, and the blinking cycle of the illumination light as the user's current position approaches the target position. The control is performed so as to increase the frequency. As long as it is a rule that recognizes each direction of the sense of direction and distance interval in the user's brain, the intensity of the illumination light and the frequency of the blinking cycle may be controlled to indicate the direction and distance.

The media distribution unit 107 transmits sound information, vibration information, and light information generated by the sound conversion unit 108, the vibration conversion unit 109, and the illumination conversion unit 110, respectively, to the user's mobile terminal. As a result, sound information when the user's mobile terminal is not in the mute mode, vibration information when the user is in the mute mode and the vibration mode, and light information when the user is in the mute mode and not the vibration mode, from the current position to the target position. Information indicating the direction and distance is notified to the user.
As described above, according to the present embodiment, when both hands of a user are blocked by work or driving and a map or the like cannot be viewed in detail as image information, the user can recognize the information as human recognizable information. When the notification is made, the portable terminal (corresponding to the specification and usage state) carried by the user has a scalability capable of selecting various media devices for transmitting information.

FIG. 2 is a diagram in which spatial character information and spatial code information are superimposed on a display screen of left imaging data and right imaging data forming a stereo image (three-dimensional image).
FIG. 2 shows left imaging data and right imaging data for forming a three-dimensional image in a three-dimensional space of the world coordinate system, which is captured by two in-vehicle imaging devices. FIG. 2A shows the left imaging data, and FIG. 2B shows the right imaging data.

In FIG. 2B, the media distribution unit 107 notifies the user's mobile terminal of spatial character information and spatial code information as image information to be superimposed on the right image data. Here, an arrow is used as the spatial code information, the direction of the arrow indicates the direction of the target position from the current position, and the thickness or length of the arrow, which is the spatial code information, indicates the distance. The numerical value indicating the distance and the text character of the unit are space character information.
Further, the coordinates obtained by capturing the left image data and the right image data are measured by GPS or the like, the measured coordinates are set as the current position, and the three-dimensional image formed from the left image data and the right image data is determined from the current position. Spatial identification information indicating the direction and distance with respect to the target position may be superimposed to display an image as shown in FIG.

  That is, the arrow in the area 501 is an example indicating that the target position is 150 m rightward from the current position. An arrow in the region 502 is an example showing that the target position is 250 m rightward from the current position. In this figure, there are not two destinations, but is described for comparison showing that the thickness and length of the arrows differ according to the distance and direction. Since there is usually only one target position, there is also one arrow. In FIG. 2B, the arrow in the area 502 is a large arrow because it indicates that the distance to the target position is far from the current position as compared to the area 501 example. Thus, even if only the arrow is displayed, the direction and distance from the current position to the target position can be recognized from the direction of the arrow and the thickness and length of the arrow.

The media distribution unit 107 transmits the spatial character information generated by the spatial character information generation unit 104 and the spatial code information generated by the spatial code information generation unit 105.
The user's mobile terminal superimposes the distributed spatial character information and spatial code information on the right image data and left image data for generating a stereo image, and displays an image as shown in FIG.
Further, the navigation device may be configured to superimpose spatial character information and spatial code information distributed corresponding to the current position on the image data captured by the mobile terminal instead of the stereo image.

FIG. 3 is a flowchart illustrating an example of navigation operation by the navigation device according to the first embodiment of the present invention.
Step S1:
The user transmits the address or coordinates (latitude and longitude) of the current position to the navigation device by e-mail or navigation application from the mobile terminal.
The mobile terminal may transmit the coordinates of the current position by GPS to the navigation device when the user inputs navigation instructions.

Step S2:
The user transmits the name, address, or coordinates (latitude / longitude) of the target position to the navigation device by e-mail or navigation application from the portable terminal.

Step S3:
The relative position analysis unit 103 obtains the coordinates of the current position from the information on the current position from the mobile terminal.
Further, the relative position analysis unit 103 obtains the coordinates of the target position from the information on the target position from the mobile terminal. Here, when the destination location information or the identification information for identifying the destination facility is supplied as the destination location information, the relative position analysis unit 103 selects the facility name or identification from the facility table of the facility database 111. Extract the coordinates of the equipment corresponding to the information. In the equipment table, information such as equipment coordinates and types is written and stored in advance in correspondence with the equipment name and identification information.

  The relative position analysis unit 103 reads a map image of an area including the coordinates of the current position and the target position from the image information database 112. In addition, the relative position analysis unit 103 obtains direction information indicating the direction of the target position viewed from the current position and distance information indicating the distance from the current position to the target position from the relative coordinates of the current position and the target position. The direction information is information indicating whether or not the latitude and longitude of the coordinates of the target position increase with respect to the coordinates of the current position, that is, the difference between the latitude and longitude of the coordinates of the target position and the coordinates of the current position. The direction of east, west, south, and north can be indicated by the polarities and ratios of the latitude and longitude differences.

Step S4:
The spatial character information generation unit 104 obtains the distance from the current position to the target position on the map. That is, the space character information generation unit 104 generates a text character indicated by 0 to 9 indicating a numerical value that is character information indicating a distance, and a text character indicating a unit of distance such as km or m. Further, the space character information generation unit 104 generates text characters indicating the directions of east, west, north, south, and up, down, left, and right based on the polarities and ratios of the latitude and longitude differences between the current position and the target position. For example, a table in which the polarities and ratios of the latitude and longitude differences correspond to the information indicating the direction is used.

  The spatial code information generation unit 105 generates spatial code information such as information on the direction of an arrow indicating the direction from the current position to the target position based on the polarities and ratios of the latitude and longitude differences between the current position and the target position. To do. The spatial code information generation unit 105 also generates spatial code information such as the thickness and length of an arrow indicating the distance from the current position to the target position.

Step S5:
The five sense information conversion unit 106 generates five sense information that allows a human to recognize the direction and distance based on the five senses based on the spatial character information and the spatial code information.
Then, the sound conversion unit 108 converts the signal generated as the five sense information into sound information in which the direction and distance are expressed by frequency and intensity.
Further, the vibration conversion unit 109 converts the signal generated as the five sense information into vibration information in which the direction and the distance are expressed by the frequency and the intensity.
The illumination conversion unit 110 converts the signal generated as the five sense information into optical information in which the direction and distance are expressed by frequency and intensity.

Step S6:
The media distribution unit 107 sends the sound information, vibration information, and light information generated by the conversion of the sound conversion unit 108, the vibration conversion unit 109, and the illumination conversion unit 110 to the mobile terminal of the user who requested navigation. To send.
Thereby, in the user's mobile terminal, navigation that constitutes the spatial recognition of each of the sense of direction and the distance interval in the user's brain is performed based on the spatial recognition information such as sound information, vibration information, and optical information.
In addition, as shown in FIG. 2, spatial character information and spatial code information may be displayed superimposed on image data indicating the user's traveling direction displayed on the display screen of the mobile terminal.

Step S7:
The navigation device detects whether or not information indicating that the navigation is stopped is supplied from the user, and determines whether or not to continue the process.
At this time, when the information indicating that the navigation is stopped is supplied from the user, the navigation device ends the process.
On the other hand, when the information indicating that the navigation is stopped is not supplied from the user, the navigation device advances the process to step S1.

As described above, according to the present embodiment, unlike the geometric information of the map, the spatial recognition information that is considered to be the closest to the information format used in the spatial recognition of the sense of direction and the distance interval in the human brain. (Information that allows the human brain to recognize the direction sense and the distance interval spatially) notifies the user of information for navigation from the current position to the target position, so that the user can easily target the target position in the brain. Can be recognized, and a route to the target position can be easily assumed.
In addition, according to the present embodiment, it is compatible with a portable terminal (media device for transmitting information) carried by the user, and the space recognition information can be easily distributed by effectively using the human senses. It has sex.
In addition, according to the present embodiment, by using the five senses, there is an advantage that space recognition in the brain can be more effectively performed without looking at the space character information and the space code information on the display screen.

<Second Embodiment>
The second embodiment of the present invention will be described below with reference to the drawings. FIG. 4 is a schematic block diagram showing a configuration example of a navigation device according to the second embodiment of the present invention. In the embodiment of the present invention in FIG. 4, a three-dimensional image forming unit 113 and a single image measurement location selection unit 114 are added to the configuration of the first embodiment shown in FIG. 1. Hereinafter, configurations and operations different from those of the first embodiment will be described.

  The imaging data input unit 102 includes user identification information and captured coordinates (latitude and longitude) in each of left image data and right image data supplied from a stereo camera (not shown) every time a predetermined distance passes. The image information database 112 is written and stored in time series. In the stereo camera, each of the left image data and the right image data is imaged synchronously (at the same timing) at every predetermined distance. Each of the left image data and the right image data captured by the stereo camera is supplied from the stereo camera to the three-dimensional image forming unit 113 via the imaging data input unit 102.

The three-dimensional image forming unit 113 uses the left image data and the right image data supplied from the stereo camera by the imaging data input unit 102, and forms a three-dimensional image in a three-dimensional space of the world coordinate system by using, for example, a relational expression. Here, the above relational expression is obtained by stereo analysis for associating pixels indicating the same coordinates in the three-dimensional space of the world coordinate system from each of the left image data and the right image data in the camera coordinate system of the left and right line of sight. It is a formula that forms a three-dimensional image in space. Here, each of the left image data and the right image data is an image captured at the same timing by the stereo camera configured by two imaging devices.
Further, the three-dimensional image forming unit 113 writes and stores it in the image information database 112 corresponding to the coordinates at which the formed three-dimensional image is captured. Here, the coordinates may be measured by GPS, or may be obtained from position information such as an address input by the user.

  The single image measurement location selection unit 114 selects the measurement position selected as the position for measuring the dimension on the display screen of either the left image data or the right image data, for the 3D image formed by the 3D image forming unit 113. Convert to coordinates in 3D space. The dimension is a distance between two predetermined points (two measurement points) of a three-dimensional image in a three-dimensional space of the world coordinate system, an area of a region surrounded by a plurality of three or more points, and a line indicated by three points For example, the angle indicated by the minute. Here, the single image measurement point selection unit 114 reduces the operation (selection) in the selection of the position for measuring the dimension on the display screen of the user, on the display screen of either the left image data or the right image data. A position for measuring a dimension is selected, and a measurement point for a three-dimensional image in a three-dimensional space is detected. This is because the measurement point is input at the same position in the left image data and the right image data, and thus there has been a problem that the operation load for inputting the measurement point by the user is heavy. For this reason, the single image measurement point selection unit 114 according to the present embodiment inputs the measurement point only to either the left image data or the right image data, thereby minimizing the operation load of the user to input the measurement point. It is a configuration realized by paying attention to.

  FIG. 5 is a diagram illustrating display images of left image data and right image data captured by a stereo camera. In FIG. 5, the measurement point selection process for dimension measurement in the three-dimensional image of the world coordinate system formed by the left image data and the right image data will be described. FIG. 5A shows left image data taken by an imaging device arranged on the left side of the stereo camera, and FIG. 5B shows right image data taken by an imaging device arranged on the right side of the stereo camera.

In general, as shown in FIG. 5, in the display screens of both the left image data and the right image data, pixels that respectively indicate the same measurement point in the display images of FIGS. 5 (a) and 5 (b). Specify the direction. When measuring the distance from the shoulder of the sidewalk to a predetermined position on the road, as shown in FIG. 5A, the mark ML1 is input to the image of the shoulder and the mark ML2 is input to the predetermined position on the road. . Further, as shown in FIG. 5B, the mark MR1 is input to the road shoulder image at the same position as the mark ML1, and the mark MR2 is input at a predetermined position on the road at the same position as the mark ML2.
Thus, the coordinates of the measurement point in the three-dimensional coordinate system in which the three-dimensional image in the three-dimensional space of the world coordinate system formed from the left image data and the right image data is arranged, that is, the mark ML1 and the make MR1 are respectively corresponded. The first coordinate and the second coordinate corresponding to each of the mark ML2 and the make MR2 are obtained, and the distance between the first coordinate and the second coordinate of the three-dimensional image in the three-dimensional space is calculated.

On the other hand, in the present embodiment, as shown in the description of the single image measurement location selection unit 114, the measurement point is not specified by both the left image data and the right image data, but the left image data or the right image data. Therefore, it is only necessary to specify and specify the measurement point, so that it is possible to reduce the operation load in selecting the measurement point by the user.
Further, in the image information database 112, for each coordinate at which an image pair of left image data and right image data captured by a stereo camera is captured, it is written and stored together with the coordinates. For example, the left image data and the right image data are captured at a predetermined distance (as an example, every 5 m) by a stereo camera mounted on an automobile. In this case, coordinates measured by GPS are used as coordinates.

FIG. 6 is a flowchart showing an operation example of the process of measuring the dimensions of the three-dimensional image in the three-dimensional space of the world coordinate system by the navigation device of the second embodiment of the present invention.
Step S11:
The three-dimensional image forming unit 113 associates an image pair of left image data and right image data in which a measurement target three-dimensional image selected by a user via a mobile terminal or the like is associated with coordinates input by the user. Read from the image information database 112.
The three-dimensional image forming unit 113 displays each display image of the read left image data and right image data on a display device such as a portable terminal.

Step S12:
The three-dimensional image forming unit 113 forms a captured three-dimensional image from the read image data and right image data using a relational expression.
Then, the three-dimensional image forming unit 113 writes and stores the formed three-dimensional image in the image information database 112 in correspondence with the coordinates of the left image data and the right image data.

Step S13:
The single image measurement point selection unit 114 reads a three-dimensional image from the image information database 112 in correspondence with the coordinates of the measurement target selected by the user.
The single image measurement location selection unit 114 projects a three-dimensional image on the left image data and the right image data, coordinates of the respective three-dimensional images in the three-dimensional space of the world coordinate system, the left image data, and the right image data. Are associated with each pixel to obtain a correspondence relationship.

Step S14:
The single image measurement location selection unit 114 inputs the measurement point to the user after the association between the coordinates in the three-dimensional space of the world coordinate system and the pixels of the left image data and the right image data is completed. Prompt image display. As a result, the user inputs a measurement point on the display screen of either the left image data or the right image data using an input unit (not shown).

Step S15:
Using the above correspondence, the single image measurement location selection unit 114 sets the pixel of the measurement point input by the user on the display screen of either the left image data or the right image data as a coordinate point in the three-dimensional coordinate system. Associate. Here, the single image measurement location selection unit 114 determines the position of the measurement point input on the display screen of either the left image data or the right image data, and correlates the image between the one image data and the other image data. Estimation is performed by performing processing (this processing is performed every time a measurement point is input).
Then, the single image measurement point selection unit 114 uses the correspondence relationship based on the position of the measurement point in each of the left image data and the right image data to coordinate the measurement point corresponding to the coordinate in the three-dimensional space of the world coordinate system. And perform measurement corresponding to the measurement point. The single image measurement point selection unit 114 converts the position of the measurement point input on either the left image data or the right image data into the coordinates of the measurement point associated with the coordinates in the three-dimensional space. Process immediately. At this point, the three-dimensional image has already been generated in advance. When the measurement amount of the dimension to be measured is the distance between two measurement points, the distance between the coordinates of the two points corresponding to the measurement point is calculated. Further, when the measurement amount of the dimension to be measured is the area of the region surrounded by three or more measurement points, the area of the region surrounded by the coordinates associated with the plurality of measurement points is calculated. If the measurement amount of the dimension to be measured is the angle between the start point, midpoint, and end point, and the straight line connecting the start point and midpoint and the straight line connecting the midpoint and end point, the start point, midpoint, and end point Are associated with coordinates in the three-dimensional space of the world coordinate system, a straight line connecting the start point and the middle point in the three-dimensional space and a straight line connecting the middle point and the end point are obtained, and an angle formed by these straight lines is calculated.

Step S16:
The single image measurement location selection unit 114 displays measurement values (distance, area, angle, etc.) of dimensions measured based on the measurement points in the vicinity of the measurement points input by the user.

Step S17:
The navigation device detects whether or not information indicating that the dimension measurement process is stopped is supplied from the user.
At this time, the navigation device ends the process when information indicating that the dimension measurement process is stopped is supplied from the user.
On the other hand, when the information indicating that the dimension measurement process is stopped is not supplied from the user, the navigation apparatus advances the process to step S11.

FIG. 7 is a flowchart showing another operation example of the dimension measurement process in the three-dimensional space of the world coordinate system by the navigation device according to the second embodiment of the present invention. In the following description, the flow of processing when a measurement point is input on the right image data display screen will be described.
Step S21:
The three-dimensional image forming unit 113 associates the image pair of the left image data and the right image data in which the measurement target three-dimensional image selected by the user is captured with the coordinates input by the user, from the image information database 112. read out.
The three-dimensional image forming unit 113 displays each display image of the read left image data and right image data on a display device such as a portable terminal.

Step S22:
The user inputs a measurement point by the above input means on either the left image data or the right image data, for example, on the display screen of the right image data.

Step S23:
The single image measurement location selection unit 114 uses the luminance value and color of the selected pixel to obtain the extracted pixel of the left image data corresponding to the selected pixel as the measurement point input by the user on the display screen of the right image data. Extract feature points from left pixel data.
Then, the single image measurement location selection unit 114 detects the extracted pixel corresponding to the selected pixel from the pixels extracted as the feature points of the left image data by the matching method, and sets the extracted pixel.
Here, the pixel of the left image data corresponding to the pixel of the right image data means a pixel having the same coordinates in the three-dimensional space of the world coordinate system generated from the left image data and the right image data.

Step S24:
The three-dimensional image forming unit 113 forms a captured three-dimensional image from the read left image data and right image data using a relational expression.
At this time, the single image measurement location selection unit 114 projects the three-dimensional image on the left image data and the right image data, and coordinates of the left image data and the right image data in the three-dimensional space of the world coordinate system. Correspondence is obtained by associating with each pixel.
Then, the three-dimensional image forming unit 113 writes and stores the formed three-dimensional image in the image information database 112 in correspondence with the coordinates of the left image data and the right image data.

Step S25:
Then, the single image measurement point selection unit 114 performs measurement using measurement points associated with coordinates in the three-dimensional space of the world coordinate system. As for the processing contents, the same dimension measurement processing as in step S15 in FIG.

Step S26:
The single image measurement location selection unit 114 displays measurement values (distance, area, angle, etc.) of dimensions measured based on the measurement points in the vicinity of the measurement points input by the user.

Step S27:
The navigation device detects whether or not information indicating that the dimension measurement process is stopped is supplied from the user, and determines whether or not to continue the process.
At this time, the navigation device ends the process when information indicating that the dimension measurement process is stopped is supplied from the user.
On the other hand, when the information indicating that the dimension measurement process is stopped is not supplied from the user, the navigation apparatus advances the process to step S21.

  With the configuration described above, conventionally, it has been necessary to separately input measurement points to the display screens of both the left image data and the right image data. However, according to the present embodiment, the left image data or the right image data Since the dimensions of the three-dimensional image formed from the left image data and the right image data can be measured only by inputting the measurement point only on one of the image data display screens, the load on the user is reduced compared to the conventional case. Can be reduced.

<Third Embodiment>
The third embodiment of the present invention will be described below with reference to the drawings. FIG. 8 is a schematic block diagram showing a configuration example of a navigation device according to the third embodiment of the present invention. In the embodiment of the present invention in FIG. 8, a three-dimensional image forming unit 113 and a planar image generating unit 115 are added to the configuration of the first embodiment shown in FIG. Hereinafter, configurations and operations different from those of the first embodiment will be described.

  FIG. 9 is a diagram illustrating a process of generating a planar image of a road that has been moved from a plurality of three-dimensional images formed from left image data and right image data captured at a plurality of consecutive positions. . FIG. 9A shows a process in which the imaging data input unit 102 captures left image data and right image data for each predetermined angle when, for example, a stereo camera mounted on the rotating table is rotated 360 ° by the rotating table. ing. And the left image data and right image data which the stereo camera imaged are input into a navigation apparatus from a portable terminal. FIG. 9B shows a composition in which a three-dimensional image formed from left image data and right image data is projected onto a two-dimensional plane in a three-dimensional space of the world coordinate system, and the generated peripheral two-dimensional image is synthesized. It is a road plane image as a plane image (ortho image) generated by superimposing and synthesizing two-dimensional images.

  Here, every time the vehicle travels a preset distance (as an example, 5 m), the stereo camera is rotated by the turntable at that position, and the stereo camera rotates the left image data and the right image in the imaging direction at each predetermined angle. Image each of the data. The process of rotating the turntable and causing the stereo camera to capture the left image data and the right image data may be performed by the mobile terminal or by another in-vehicle control device.

The imaging data input unit 102 stores each of the left image data and the right image data supplied from the mobile terminal in time series in the image information database 112 in association with the added imaging position and imaging angle. Let
The portable terminal adds the information of the imaging position and the imaging angle to each of the left image data and the right image data, and transmits the information to the navigation device. For example, coordinates acquired by an in-vehicle GPS are used as positions for capturing each of the left image data and the right image data.

  The three-dimensional image forming unit 113 converts a three-dimensional image at each captured position and angle from each of the left image data and the right image data captured at each captured angle at each position. It is formed using a relational expression in a dimensional space. Then, the three-dimensional image forming unit 113 writes and stores each of the three-dimensional images formed in the three-dimensional space of the world coordinate system in the image information database 112 in time series together with the corresponding imaged position and angle. Let Here, the formed three-dimensional image is represented by coordinates in a three-dimensional space of the world coordinate system.

  The planar image generation unit 115 sequentially reads out the three-dimensional images at the captured positions and angles in the position range designated by the user from the image information database 112. Then, the planar image generation unit 115 projects each read three-dimensional image onto a two-dimensional plane corresponding to the ground surface of the three-dimensional space, and generates a two-dimensional image from the three-dimensional image. Further, the planar image generation unit 115 combines the two-dimensional images at the respective positions so that the same coordinates in the three-dimensional space of the world coordinate system overlap, and generates a combined two-dimensional image for each position.

Then, the planar image generation unit 115 combines the combined two-dimensional image for each position so that the same coordinates in the three-dimensional space of the world coordinate system overlap, and generates a planar image in the position range specified by the user.
Since the three-dimensional image in the three-dimensional space is projected on the two-dimensional plane corresponding to the ground surface, this plane image can obtain an image similar to the aerial photograph taken from the air as shown in FIG. 9B.

  FIG. 10 is a flowchart illustrating an example of operation of generating a planar road image (planar image) from a three-dimensional image in a three-dimensional space of the world coordinate system by the navigation device according to the third embodiment of the present invention. In the following description, as an example, as shown in FIG. 9 (a), an in-vehicle stereo camera is positioned at every 5 m at 90 ° angles (0 °, 90 °, 180 °, The left image data and the right image data are captured every 270 °. Further, the left image data and the right image data are written and stored in the image information database 112 in a time series corresponding to the positions and angles at which the left image data and the right image data are captured.

Step S31:
The user selects a position range for generating a planar image as a processing area, and transmits it to the navigation device via the portable terminal.
The position input unit 101 transmits a position range specified by the user supplied from the mobile terminal to the imaging data input unit 102.

Step S32:
The three-dimensional image forming unit 113 sequentially reads the image pair of the left image data and the right image data at the imaging position in the position range specified by the user from the image information database 112 for each captured angle.

Step S33:
The three-dimensional image forming unit 113 generates a three-dimensional image in the three-dimensional space of the world coordinate system from each of the left image data and the right image data at each angle at the same position for each angle at which each position is imaged. A plurality are formed.
Then, the three-dimensional image forming unit 113, the three-dimensional image forming unit 113, each of the three-dimensional images formed for each captured angle of each position, together with the corresponding captured position and angle, the image information database 112. Are written and stored in time series.

Step S34:
The planar image generation unit 115 sequentially reads out each of the three-dimensional images formed for each captured angle at each position from the image information database 112.
Then, the planar image generation unit 115 generates a two-dimensional image by projecting each of the read three-dimensional images for each angle onto a two-dimensional plane corresponding to the ground surface.
The planar image generation unit 115 sequentially writes and stores the generated two-dimensional image in the image information database 112 in correspondence with the captured position and angle.

Step S35:
Next, the planar image generation unit 115 detects that the process of generating a two-dimensional image from the three-dimensional image at each angle of the imaged position in the position range specified by the user is completed.
Then, the planar image generation unit 115 sequentially reads out the two-dimensional image at each angle at the captured position from the image information database 112.
The planar image generation unit 115 synthesizes two-dimensional images corresponding to all the angles of the captured positions by superimposing pixels having the same coordinates in the world coordinate system. As a result, the planar image generation unit 115 generates a combined two-dimensional image that is a projected two-dimensional image of a 360-degree three-dimensional image around each position.

Next, the planar image generation unit 115 synthesizes the synthesized two-dimensional image for each position by superimposing pixels having the same coordinates in the world coordinate system. As a result, the planar image generation unit 115 generates a planar image, which is a two-dimensional image shown in FIG. 9B, in which 360-degree three-dimensional images around all positions in the position range designated by the user are projected. To do.
When this plane image is a road plane image obtained by imaging a road, as shown in FIG. 9B, only the road portion in the two-dimensional image is cut out to generate a combined two-dimensional image, and a plane image is generated. May be. For the extraction of the road portion, the plane image generation unit 115 extracts the correspondence between the road on the map stored in the equipment database 111 and coordinates in the vicinity thereof, and each coordinate of the two-dimensional image, and images other than the road This is done by deleting.

Step S36:
As shown in FIG. 9B, the plane image generation unit 115 generates a plane image generated from the left image data and the right image data captured at predetermined angles at each position in the position range designated by the user as shown in FIG. Display an image on the terminal.

Step S37:
The navigation device detects whether or not information indicating that the plane image generation process is to be stopped is supplied from the user, and determines whether or not to continue the process.
At this time, the navigation apparatus ends the process when information indicating that the process of generating the planar image is stopped is supplied from the user.
On the other hand, when the information indicating that the plane image generation process is stopped is not supplied from the user, the navigation apparatus advances the process to step S31.

  With the configuration described above, according to the present embodiment, a two-dimensional image obtained by projecting this three-dimensional image onto a plane using a three-dimensional image formed from each of the left image data and the right image data at each predetermined angle at each position. By forming an image and superimposing the same coordinates in the world coordinate system on this two-dimensional image, it is possible to easily generate a planar image (ortho image) around the predetermined imaging position range.

  Also, a program for realizing the function of the navigation device shown in each of FIGS. 1, 4 and 8 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. Each process of navigation may be performed by executing. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system.

  Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. Further, the program may be a program for realizing a part of the functions described above, and is a so-called differential file that can realize the functions described above in combination with a program already recorded in the computer system. May be.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within a scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 101 ... Current position input part 102 ... Imaging data input part 103 ... Relative position analysis part 104 ... Spatial character information generation part 105 ... Spatial code information generation part 106 ... Five sense information conversion part 107 ... Media conversion part 108 ... Sound conversion part 109 ... Vibration conversion unit 110 ... Illumination conversion unit 111 ... Equipment database 112 ... Image information database 113 ... Three-dimensional image formation unit 114 ... Single image measurement location selection unit 115 ... Plane image generation unit

Claims (9)

A relative position analysis unit that extracts direction information and distance information from the current position to the target position from the coordinates of the current position and the target position;
From each of the direction information and the distance information, a space recognition information generating unit that generates space recognition information for allowing a human to recognize a direction sense and a distance sense of a route from the current position to the target position in the brain;
A five-sense information conversion unit that converts the spatial recognition information into five-sense information that notifies a human through the five senses;
A three-dimensional image forming unit that forms a three-dimensional image corresponding to the coordinates of the left image data and the right image data;
Single image measurement for converting a measurement location selected in either the left image or the right image into a selection location in the three-dimensional image in order to reduce user operations when selecting a measurement location in the 3D image With a location selector and
The single image measurement point selection unit reads out the three-dimensional image corresponding to the position coordinates selected by the user, projects it as left image data and right image data, and inputs the measurement points input to either one of them. A navigation device that converts to coordinates in the three-dimensional image and calculates a distance between measurement points or an area of a region surrounded by the measurement points. The single image measurement location selection unit,
By estimating the three-dimensional image formed from each of the left image and the right image for each operation that selects one of the same pixel points of the left image and the right image by image correlation, The navigation apparatus according to claim 1 , wherein coordinates selected in either the left image or the right image are converted into coordinate points of a three-dimensional image. The five sense information is auditory information, visual information or tactile information,
The auditory information is a sound signal having a frequency and intensity recognizable in human hearing, the visual information is illumination light having a frequency and intensity recognizable in human vision, and the tactile information is recognizable in human tactile vibration information. The navigation device according to claim 1 or 2 , wherein: In the case of the sound signal, the intensity of the sound is changed according to the direction of the target position in the traveling direction, the frequency is changed according to the distance from the current position to the target position,
In the case of the illumination light, the intensity of light is changed according to the direction of the target position in the traveling direction, the frequency of the blinking period is changed according to the distance from the current position to the target position,
If the haptic information, the process proceeds direction of the changes the intensity of vibration by response to the direction of the target position, according to claim 3, wherein changing the frequency of the vibration by the distance from the current position to the target position The navigation device described in 1. The spatial recognition information is
Space character information in the human brain is a character reminiscent of sense of direction and depth perception, and claims 1, characterized in that a space code information including a symbol evoking sense of direction and distance feeling of claim 4 The navigation device according to any one of the above. The image processing apparatus further comprises a media distribution unit that superimposes and displays the spatial recognition information on a two-dimensional image used to generate a three-dimensional image in its own line-of-sight direction or a generated three-dimensional image. The navigation device according to any one of claims 1 to 3. For each predetermined distance interval, a plurality of surrounding three-dimensional images for each predetermined distance interval are formed from the left image and the right image captured at a predetermined angle, and the three-dimensional images are projected onto the two-dimensional coordinates corresponding to the road plane. The navigation apparatus according to any one of claims 1 to 6, further comprising a road plane image generation unit that extracts a plane image of the road. A relative position analysis unit that extracts direction information and distance information from the current position to the target position from the coordinates of the current position and the target position;
From each of the direction information and the distance information, a space recognition information generating unit that generates space recognition information for allowing a human to recognize a direction sense and a distance sense of a route from the current position to the target position in the brain;
A five-sense information conversion unit that converts the spatial recognition information into five-sense information that notifies a human through the five senses;
A three-dimensional image forming unit that forms a three-dimensional image corresponding to the coordinates of the left image data and the right image data;
When the single image measurement point selection unit selects a measurement point in the three-dimensional image, the measurement point selected in either the left image or the right image is selected in the three-dimensional image in order to reduce the user operation. A single image measurement location selection process for converting to a selection location, and
The single image measurement point selection unit reads out the three-dimensional image corresponding to the position coordinates selected by the user, projects it as left image data and right image data, and inputs the measurement points input to either one of them. A navigation method comprising: converting to coordinates in the three-dimensional image and calculating a distance between measurement points or an area of a region surrounded by the measurement points. The five sense information is auditory information, visual information or tactile information,
The auditory information is a sound signal having a frequency and intensity recognizable in human hearing, the visual information is illumination light having a frequency and intensity recognizable in human vision, and the tactile information is recognizable in human tactile vibration information. The navigation method according to claim 8, wherein:

Images