JP7715583B2 - Location identification support system and location identification support method - Google Patents

Description

The present invention relates to a location identification support system that generates information for identifying a location from the representation of an object perceived by a querying subject.

Today, using GNSS (Global Navigation Satellite System) one can determine one's location. However, in urban areas (for example, between buildings), it is not possible to receive radio waves from the number of satellites required for positioning calculations, and location cannot be determined using GNSS. For this reason, there is a need to determine location using information about objects that the inquiring party can perceive. For example, in the case of a traffic accident or emergency, it is desirable to accurately determine the location even when the inquiring party is not familiar with the local geography, such as when out and about or traveling, and does not know the address or names of landmarks.

The following prior art exists as background art in this technical field. Patent Document 1 (JP 2001-133283 A) describes a navigation device equipped with a voice recognition means that receives voice input and outputs various instruction signals, an information search means that searches for information data from an information recording medium using a search instruction signal received from the voice recognition means, and a voice output means that outputs the search results by voice. The information search means includes a search genre discrimination unit that determines a signal indicating a search genre from the received search instruction signal, a search direction discrimination unit that determines a search instruction signal indicating a search direction, a search range setting unit that sets a search range, and a search unit that searches for the information data according to the determined search genre, search direction, and search range (see claim 1).

Japanese Patent Application Laid-Open No. 2001-133283

Conventionally, location-related information (address, landmark names, GPS positioning information, etc.) is entered into the system, and the corresponding location is searched for in a map information system. However, if the person making the inquiry or the operator is not familiar with the local geography (for example, they do not know the address or the names of landmarks), searching using landmarks becomes difficult. In this case, information about objects visible to the user (for example, "I can see a building with a red pointed roof" or "It's near a store with a blue sign") is effective in identifying the location, but such information is not registered in the map information system.

Also, even if you know the name of a landmark, you can further narrow down the location if you can also use information that the landmark is "visible."

Furthermore, in the background art mentioned above, a navigation device can search map data using words indicating directions such as "left" or "right" used by the speaker. However, this is a method for adding variety to the way destinations are expressed in situations where the speaker's location information is known in advance, and is not a technology that can be used in situations where the reference location is unknown.

Furthermore, the objects that humans can perceive are not limited to static landscapes; they include people, animals, and moving objects (e.g., distinctive vehicles such as fire engines), and they can also perceive environmental information (e.g., temperature, smell, and sound). Utilizing this perceptual information is expected to improve the accuracy of localization.

The present invention aims to realize a location identification support system that identifies the location of an object based on information perceived by the person making the inquiry.

A representative example of the invention disclosed in the present application is as follows: That is, a location identification support system that generates data for identifying a location includes a calculation device that executes predetermined processing and a storage device accessible by the calculation device, the storage device stores integrated 3D data including attributes of stationary objects and attributes of moving objects, a reception unit in the calculation device that receives an input of a representation of a moving object recognized by a query subject, a semantic analysis unit in the calculation device that identifies a search term for searching the integrated 3D data from the received representation of the moving object and generates a search query including one or more area specification conditions, and a location estimation unit in the calculation device that searches the integrated 3D data with the generated search query, obtains information for narrowing down a location from the received representation of the moving object, and generates the information for narrowing down the location.

According to one aspect of the present invention, the location of an object perceived by a querying subject can be determined from information about the object. Problems, configurations, and advantages other than those described above will become clear from the description of the following embodiments.

1 is a block diagram showing a configuration of a location identification support system according to an embodiment of the present invention; FIG. 2 is a block diagram showing the physical configuration of a location identification assistance server according to an embodiment of the present invention. FIG. 2 is a diagram showing a logical configuration of the location identification assistance server according to the embodiment of the present invention when generating integrated 3D data. FIG. 2 is a diagram illustrating a logical configuration of a location identification assistance server according to an embodiment of the present invention when estimating a location. 10 is a flowchart of a position estimation process executed by a position estimation unit according to an embodiment of the present invention. 10 is a detailed flowchart of a region of interest calculation process for a stationary object according to an embodiment of the present invention. 10 is a detailed flowchart of a region of interest calculation process for a moving object according to an embodiment of the present invention. 10 is a flowchart of an object area calculation process for environmental information according to an embodiment of the present invention. FIG. 10 is a diagram illustrating an example of a search result screen displayed on a user terminal according to an embodiment of the present invention.

<Network configuration example>
FIG. 1 is a block diagram showing the configuration of a location identification support system according to an embodiment of the present invention.

The location identification assistance system of this embodiment is composed of a location identification assistance server 10, and the location identification assistance server 10 and user terminal 30 are connected via a network.

The location identification assistance server 10 generates and stores integrated 3D data 113 in real time, which associates metadata 105 representing surrounding environmental information with three-dimensional data 108.

An inquiring entity (including a reporter other than the person receiving the service) receiving the location identification assistance service inputs multiple pieces of information to the location identification assistance server 10, such as the relative position of perceptible stationary and moving objects, and information about the surrounding environment (for example, "There's a coffee shop on your left," "I can hear music playing," "A clown just passed by"). The location identification assistance server 10 searches the integrated 3D data 113, calculates possible location ranges that can be estimated from each piece of input information, and superimposes the calculated multiple possible location ranges to narrow down the location of the inquiring entity with a high probability.

The location identification assistance server 10 can provide location identification assistance services, route guidance services, mobility assistance services, etc. to individuals via the user terminal 30, provide location identification assistance services to people or organizations that need to identify the location of others (for example, fire or police command centers, or information centers at large facilities), and provide location identification assistance services to robots 40 and automated guided vehicles that travel indoors.

In a solution that provides location identification assistance services to individuals, the user inputs information about objects perceived by the inquiring entity (e.g., "There's a coffee shop on your left," "I can hear music playing," "A clown just passed by") into a user terminal 30 connected to the location identification assistance server 10, and the user's likely location is displayed on a map. For example, when browsing a map on the Internet, if the user's location information cannot be obtained using GNSS, the location identification assistance service of this embodiment can display the user's location on a map, allowing the user to identify the location even if the user is unfamiliar with the local geography. In this embodiment, objects include not only still landscapes, but also people, animals, and moving objects (e.g., distinctive vehicles such as fire engines).

In a solution that provides location identification assistance services to people or organizations that need to locate others, an operator (such as a dispatcher or guide) listens to information about objects perceived by others (e.g., "There's a coffee shop on your left," "I can hear music playing," "A clown just passed by") and inputs it into a user terminal 30 connected to the location identification assistance server 10, thereby displaying the likely location of the other person on a map. In the fire department or police command center that receives emergency calls, the inquiring party's location is narrowed down by providing the operator with an address, landmark names, etc. However, if the inquiring party is unfamiliar with the local geography, the operator will narrow down the location by listening to information about landmarks visible to the inquiring party. However, this method relies on the operator's local knowledge and skills. Therefore, the location identification assistance service of this embodiment can display the inquiring party's location on a map, allowing the location of an incident to be quickly identified in response to an emergency call.

In a solution that provides a location identification assistance service for a robot 40 or the like, information on brightness, sound, air pressure, smell, and surrounding objects acquired by sensors mounted on the robot 40 is sent to a location identification assistance server 10, which then provides candidate locations for the robot 40. It is difficult to position a robot 40 that patrols indoors using GNSS (Global Navigation Satellite System), and it is difficult to identify locations in similar environments (e.g., office buildings, underground shopping malls, hotel corridors) using LiDAR to measure three-dimensional data of the surrounding area. For this reason, the location identification assistance service of this embodiment makes it possible to identify the location of the robot 40 without installing beacons or markers.

The localization assistance server 10 is connected to sensing devices (e.g., an audio capture device 21, a video capture device 22, and an environmental measurement device 23), and collects surrounding environmental information from these sensing devices 21, 22, and 23 and stores it in metadata 105. The sensing devices 21, 22, and 23 may be installed in a fixed, dispersed manner in a specified area where the localization assistance service is provided, or may be mounted on a robot 40 that patrols the specified area where the localization assistance service is provided.

The user terminal 30 is configured as a computer having a processor (CPU), memory, auxiliary storage device, communication interface, input interface, and output interface, and provides search request input functions and screen display functions. For example, the search request input functions and screen display functions may be provided by a web browser executed by the user terminal 30, or by a dedicated application program executed by the user terminal 30.

Figure 2 is a block diagram showing the physical configuration of the location identification assistance server 10 in this embodiment.

The location identification assistance server 10 is composed of a computer having a processor (CPU) 1, memory 2, auxiliary storage device 3, and communication interface 4. The location identification assistance server 10 may also have an input interface 5 and an output interface 6.

The processor 1 is a computing device that executes programs stored in the memory 2. The processor 1 executes various programs to realize the functions of each functional unit of the location identification assistance server 10 (e.g., the audio analysis unit 101, the video analysis unit 102, the sensor information analysis unit 103, the information receiving unit 104, the 3D data analysis unit 106, the 3D data receiving unit 107, the facility information input unit 109, the information integration unit 111, etc.). Note that some of the processing performed by the processor 1 when executing the programs may be executed by another computing device (e.g., hardware such as an ASIC or FPGA).

Memory 2 includes ROM, a non-volatile storage element, and RAM, a volatile storage element. ROM stores unchanging programs (e.g., BIOS). RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by processor 1 and data used when executing the programs.

The auxiliary storage device 3 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or flash memory (SSD). The auxiliary storage device 3 also stores data used by the processor 1 when executing programs (e.g., metadata 105, three-dimensional data 108, facility information 110, dictionary data 112, integrated 3D data 113, etc.), as well as the programs executed by the processor 1. In other words, the programs are read from the auxiliary storage device 3, loaded into memory 2, and executed by the processor 1 to realize the various functions of the location identification assistance server 10.

The communication interface 4 is a network interface device that controls communication with other devices (e.g., user terminal 30) according to a specified protocol.

The input interface 5 is an interface to which input devices such as a keyboard 7 and a mouse 8 are connected and which receives input from an operator. The output interface 6 is an interface to which output devices such as a display device 9 and a printer (not shown) are connected and which outputs the results of program execution in a format visible to the operator.

The program executed by the processor 1 is provided to the location identification assistance server 10 via removable media (CD-ROM, flash memory, etc.) or a network, and is stored in the non-volatile auxiliary storage device 3, which is a non-transitory storage medium. For this reason, the location identification assistance server 10 should preferably have an interface for reading data from removable media.

The location identification assistance server 10 is a computer system configured on a single physical computer or on multiple logically or physically configured computers, and may operate on a virtual computer built on multiple physical computer resources. For example, the audio analysis unit 101, video analysis unit 102, sensor information analysis unit 103, information receiving unit 104, 3D data analysis unit 106, 3D data receiving unit 107, facility information input unit 109, and information integration unit 111 may each operate on separate physical or logical computers, or multiple units may be combined to operate on a single physical or logical computer.

Figures 3 and 4 show the logical configuration of the location identification assistance server 10 in this embodiment. Figure 3 shows the configuration when generating integrated 3D data in which metadata is associated with three-dimensional data, and Figure 4 shows the configuration when estimating location.

As shown in Figure 3, when generating integrated 3D data, the location identification assistance server 10 has an audio analysis unit 101, a video analysis unit 102, a sensor information analysis unit 103, an information receiving unit 104, a 3D data analysis unit 106, a 3D data receiving unit 107, a facility information input unit 109, and an information integration unit 111, and stores metadata 105, three-dimensional data 108, dictionary data 112, and integrated 3D data 113.

The audio analysis unit 101 generates the results of analyzing the acoustic signal acquired by the audio acquisition device 21. For example, it analyzes the acoustic signal acquired by the audio acquisition device 21 to identify the noise level. It also extracts characteristic sounds and generates data in which the extracted characteristic sounds are labeled. For example, if the noise level is 80 decibels or higher, it is labeled "noisy," and if the sound of a railroad crossing is extracted, it is labeled "railroad crossing" and "sound."

The audio capture device 21 is a microphone that captures acoustic signals (voice, music, environmental sounds, etc.), and as described above, may be installed in a fixed location in a dispersed manner to cover the specified area where the location identification assistance service is provided, or may be mounted on a robot that patrols the specified area where the location identification assistance service is provided.

The video analysis unit 102 generates the results of analyzing the video captured by the video capture device 22. For example, it analyzes the video captured by the audio capture device 21 to extract non-stationary moving objects, and analyzes the type (person, dog, car, truck, etc.), attributes (gender, clothing, etc.), behavior (walking, running, lying down, falling, etc.), and movement speed (walking, running, etc.) of the extracted moving objects. For example, it analyzes that a clown is walking eastward and assigns the labels "person," "clown," "eastward," and "walking."

The video capture device 22 is a camera that captures video, and as described above, may be installed in a fixed location in a dispersed manner to cover the specified area where the location identification assistance service is provided, or may be mounted on a robot that patrols the specified area. The video capture device 22 and the audio capture device 21 may be configured as a single device (a camera with a microphone).

The sensor information analysis unit 103 generates the results of analyzing the environmental information acquired by the environmental measuring device 23. For example, it categorizes and labels the brightness, temperature, humidity, moving objects, etc. measured by the environmental measuring device 23.

The environmental measurement device 23 is a sensor that measures environmental information in a specified area where the location identification assistance service is provided, and is, for example, an illuminance sensor that measures brightness, a temperature sensor that measures temperature, a humidity sensor that measures humidity, or a LiDAR that detects moving objects.

The information receiving unit 104 combines the analysis results output from the audio analysis unit 101, video analysis unit 102, and sensor information analysis unit 103 based on the location information associated with the ID of the sensor that made the observation, and generates structured metadata 105.

Metadata 105 is structured data of the analysis results output from the audio analysis unit 101, video analysis unit 102, and sensor information analysis unit 103, and is structured to be searchable by text in association with location information.

The 3D data analysis unit 106 analyzes the three-dimensional data 108 acquired by the 3D data measurement device 24 and sends it to the 3D data receiving unit 107. The 3D data analysis unit 106 may use existing three-dimensional data (for example, Plateau provided by the Ministry of Land, Infrastructure, Transport and Tourism, or three-dimensional data used during the design of a building) instead of the three-dimensional data 108 acquired by the 3D data measurement device 24. Furthermore, data such as a manually created surface mesh may also be used instead of measured data.

The 3D data measurement device 24 is a sensor capable of acquiring three-dimensional data, such as a LiDAR, stereo camera, or range imaging camera, and acquires three-dimensional data representing the spatial shape of the specified area where the location identification assistance service is provided.

The 3D data receiving unit 107 is a 3D engine (a so-called 3D game engine) that excels in handling three-dimensional data, converts the three-dimensional data output from the 3D data measuring device 24 into a predetermined format, and stores it in the three-dimensional data 108.

The facility information input unit 109 accepts input of information about facilities located in a specified area where the location identification assistance service is provided, and stores it in facility information 110. For example, information that is perceived by humans and serves as a key for inquiries, such as the store name, type of business, and the shape, materials, color, and finish of facilities (walls, signs, doors, etc.), is input in advance before the location identification assistance service is provided.

The information integration unit 111 generates integrated 3D data 113 by integrating the metadata 105 generated by the information receiving unit 104, the three-dimensional data 108 generated by the 3D data receiving unit 107, and the facility information 110 input to the facility information input unit 109 based on location information. At this time, the information integration unit 111 integrates location information of the range perceptible by humans (such as where an object can be seen, a sound can be heard, or a smell can be detected) according to the type of metadata 105. The information integration unit 111 also generates dictionary data 112, which is a synonym dictionary for aggregating search terms included in search requests. The dictionary data 112 may be input in advance before the location identification assistance service is provided.

As shown in Figure 4, during location estimation, the search request input unit 121 accepts a search request input by voice or text from the user terminal 30. The search request can accept multiple area specification conditions that express the perception of the query subject. For example, each of the area specification conditions shown in Figure 1 are "There is a coffee shop on the left," "I can hear music playing," and "A clown just passed by."

The voice analysis unit 122 uses voice recognition technology to convert the voice of the search request input to the search request input unit 121 into text. Note that if the location identification assistance server 10 does not support voice input, the voice analysis unit 122 does not need to be implemented.

The semantic analysis unit 123 uses morphological analysis technology to break down the search request entered as text in the search request input unit 121 and the search request converted to text by the speech analysis unit 122 into words, converts variations in expressions and ambiguous expressions contained in the entered natural language into specified search keys, and creates a search query containing the search key. For example, it replaces the area specification condition "There is a coffee shop on your left" with "Within a 10m radius of the cafe" and "You are in a position where you can see the cafe on your left" to generate two search queries.

The location estimation unit 124 references the integrated 3D data 113 to estimate a location corresponding to the input search request. It then superimposes the ranges of estimated locations for each of the multiple area specification conditions input as the search request to narrow down the location corresponding to the search request. Details of the location estimation process performed by the location estimation unit 124 will be described with reference to Figures 5 to 8. The location estimation unit 124 then superimposes the ranges of estimated locations for each of the multiple input search requests on a map to generate an image for narrowing down the location corresponding to the search request (for example, an image to be displayed in the map display area 910 of the search result screen shown in Figure 9).

In this embodiment, the location identification assistance server 10 outputs the estimated location as audio and/or an image. When outputting the estimated location as audio, the audio synthesis unit 125 identifies the most probable location using the range of the estimated location and its accuracy, and uses audio synthesis technology to vocalize the most probable estimated location. The result transmission unit 126 outputs the vocalized most probable location to the user terminal 30. The user terminal 30 outputs the audio signal received from the location identification assistance server 10 from a speaker to notify the inquiring entity of the location with the highest accuracy. Note that the user terminal 30 that outputs the results may be the same as or different from the user terminal 30 to which the search request is input. For example, a search request may be input into a smartphone carried by the inquiring entity, and the estimated location may be vocally output from a speaker installed near the estimated location.

When the estimated location is output as an image, the result transmission unit 126 outputs a result display screen (Figure 9) including the generated image to the user terminal 30. The user terminal 30 outputs the result display screen received from the location identification assistance server 10 to its display and notifies the inquiring subject of the likely location.

Figure 5 is a flowchart of the position estimation process performed by the position estimation unit 124.

Before the location estimation unit 124 performs the location estimation process, the semantic analysis unit 123 receives a search request containing perceptual information perceived by the query subject from the search request input unit 121 or the speech analysis unit 122 (201), breaks down the search request into words using morphological analysis technology, converts variations in expressions and ambiguous expressions contained in the input natural language into predetermined search keys, and creates a search query containing the search keys (202). The location estimation unit 124 then starts the location estimation process.

The location estimation unit 124 searches the integrated 3D data 113 using a search query generated from one area specification condition included in the search request, and acquires objects that match the area specification condition (203). For example, if the search query is "within a 10-meter radius of the cafe" and "located in a position where the cafe is visible to the left," cafes within the area where the location identification assistance service is provided are acquired as objects.

The position estimation unit 124 then begins processing each of the multiple acquired objects (204). First, the position estimation unit 124 estimates the attributes of the object (205). If the integrated 3D data 113 defines the attributes, the object attributes may be acquired from the integrated 3D data 113. The attributes of the object may also be determined based on the type of sensing device that is the basis of the area designation conditions for this object. For example, among objects detected by a camera or LiDAR, stationary objects may be considered stationary objects, objects in which movement is detected may be considered moving objects, and objects related to the area designation conditions detected by sensing devices other than a camera or LiDAR may be considered environmental information.

The position estimation unit 124 then allocates processing because the target area calculation method differs depending on the object attribute. If the object attribute is a stationary object, target area calculation processing 207 for stationary objects is executed. Details of target area calculation processing 207 for stationary objects will be explained with reference to Figure 6. If the object attribute is a moving object, target area calculation processing 208 for moving objects is executed. Details of target area calculation processing 208 for moving objects will be explained with reference to Figure 7. If the object attribute is environmental information, target area calculation processing 209 for environmental information is executed. Details of target area calculation processing 209 for environmental information will be explained with reference to Figure 8.

Then, it is determined whether there are any other unprocessed search queries (210). If there are unprocessed search queries, processing returns to step 206 and continues. Once processing of all search queries has been completed, processing for each object is repeated from step 204 (211).

Once processing of all objects has been completed, it is determined whether there are any other area designation conditions (212). If there are any unprocessed area designation conditions, the process returns to step 203 and continues. Once processing of all area designation conditions has been completed, the position estimation process ends.

Figure 6 is a detailed flowchart of the target area calculation process 207 for stationary objects.

The position estimation unit 124 acquires the area designation conditions (221) and allocates processing because the method of calculating the target area differs depending on the area designation conditions (222). If the area designation conditions relate to distance, a circle centered on the moving object corresponding to the area designation conditions is acquired (223). For example, if the search query is "within a 10 m radius of the cafe," a circle with a 10 m radius is generated. If the area designation conditions relate to direction, the area from the current candidate location to the target object is acquired (224). For example, if the search query is "I am in a position where the cafe is visible to the left," the area between the cafe and the current candidate location is generated.

The location estimation unit 124 then calculates a target region based on the spatial information (225). For example, the location estimation unit 124 superimposes the generated region to narrow down the location corresponding to the search request.

FIG. 7 is a detailed flowchart of the region of interest calculation process 208 for moving objects .

The position estimation unit 124 determines whether there is a moving object within a specified time range before and after the time specified in the area designation condition (231). At this time, the search may be narrowed down by the object's movement speed specified in the area designation condition. If there is no moving object that matches the area designation condition, the target area based on the moving object cannot be identified, and the target area calculation process for the moving object is terminated. On the other hand, if there is a moving object that matches the area designation condition, the area designation condition for that moving object is obtained (232). Since the target area calculation method differs depending on the area designation condition, processing is assigned (233). If the area designation condition is related to distance, a circle centered on the moving object that matches the area designation condition is obtained (234). If the area designation condition is related to distance, a circle centered on the moving object that matches the area designation condition is generated (234). For example, if the area designation condition is "A clown just passed by," a circle with a radius of 4 m is generated from the clown's position at the time specified in the area designation condition. If the area designation condition is related to direction, the area from the current candidate location to the target object is obtained (235). For example, area designation conditions related to direction include "in front," "in front," "on the left," "on the right," and "toward the wall." If the area designation condition is related to time, the total area of a circle centered on the target object that exists within a specific time range in the area designation condition is obtained (236). For example, if the area designation condition includes "running away," the time range is set to 3 seconds during which the object is seen running, and a circle centered on the position of the target object for 3 seconds is generated.

The location estimation unit 124 then calculates a target area based on the spatial information (237). For example, the location estimation unit 124 superimposes the calculated target area to narrow down the location corresponding to the search request.

Figure 8 is a flowchart of the target area calculation process 209 for environmental information.

First, the semantic analysis unit 123 extracts words corresponding to environmental information from the area specification conditions (241). For example, environmental information such as hot, bright, windy, smelly, and music is extracted.

The position estimation unit 124 then references the integrated 3D data 113 using the extracted environmental information to select an area corresponding to the extracted environmental information (242). For example, depending on environmental information such as hot, bright, windy, smelly, or musical, the location estimation unit 124 generates a perceptible area that can be perceived by the inquiring subject using propagation characteristics (attenuation rate, diffraction, etc.), spatial characteristics (spatial shape, absorption rate, reflectance, etc.), and the perceptual characteristics of the inquiring subject (gaze height, etc.), and calculates a probability density according to the degree to which the environmental information is perceived.

The location estimation unit 124 then calculates a target area based on the spatial information (237). For example, the location estimation unit 124 superimposes the calculated target area to narrow down the location corresponding to the search request.

Figure 9 shows an example of a search result screen displayed on the user terminal 30.

The search results screen shown in Figure 9 has a search request input field 901 and a refinement button 902 on the left side, and a map display area 910 on the right side.

The search request input field 901 displays, as text, the words entered as a search request into the user terminal 30 based on the voice recognition results. A text input field may also be provided to accept text input via keyboard operation. When the inquirer operates the refine button 902, the search request displayed in the search request input field 901 is sent to the location identification assistance server 10, and a display of the area identified by the search criteria is added to the map display area 910. The search request input field 901 highlights the words used to search the integrated 3D data 113. To the left of each area specification condition in the search request, a mark corresponding to the area specification condition is displayed superimposed on the map in the map display area 910. A map with the area identified by the search superimposed is displayed in the map display area 910.

As described above, according to an embodiment of the present invention, a location can be identified from information about objects perceived by the querying subject. Since not only still landscapes but also moving objects and environmental information are used as objects that humans can perceive, the accuracy of location identification can be improved.

The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations that include all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or in software, by a processor interpreting and executing a program that realizes each function.

Information such as programs, tables, and files that implement each function can be stored in storage devices such as memory, hard disks, and solid-state drives (SSDs), or in recording media such as IC cards, SD cards, and DVDs.

Furthermore, the control lines and information lines shown are those considered necessary for explanation, and do not necessarily represent all control lines and information lines necessary for implementation. In reality, it is safe to assume that almost all components are interconnected.

REFERENCE SIGNS LIST 1 Processor 2 Memory 3 Auxiliary storage device 4 Communication interface 5 Input interface 6 Output interface 7 Keyboard 8 Mouse 9 Display device 10 Location identification assistance server 11 Search request reception unit 12 Search key generation unit 13 Target object search unit 14 Target area extraction unit 15 Terminology dictionary 16 3D city model 17 Visible point information 18 Map information 19 Screen generation unit 30 User terminal 21 Search request input function 22 Screen display function

Claims (16)

A location identification support system that generates data for identifying a location,
A computing device that executes predetermined processing and a storage device that can be accessed by the computing device,
the storage device stores integrated 3D data including attributes of stationary objects and attributes of moving objects;
the computing device includes a receiving unit that receives an input of a representation of a moving object recognized by a query subject;
a semantic analysis unit configured to identify a search term for searching the integrated 3D data from the received representation of the moving object and generate a search query including one or more area specifying conditions;
a location estimation unit configured to search the integrated 3D data using the generated search query, obtain information for narrowing down a location from the representation of the accepted moving object, and generate information for narrowing down the location. 2. The location identification assistance system according to claim 1,
the integrated 3D data includes information about the surrounding environment;
the receiving unit receives an input of an expression of an environment recognized by an inquiry subject;
the semantic analysis unit identifies search terms for searching the integrated 3D data from the received environment representation to generate a search query;
The location estimation unit searches the integrated 3D data using the generated search query and obtains information for narrowing down the location from the received representation of the environment. 2. The location identification assistance system according to claim 1,
The location estimation unit generates screen data for superimposing on a map areas corresponding to the plurality of area designation conditions related to the moving object recognized by the inquiry subject. 2. The location identification assistance system according to claim 1,
The computing device includes a voice analysis unit that analyzes the voice acquired by the voice acquisition device;
The arithmetic device includes a video analysis unit that analyzes the video captured by the video capture device;
the computing device has an environment analysis unit that analyzes the environmental values measured by the environment measuring device;
an information receiving unit that generates metadata by combining the analysis results of the audio analysis unit, the video analysis unit, and the environment analysis unit based on location information;
an integration unit that integrates the generated metadata and three-dimensional map information based on location information to generate the integrated 3D data. 2. The location identification assistance system according to claim 1,
The position estimation unit
If the area designation condition is related to a moving object, it is determined whether or not there is a moving object within a predetermined range of time before and after the time of the area designation condition;
If there is a moving object that corresponds to the area specification condition, the location identification support system allocates processing according to the area specification condition related to the moving object and generates information for narrowing down the position. 6. The location identification support system according to claim 5,
The position estimation unit
If the area designation condition is related to distance, a circle is generated with a center of the moving object corresponding to the area designation condition;
If the area designation condition is related to a direction, an area is generated from the current candidate location to a moving object corresponding to the area designation condition;
If the area specification condition is related to time, a location identification support system generates a total area of a plurality of circles centered on the position of the target object within the time range of the search query. 2. The location identification assistance system according to claim 1,
The position estimation unit
If the area designation condition is related to a stationary object and the area designation condition is related to distance, a circle is generated with a center on a moving object corresponding to the area designation condition;
A location identification support system characterized by generating an area from a current candidate location to a moving object corresponding to the area designation condition if the area designation condition relates to a stationary object and the area designation condition relates to a direction. 3. The location identification support system according to claim 2,
The location identification support system is characterized in that the location estimation unit generates a perceptible area in which environmental information extracted from the search query can be perceived. A location identification assistance method in which a location identification assistance system generates data for identifying a plurality of locations,
the location identification support system is configured by a computer having an arithmetic unit that executes predetermined processing and a storage device that can be accessed by the arithmetic unit,
the storage device stores integrated 3D data including attributes of stationary objects and attributes of moving objects;
The location identification assistance method includes:
a receiving step in which the computing device receives an input of a representation of a moving object recognized by a query subject;
a semantic analysis step in which the computing device identifies search terms for searching the integrated 3D data from the received representation of the moving object, and generates a search query including one or more area specification conditions;
and a position estimation step in which the computing device searches the integrated 3D data using the generated search query, obtains information for narrowing down a position from the representation of the accepted moving object, and generates the information for narrowing down the position. 10. The method for assisting in identifying a location according to claim 9,
the integrated 3D data includes information about the surrounding environment;
In the receiving step, the arithmetic device receives an input of an expression of an environment recognized by an inquiry subject;
In the semantic analysis step, the computing device identifies search terms for searching the integrated 3D data from the received environment representation to generate a search query;
In the position estimation step, the calculation device searches the integrated 3D data using the generated search query and obtains information for narrowing down the position from the received representation of the environment. 10. The method for assisting in identifying a location according to claim 9,
In the position estimation step, the computing device generates screen data for superimposing on a map areas corresponding to the plurality of area designation conditions related to the moving object recognized by the inquiring subject. 10. The method for assisting in identifying a location according to claim 9,
a voice analysis step in which the arithmetic device analyzes the voice acquired by the voice acquisition device;
a video analysis step in which the arithmetic device analyzes the video captured by the video capturing device;
an environment analysis step in which the arithmetic device analyzes the environmental values measured by the environment measuring device;
an information receiving step of generating metadata by combining the analysis results of the audio analysis step, the video analysis step, and the environment analysis step based on location information;
and integrating the generated metadata and three-dimensional map information based on location information to generate the integrated 3D data. 10. The method for assisting in identifying a location according to claim 9,
In the position estimation step, if the area designation condition relates to a moving object, the arithmetic device determines whether there is a moving object within a predetermined range of time before and after the time of the area designation condition, and if there is a moving object that corresponds to the area designation condition, allocates processing according to the area designation condition related to the moving object and generates information for narrowing down the position. The method for assisting in identifying a location according to claim 13,
In the position estimation step, if the area designation condition relates to a stationary object and the area designation condition relates to distance, the calculation device generates a circle centered on a moving object corresponding to the area designation condition; if the area designation condition relates to a stationary object and the area designation condition relates to direction, the calculation device generates an area from a current candidate location to a moving object corresponding to the area designation condition; and if the area designation condition relates to time, the calculation device generates a total area of multiple circles centered on the position of a target object in a time range in the search query. 10. The method for assisting in identifying a location according to claim 9,
In the position estimation step, if the area designation condition relates to distance, the calculation device generates a circle centered on the moving object corresponding to the area designation condition, and if the area designation condition relates to direction, the calculation device generates an area from the current candidate location to the moving object corresponding to the area designation condition. The method for assisting in identifying a location according to claim 10,
The location identification assistance method is characterized in that, in the location estimation step, the calculation device generates a perceptible area in which environmental information extracted from the search query can be perceived.