JP7803795B2 - Disaster information display device, disaster information display system, and control method and program for disaster information display device - Google Patents

Description

The present invention relates to a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program.

Technology for superimposing disaster and other information onto an actually captured image is known. Patent Document 1 describes this type of technology.

Patent Document 1 relates to a synthetic image display device that displays a virtual space image superimposed on a live-action image captured of a real space. The synthetic image display device disclosed in Patent Document 1 includes an image acquisition means for acquiring a live-action image, a position measurement means for measuring the planar position of the image acquisition means, an orientation measurement means for measuring the orientation of the image acquisition means, a storage means for storing a spatial model in which a predetermined area is planarly divided into multiple meshes and flood depths are assigned, an extraction means for extracting a spatial model of the mesh corresponding to the planar position measured by the position measurement means, an image creation means for creating a virtual space image based on the spatial model extracted by the extraction means, and a display means for displaying a superimposed image of the live-action image and the virtual space image. Patent Document 1 also discloses that the image creation means performs arithmetic processing based on the imaging height of the image acquisition means and the orientation of the image acquisition means measured by the orientation measurement means, and creates a virtual space image on the display means so that it appears as a perspective view.

Patent No. 5862865

In Patent Document 1, it was necessary to store in advance on the display device a program for extracting a mesh spatial model corresponding to the planar position measured by the position measurement means and executing the process of creating a virtual space image based on that spatial model, leaving room for improvement in terms of reducing the processing burden on the disaster information display device that displays disaster information.

The present invention aims to provide a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program that can effectively convey disaster information to users while reducing the processing burden.

The present invention relates to a disaster information display device that includes an imaging device that captures images and a display device that displays various information, and that includes: a location acquisition unit that acquires location information; a data acquisition unit that transmits the location information acquired by the location acquisition unit to an external server that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where a disaster is predicted; 3D model data location information indicating the location of the 3D model data; and 3D model data-related information that associates orientation information; and acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; and a display processing unit that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on the display device.

The display processing unit may perform processing to superimpose information about the disaster along with the 3D model data on real-time images captured by the imaging device.

When the display processing unit receives a user operation to change the height information of the position, it may perform processing to change the size of the 3D model data based on the received operation.

When the display processing unit receives a user operation to change the elevation angle, it may perform processing to change the position where the 3D model data is displayed based on the received operation.

The display processing unit may perform processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device, together with a two-dimensional map image that includes the same location as the 3D model data.

The present invention also relates to a disaster information display system comprising: 3D model data configured so that location elevation information is included in map information indicating a predetermined range of alert zones where a disaster is expected; a server storing 3D model data-related information in which 3D model data location information indicating the location of the 3D model data and orientation information are associated; and a disaster information display device including an imaging device that captures images and a display device that displays various information. The disaster information display device comprises: a location acquisition unit that acquires location information; a data acquisition unit that transmits the location information acquired by the location acquisition unit to the server and acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; and a display processing unit that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image with the 3D model data superimposed on it on the display device.

The present invention also relates to a control method for a disaster information display device equipped with an imaging device that captures images and a display device that displays various information, the control method including: a location information acquisition step that acquires location information; a data transmission step that transmits the location information acquired in the location information acquisition step to an external server that stores 3D model data configured such that location elevation information is included in map information that indicates a predetermined range of alert zones where a disaster is expected; 3D model data-related information that associates 3D model data location information indicating the location of the 3D model data with orientation information; a data acquisition step that acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; and a display processing step that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on the display device.

The present invention also relates to a program that causes a computer to execute the following functions: a location information acquisition function that acquires location information; a data transmission function that transmits the location information acquired by the location information acquisition function to an external server that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where disasters are predicted; 3D model data location information indicating the location of the 3D model data; and 3D model data-related information that associates orientation information with azimuth information; a data acquisition function that acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; and a display processing function that determines the orientation of the 3D model data based on the orientation and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image with the 3D model data superimposed on it on a display device.

The present invention provides a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program that can effectively convey disaster information to users while reducing the processing load.

1 is a schematic diagram of a disaster information display system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to the present embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of a user terminal according to the present embodiment. FIG. 2 is a block diagram showing an example of a functional configuration for displaying disaster information of the disaster information display system of the present embodiment. 10A and 10B are schematic diagrams illustrating the relationship between a model search range and 3D model data in the search processing function of this embodiment. 10A and 10B are schematic diagrams illustrating the relationship between a model search range that is wider than the model normal display range of the search processing function of this embodiment and 3D model data. 10 is a flowchart showing an example of the flow of a disaster display process according to the present embodiment. FIG. 2 is a schematic diagram of an AR image displayed on a user terminal according to the present embodiment. FIG. 10 is a schematic diagram illustrating an AR image according to the present embodiment to which area information has been added. FIG. 10 is a schematic diagram illustrating an AR image in which a map image is added to the AR image according to the present embodiment. 10A and 10B are schematic diagrams for explaining position adjustment of an AR image according to the present embodiment.

Embodiments of the present invention will be described below with reference to the drawings.

Figure 1 is a schematic diagram of a disaster information display system 1 according to one embodiment of the present invention. As shown in Figure 1, the disaster information display system 1 of this embodiment includes a GIS (Geographic Information System) 3, a server 10, and a user terminal 20.

GIS3 is a geographic information system that creates map information and related information on a computer. GIS3 and 3D CAD software 4 are used to generate 3D model data of designated alert areas where disasters such as landslides, tornadoes, heavy rain, and tsunamis are predicted, as well as attribute information corresponding to the 3D model data.

In this embodiment, the GIS3 generates 3D model data in a format such as .glb based on map information, which is vector data (two-dimensional data) such as SHAPE format for the restricted area and special restricted area, and elevation data, which is a digital elevation model such as DEM (digital elevation model). That is, the 3D model data is configured by including location elevation information in map information indicating the restricted area. The GIS3 also generates attribute information, which is disaster-related information corresponding to the 3D model data, such as latitude, longitude, elevation, and area name, in text format such as CSV (comma-separated values), along with the 3D model data. The GIS3 also generates 3D model data position information and orientation information indicating the location of the 3D model data. The generated 3D model data and attribute information are sent to the server 10. At this time, the 3D model data, along with the 3D model data position information and orientation information, are sent to the server 10 as 3D model data-related information that is associated with each other. The 3D model data position information may be, for example, information indicating the position of a reference point within the 3D model data. The reference point may be, for example, the center of the 3D model data, or a point other than the center. The GIS 3 may be installed as software on the server 10, or may be installed as software on an external computer, or may function as a server. Map information and elevation data may be obtained, for example, by downloading open data such as government statistics from a website or by using a medium.

Next, we will explain the server 10. The server 10 is a computer that performs processing to send information for displaying disaster information in AR (Augmented Reality) to the user terminal 20 based on 3D model data and attribute information. The server 10 and user terminal 20 are connected so that they can communicate with each other via a network 2 such as the Internet.

Figure 2 is a block diagram showing an example of the hardware configuration of server 10 in this embodiment. As shown in Figure 2, server 10 includes a processor 11, ROM (read-only memory) 12, RAM (random-access memory) 13, auxiliary storage device 14, and communication I/F (interface) 15, with each component connected via a bus or the like.

The processor 11 is the central part of the computer that performs the calculations, control, and other processes necessary for the operation of the server 10, and performs various calculations and processes. The processor 11 is, for example, a CPU (central processing unit), MPU (micro processing unit), SoC (system on a chip), DSP (digital signal processor), GPU (graphics processing unit), ASIC (application specific integrated circuit), PLD (programmable logic device), or FPGA (field-programmable gate array). Alternatively, the processor 11 may be a combination of two or more of these. The processor 11 may also be a combination of these with a hardware accelerator, etc.

Processor 11 controls each component to realize various functions based on programs such as firmware, system software, and application software stored in ROM 12 or auxiliary storage device 14. Processor 11 also executes the processes described below based on these programs. Note that some or all of these programs may be incorporated into the circuitry of processor 11.

ROM 12 and RAM 13 are the main memory devices of the computer, with processor 11 at its core. ROM 12 is a non-volatile memory used exclusively for reading data. ROM 12 stores firmware and other programs among the above. ROM 12 also stores data used by processor 11 when performing various processes. RAM 13 is a memory used for reading and writing data. RAM 13 is used as a work area for storing data temporarily used by processor 11 when performing various processes. RAM 13 is typically a volatile memory.

The auxiliary storage device 14 is, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), or flash memory. The auxiliary storage device 14 stores, among the above programs, for example, system software and application software. The auxiliary storage device 14 also stores data used by the processor 11 when performing various processes, data generated by the processes in the processor 11, various setting values, etc.

The communication I/F 15 is an interface for communicating with external devices such as the user terminal 20.

Next, we will explain the user terminal 20. The user terminal 20 is a disaster information display device that displays disaster information superimposed on an actually captured image. Note that the images referred to here are images that are continuous over time, and include videos.

The hardware configuration of the user terminal 20 will now be described. Figure 3 is a block diagram showing an example of the hardware configuration of the user terminal 20 in this embodiment.

As shown in FIG. 3, the user terminal 20 includes a processor 21, ROM 22, RAM 23, auxiliary storage device 24, communication I/F 25, input device 26, display device 27, imaging device 28, GNSS (global navigation satellite system) device 29, and sensor 30, and each component is connected via a bus or the like.

Of the components of the user terminal 20, the processor 21, ROM 22, RAM 23, auxiliary storage device 24, and communication I/F (interface) 25 are similar in configuration to the processor 11, ROM 12, RAM 13, auxiliary storage device 14, and communication I/F 15 of the server 10, respectively.

The input device 26 is a means for accepting user operations, and the display device 27 is a means for displaying various types of information to the user. In this embodiment, the input device 26 and the display device 27 are configured as touch panel displays. The input device 26 may also be a type that accepts voice input using a microphone. Examples of the various types of information displayed by the display device 27 include images captured by the imaging device 28, 3D model data, attribute information, icons for accepting user operations, etc.

The imaging device 28 is a camera composed of an optical lens, an image sensor, etc. The GNSS device 29 is a means for determining the position of the user terminal 20 using the GNSS. Examples of GNSS that are used include GPS, GLONASS, Galileo, and quasi-zenith satellites. The GNSS device 29 of this embodiment includes an antenna and receives positioning satellite signals from multiple positioning satellites to obtain position information for the user terminal 20. Note that the GNSS device 29 may be configured to be external to the user terminal 20 and obtain position information for the user terminal 20 by wireless or wired communication with a GNSS terminal that receives the positioning satellite signals.

The sensor 30 is composed of various sensors (such as an acceleration sensor and an angular velocity sensor) and a magnetic sensor for measuring the three-dimensional movement of the user terminal 20 itself. The sensor 30 detects the acceleration and angular velocity applied to the user terminal 20 in response to movement and outputs this as sensor information. The orientation of the user terminal 20, etc., is determined based on this sensor information. The sensor 30 also functions as a compass to determine direction.

Next, the process for displaying disaster information will be described. Figure 4 is a block diagram showing an example of the functional configuration for displaying disaster information in the disaster information display system 1 of this embodiment.

As shown in FIG. 4, a model information database 35 and a 3D model database 36 are constructed on the server 10 using an auxiliary storage device 14, an external storage device, etc.

The model information database 35 is a database in which attribute information generated by GIS3 is stored. In this embodiment, the latitude, longitude, and search range of the current location of the user terminal 20 are set as parameters for calling attribute information as return values from the model information database 35. In addition, for example, the model type, model data proxy parameters, model latitude, model longitude, model elevation, model color, natural phenomenon name, area number, area name, county, city, ward, town, village, and district are set as return values.

The 3D model database 36 stores 3D model data-related information that associates 3D model data to be superimposed on images with 3D model data position information and orientation information. The 3D model data is created based on a specific orientation, such as north. In this embodiment, 3D model data in a format such as glb generated by GIS3 is converted into 3D model data in a format such as glTF, and then stored in the 3D model database 36.

The server 10 also includes a location reception unit (location reception function) 31 and a data processing unit (data processing function) 32 as functional units executed by the processor 11.

The position receiving unit 31 performs processing to acquire position information of the user terminal 20 that displays the AR image. The data processing unit 32 performs processing to transmit 3D model data-related information and attribute information to the user terminal 20 for displaying the AR image on the user terminal 20.

The data processing unit 32 of this embodiment includes a search processing unit 33 and a proxy unit 34. The search processing unit 33 is a search API (Application Programming Interface) that identifies 3D model data within a search range set based on the current location of the user terminal 20 and transmits attribute information corresponding to the identified 3D model data to the user terminal 20. The search processing unit 33 identifies 3D model data-related information, for example, by comparing the location information of the user terminal 20 with the 3D model data location information.

An example of a method for searching for 3D model data using the search processing unit 33 will be described with reference to Figure 5. Figure 5 is a schematic diagram illustrating the relationship between the model search range and 3D model data in the search processing function of this embodiment. Figure 5 shows a search range centered on the latitude and longitude of the current location, and a model normal display range in which the 3D data model can be displayed correctly in the AR image. In Figure 5, the search range is shown as a solid rectangle, with its center set at the current location. The model normal display range is shown as a dashed circle, with its center set at the current location. In the example of Figure 5, the model search range is set so that it is within the model normal display range.

The 3D model data includes alert zones and special alert zones. Alert zones are areas where users should be alerted, and a model origin is set as a reference. Special alert zones are areas where users should be even more alert than alert zones, and a separate model origin is set. Special alert zones are areas inside alert zones.

If either the model origin of the restricted area or the special restricted area falls within the search range, that 3D model data will be displayed in the AR image. In Figure 5, a total of four 3D model data items are shown inside the normal model display range: 3D model A, 3D model B, 3D model C, and 3D model D. 3D model A is included in the display range because both the model origin of the restricted area and the model origin of the special restricted area are inside the model search range. 3D model B is included in the display range because the model origin of the restricted area is outside the search range, but the model origin of the special restricted area is inside the model search range. 3D model C is included in the display range because the model origin of the special restricted area is outside the search range, but the model origin of the restricted area is inside the model search range. On the other hand, 3D model D is not included in the display range because both the model origin of the restricted area and the model origin of the special restricted area are outside the model search range.

In the example of Figure 5, the model search range is set so that it is included in the model normal display range, but the model search range may also be set so that it includes the outside of the model normal display range. Figure 6 is a schematic diagram illustrating the relationship between the model search range, which is wider than the model normal display range of the search processing function of this embodiment, and the 3D model data.

In the example of Figure 6, a total of three pieces of 3D model data are displayed inside the model search range: 3D model E, 3D model F, and 3D model G. 3D model E and 3D model F are both inside the model normal display range, and both the model origin of the alert area and the model origin of the special alert area are inside the model search range, so they are the 3D model data to be displayed correctly.

Although the model origin of the restricted area is outside the model search range, 3D model G is included in the display range because the model origin of the restricted area is inside the model search range. However, because both the model origin of the restricted area and the model origin of the special restricted area are outside the normal model display range, the 3D model data will not be displayed correctly on the user terminal 20. For example, it will be displayed as a distorted shape on the AR image.

Next, we will explain the proxy unit 34. The proxy unit 34 transmits corresponding 3D model data to the user terminal 20 in response to a data request from the user terminal 20.

In this embodiment, if a cache file for the corresponding 3D model data does not exist, the proxy unit 34 obtains the 3D data model from the 3D model database 36, transmits it to the user terminal 20, and saves the 3D model data as a cache file. If a cache file for the corresponding 3D model data exists, the proxy unit 34 transmits the 3D model data from the cache file to the user terminal 20.

Next, the functions of the user terminal 20 will be described. The user terminal 20 has functional units executed by the processor 21, including a position acquisition unit (position acquisition function) 41, a data acquisition unit (data acquisition function) 42, and a display processing unit (display processing function) 43.

The position acquisition unit 41 executes a process to determine the current position of the user terminal 20 based on positioning information from satellites (e.g., GPS satellites) received by the GNSS device 29. Information indicating the current position includes latitude, longitude, altitude, etc.

The data acquisition unit 42 executes a process to acquire from the server 10 the data necessary to superimpose an image onto a live-action image. The data that the user terminal 20 acquires from the server 10 includes, for example, 3D model data-related information, attribute information linked to the 3D model data included in the 3D model data-related information, and a program for displaying the 3D model data on a web browser.

The data acquisition unit 42 downloads from the server 10 multiple 3D model data items that exist within a predetermined range (model search range or model normal display range) around the user terminal 20 in order to reproduce a 3D scene on the user terminal 20.

The display processing unit 43 executes processing to display on the display device 27 real-time images captured by the imaging device 28, as well as 3D model data and attribute information superimposed on the images. In this embodiment, the display processing unit 43 calls the imaging device 28 on a web browser and generates an AR image by superimposing an image based on the 3D model data on a real-life image.

The superimposition process performed by the display processing unit 43 will now be described. The display processing unit 43 calculates the horizontal orientation of the viewpoint from the horizontal azimuth value obtained from output information from the magnetic sensor and other sensors of the sensors 30 of the user terminal 20, and calculates the vertical orientation of the viewpoint based on the elevation and depression angles obtained from output information from the acceleration sensor and other sensors. The display processing unit 43 then calculates the horizontal position at which to display the 3D model data from the latitude and longitude of the current position of the GNSS device 29, and calculates the vertical position at which to display the 3D model data from the altitude (altitude) of the current position of the GNSS device 29. In other words, the display processing unit 43 determines the orientation of the 3D model data based on the azimuth and elevation angle of the imaging device 28.

In this embodiment, the display processing unit 43 uses the web browser of the user terminal 20 to display the AR image on the display device 27. Specifically, the display processing unit 43 superimposes the orientation-determined 3D model data on a real-time image captured by the imaging device 28, and displays the real-time image with the superimposed 3D model data in the web browser displayed on the display device 27. Web AR can be achieved by downloading a program for displaying the 3D model on the web browser together with the 3D model data from the server 10. In this embodiment, the display processing unit 43 draws the AR image when the server 10 is accessed and the 3D model data is downloaded, such as when a web page is opened or reloaded, but the drawing timing is not limited to this timing.

When the imaging direction of the imaging device 28 changes, the display processing unit 43 performs viewpoint manipulation to change the superimposed image, thereby changing the superimposed image. The viewpoint manipulation is performed based on the newly acquired horizontal and vertical orientations. The superimposed image changes based on this viewpoint manipulation.

The display processing unit 43 also performs processing to display information related to disaster information in AR images, as well as processing to display setting images necessary to correct the user terminal 20's current satellite-based position and orientation based on the compass function.

The AR image display process is initiated, for example, by operating an AR button on a website displaying disaster information. Next, the process flow for displaying AR images will be explained with reference to Figure 7. Figure 7 is a flowchart showing an example of the disaster display process flow in this embodiment.

When the disaster display process is initiated, the location acquisition unit 41 of the user terminal 20 acquires device location information indicating the current location of the user terminal 20, determines the device location, and transmits it to the server 10 (step S101). The search processing unit 33 of the server 10 determines whether or not it has received device location information indicating the current location of the user terminal 20, and if it determines that it has received device location information, it proceeds to step S103 (step S102; Yes). The search processing unit 33 of the server 10 continues the determination process until it receives device location information (step S102; No).

The search processing unit 33 of the server 10 extracts 3D model data-related information based on the received terminal location information and the stored 3D model data location information. The search processing unit 33 then obtains attribute information for the 3D model data included in the identified 3D model data-related information from the model information database 35 (step S103), and transmits the obtained attribute information to the user terminal 20 (step S104).

The user terminal 20 sends a data request to the server 10 to obtain 3D model data based on the received attribute information (step S105).

The proxy unit 34 of the server 10 determines whether a data request has been received, and if it determines that a data request has been received, proceeds to step S107 (step S106; Yes). The proxy unit 34 of the server 10 continues the determination process until a data request is received (step S106; No).

In step S107, the proxy unit 34 of the server 10 transmits the 3D model data acquired in response to the data request to the user terminal 20 (step S107). The user terminal 20 acquires the 3D model data from the server 10 (step S108) and executes a process of displaying disaster information on the display device 27 using the acquired 3D model data (step S109).

Next, we will explain the process of displaying an AR image on the display device 27 of the user terminal 20. Figure 8 is a schematic diagram of an AR image displayed on the user terminal 20 of this embodiment. In Figure 8, a warning area image 110 indicating a warning area and a special warning area image 111 indicating a special warning area are superimposed by the display processing unit 43 on an actual image captured by the imaging device 28 of the user terminal 20.

The display processing unit 43 also performs processing to display compass information 101, text messages 102, GPS reception strength information 103, sensor information display switch 104, and various setting display buttons 105.

The compass information 101 is an image that shows the direction of the compass function. The text message 102 is an image that displays information about the disaster at the current location in text. In this example, the text message 102 displays "May be in a landslide warning area" because the user terminal 20 is located in a warning area. That is, as shown in FIG. 8, the display processing unit 43 may perform processing to superimpose information about the disaster along with 3D model data on real-time images captured by the imaging device 28. The GPS reception strength information 103 is an image that visually displays the reception strength of satellites received by the user terminal 20. For example, the display processing unit 43 makes visual changes such as reducing the number of arcs representing radio waves in the image when the reception strength decreases.

The sensor information display switch 104 is a tap operation part for determining whether or not to display accuracy information such as latitude, longitude, altitude, and direction, which will be described later (see Figure 11 below). In Figure 9, the sensor information display switch 104 is OFF, and accuracy information is not displayed. The various settings display button 105 is a tap operation part for displaying a screen on which the user can make various settings.

Next, we will explain the information added to the AR image. Figure 9 is a schematic diagram of the AR image of this embodiment with area information 120 added. The area information 120 is information acquired by the user terminal 20 from the server 10 as attribute information. In this embodiment, tapping the center of the AR image shown in Figure 8 adds the area information 120 shown in Figure 9.

The area information 120 includes a selection image 121 for selecting an area, a disaster information image 122 showing disaster-related information, and a file link display image 123 for displaying disaster information files in, for example, PDF (portable document format) format.

The selection image 121 is an image displayed in the operation section for the user to select an area when multiple areas overlap around the current location of the user terminal 20. The selection image 121 displays multiple areas, and when the user taps on one of the areas, the disaster information image 122 changes to one corresponding to the selected area.

The disaster information image 122 is an image that displays information about the disaster set for the area and information about the area. In this example, a debris flow is set as the natural phenomenon for this area, and a landslide warning area is set as the designated type. Information about the area's location, such as the area number, is also displayed.

The file link display image 123 is an image for displaying the contents of a notice document regarding a disaster in an area based on an administrative agency, etc. In this example, when the user taps the file link display image 123, the data in PDF format is displayed on the display device 27.

Next, a process for displaying a two-dimensional map image 130 together with an AR image will be described. Figure 10 is a schematic diagram in which a map image 130 has been added to an AR image of this embodiment. In Figure 10, the map image 130 is displayed below the AR image. In this embodiment, the user taps the various settings display button 105 to open a settings screen, and the map image 130 and the AR image are displayed side by side vertically by performing a tap operation to set the map image display. In other words, the display processing unit 43 may perform a process for displaying both an image in which 3D model data is superimposed on a real-time image captured by the imaging device 28, and a two-dimensional map image 130 that includes the same location as the 3D model data. Note that the map image 130 and AR image may also be displayed side by side horizontally on the screen of the display device 27.

The display processing unit 43 also executes processing to display an arrow image 131 indicating the current position of the user terminal 20 as well as the orientation of the user terminal 20. The orientation of the user terminal 20 indicates the imaging direction of the imaging device 28 (the orientation of the user terminal 20) obtained from the output information from the sensor 30.

Next, we will explain the process of adjusting the position of an AR image. Figure 11 is a schematic diagram for explaining the position adjustment of an AR image in this embodiment. Figure 11 shows a position adjustment controller 150 along with a compass information image 140.

The compass information image 140 is accuracy information that indicates, for example, the accuracy of the compass (latitude, longitude, altitude, and direction). The compass information image 140 also displays the XY error, which is an error in planar view, the Z error, which is an error in altitude, and the C error, which is an error in direction. This compass information image 140 is displayed by turning on the sensor information display switch 104.

The position adjustment controller 150 is an image displayed on the operation section for adjusting the position. The position adjustment controller 150 is composed of a height adjustment section 151 for adjusting the height and an orientation adjustment section 152 for adjusting the orientation.

The height adjustment section 151 is an image displayed on the operation section for increasing or decreasing the height, and the user can manually adjust the altitude (height) by tapping this section. This also makes it possible to correct errors based on the output information of the GNSS device 29.

The orientation adjustment unit 152 is an image displayed on an operation section for adjusting the orientation of the user terminal 20 + or -, allowing the user to manually adjust the orientation by tapping on this section. Errors based on the output information of the sensor 30 can also be corrected. That is, when the display processing unit 43 receives a user operation to change the elevation angle, it performs processing to change the position where the 3D model data is displayed based on the received operation. It is also possible to obtain a bird's-eye view of the current position and its surroundings by tapping the height adjustment unit 151 and setting the height much higher than the ground and pointing the imaging device 28 downward. That is, when the display processing unit 43 receives a user operation to change the altitude, it performs processing to change the position where the 3D model data is displayed based on the received operation.

As described above, the user terminal 20 includes an imaging device 28 that captures images and a display device 27 that displays various information. The user terminal 20 also includes a location acquisition unit 41 that acquires location information indicating the current location of the user terminal 20, a data acquisition unit 42 that transmits the location information acquired by the location acquisition unit 41 to an external server 10 that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where disasters are predicted, and 3D model data-related information that associates 3D model data location information indicating the location of the 3D model data with orientation information, and acquires 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information, and a display processing unit 43 that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image on which the 3D model data is superimposed on the display device 27.

As a result, the generation of 3D model data is not performed on the server 10 or user terminal 20 side, effectively reducing the load on the server 10 or user terminal 20, while allowing the user to intuitively recognize the risk of disasters using AR images on the display device 27 of the user terminal 20. Furthermore, since AR images can be displayed on a web browser that is less device-dependent, it is possible to effectively make a wide range of users aware of the risk of disasters.

In addition, in this embodiment, the display processing unit 43 performs processing to superimpose information about the disaster along with 3D model data onto real-time images captured by the imaging device 28.

This allows users to be more aware of disaster risk and location information for each area, such as alert zones and special alert zones, in more detail.

In addition, in this embodiment, when the display processing unit 43 receives a user operation to change the height information of a position, it performs processing to change the size of the 3D model data based on the received operation.

As a result, even if the orientation cannot be accurately acquired due to a malfunction of the sensor 30 of the user terminal 20, causing the superimposition position of the 3D model data to shift horizontally, the superimposition position can be returned to the appropriate position by adjustment.

In addition, in this embodiment, when the display processing unit 43 receives a user operation to change the elevation angle, it performs processing to change the position at which the 3D model data is displayed based on the received operation.

As a result, even if the altitude cannot be accurately obtained due to a malfunction of the GNSS device 29 of the user terminal 20, causing the superimposition position of the 3D model data to shift vertically, the superimposition position can be returned to the appropriate position by adjustment.

In addition, in this embodiment, the display processing unit 43 performs processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device 28, together with a two-dimensional map image 130 that includes the same location as the 3D model data.

This allows you to see your current location from a bird's-eye view through the map image 130. You can also check the map image 130 to see if there is an error in the direction relative to the azimuth.

The disaster information display system 1 of this embodiment also includes: a server that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where a disaster is expected; 3D model data location information indicating the location of the 3D model data; and 3D model data-related information that associates orientation information; and a user terminal 20 that includes an imaging device 28 that captures images and a display device 27 that displays various information. The user terminal 20 also includes a position acquisition unit 41 that acquires location information; a data acquisition unit 42 that transmits the location information acquired by the position acquisition unit 41 to the server 10 and acquires 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information; and a display processing unit 43 that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image with the 3D model data superimposed on the display device 27.

The control method of the user terminal 20 as a disaster information display device also includes: a location information acquisition step of acquiring location information; a data transmission step of transmitting the location information acquired by the location acquisition unit 41 to an external server 10 that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where a disaster is expected; 3D model data-related information that associates 3D model data location information indicating the location of the 3D model data with orientation information; a data acquisition step of acquiring 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information; and a display processing step of determining the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposing the determined 3D model data on a real-time image being captured by the imaging device 28, and displaying the real-time image on which the 3D model data is superimposed on the display device 27.

The program of this embodiment also causes the computer to execute a location information acquisition function that acquires location information, a data transmission function that transmits the location information acquired by the location acquisition unit 41 to an external server 10 that stores 3D model data configured such that location elevation information is included in map information indicating a predetermined range of alert zones where disasters are expected, and 3D model data-related information that associates 3D model data location information indicating the location of the 3D model data with orientation information, a data acquisition function that acquires 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information, and a display processing function that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image on which the 3D model data is superimposed on the display device 27.

The control method and program for the disaster information display system 1 and the user terminal 20 do not generate 3D model data on the server 10 or user terminal 20 side, effectively reducing the load on the server 10 or user terminal 20, while allowing users to intuitively recognize the risk of disasters using AR images on the display device 27 of the user terminal 20. Furthermore, because AR images can be displayed on a web browser that is less device-dependent, a wide range of users can be effectively made aware of the risk of disasters.

The above describes an embodiment of the present invention, but the present invention is not limited to the above embodiment and can be modified as appropriate.

For example, the display processing unit 43 can also add evacuation information related to evacuation to the AR image of the above embodiment. More specifically, the server 10 may store evacuation image information and evacuation text information indicating the location of evacuation shelters, evacuation routes, evacuation directions, etc., and the user terminal 20 may download the evacuation image information and evacuation text information from the server 10 and display them on the AR image.

In addition, the display processing unit 43 may acquire information on disasters that have actually occurred in the past, real-time weather information, the probability of disaster occurrence and risk level based on weather information, etc. from an external website and display this information on the AR image in the above embodiment.

Furthermore, the display processing unit 43 may be configured to additionally display a guide image that serves as a reference for position adjustment while the above-mentioned position adjustment controller 150 is displayed.

Furthermore, the display processing unit 43 may also display additional information other than disaster information, such as information for travelers, in the AR image.

Furthermore, the series of processes in the above-described embodiments and variations can be executed by hardware or software. When the series of processes are executed by software, the programs that make up the software are installed onto a computer or the like from a network or recording medium. The computer may also be a computer built into dedicated hardware. The computer may also be a computer that can execute various functions by installing various programs, such as a general-purpose personal computer.

1 Disaster information display system 10 Server 20 User terminal (disaster information display device)
27 Display device 28 Imaging device 41 Position acquisition unit 42 Data acquisition unit 43 Display processing unit

Claims (8)

A disaster information display device including an imaging device that captures images and a display device that displays various information,
a location acquisition unit that acquires location information;
a data acquisition unit that transmits the location information acquired by the location acquisition unit to an external server that stores 3D model data configured such that location height information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data location information that indicates the location of the 3D model data, and 3D model data-related information that is associated with orientation information, and acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information;
a display processing unit that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and
A disaster information display device comprising: The display processing unit
The disaster information display device according to claim 1 , wherein the disaster information display device performs a process of superimposing the disaster information together with the 3D model data on a real-time image being captured by the imaging device. The display processing unit
The disaster information display device according to claim 1 or 2, wherein when a user's operation to change the height information of the position is accepted, a process of changing the size of the 3D model data is performed based on the accepted operation. The display processing unit
The disaster information display device according to claim 1 or 2, wherein, when a user's operation to change the elevation angle is accepted, a process of changing the position where the 3D model data is displayed is performed based on the accepted operation. The display processing unit
3. The disaster information display device according to claim 1, wherein the disaster information display device performs processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device, and a two-dimensional map image including the same location as the 3D model data. a server storing 3D model data configured by including height information of a position in map information indicating a warning zone within a predetermined range where a disaster is expected, 3D model data position information indicating the position of the 3D model data, and 3D model data related information in which direction information is associated with the 3D model data;
a disaster information display device including an imaging device for capturing images and a display device for displaying various types of information;
The disaster information display device includes:
a location acquisition unit that acquires location information;
a data acquisition unit that transmits the location information acquired by the location acquisition unit to the server, and acquires the 3D model data related information extracted by the server based on the location information and 3D model data location information;
a display processing unit that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and
A disaster information display system comprising: A method for controlling a disaster information display device that includes an imaging device that captures images and a display device that displays various information,
a location information acquisition step of acquiring location information;
a data transmission step of transmitting the location information acquired in the location information acquisition step to an external server storing 3D model data configured such that location height information is included in map information indicating a warning zone within a predetermined range where a disaster is expected, 3D model data location information indicating the location of the 3D model data, and 3D model data related information in which direction information is associated; and
a data acquisition step of acquiring the 3D model data related information extracted by the server based on the location information and the 3D model data location information;
a display processing step of determining the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposing the determined 3D model data on a real-time image being captured by the imaging device, and displaying the real-time image on which the 3D model data is superimposed on a display device; and
A method for controlling a disaster information display device, comprising: On the computer,
A location information acquisition function that acquires location information;
a data transmission function that transmits the location information acquired by the location information acquisition function to an external server that stores 3D model data configured such that location height information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data location information that indicates the location of the 3D model data, and 3D model data related information that is associated with direction information; and
a data acquisition function for acquiring the 3D model data related information extracted by the server based on the location information and the 3D model data location information;
a display processing function that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and
A program that executes the following.