JP7065477B2 - Disaster situation judgment system and disaster judgment flight system - Google Patents

Description

The present invention relates to a disaster situation determination system and a disaster determination flight system.

Conventionally, various researches and developments have been carried out regarding disasters. For example, Patent Document 1 (Patent No. 5760155) discloses a search support system that improves search efficiency in order to realize rescue within 72 hours.

The search activity support system described in Patent Document 1 is a system that supports search activities, and includes a first terminal installed in a disaster countermeasures headquarters and a management device installed in a data center, which are connected to each network. The first terminal has a second terminal carried by the search commander and a third terminal carried by the searcher, and the first terminal outputs map data including the disaster occurrence area to the management device based on the occurrence of the disaster. A transmission means for transmitting a signal requesting transmission to, a reception means for receiving map data sent from a management device, a display means for displaying a map based on the received map data, and a search on the displayed map. Receives input means to create a search map by specifying a range, transmission means to send the search map to the management device, and search requirements sent by the management device, which are the time and human requirements required for the search. A search to be transmitted after displaying the position of each searcher on the search map based on the receiving means to be performed, the display means to display the received search requirements, and the position information of each searcher received from the third terminal by the management device. It has a receiving means for receiving a map and a display means for displaying the received search map, and the management device acquires specific map data based on the map data transmission request signal received from the first terminal. Means, transmission means for transmitting the acquired map data to the first terminal, storage means for storing the search map received from the first terminal, and search requirements within the search range specified based on the stored search map. Is stored in the storage means based on the processing means for obtaining the data, the transmitting means for transmitting the searched search requirements to the first terminal or the first terminal and the second terminal, and the position information of each searcher received from the third terminal. A processing means for adding a symbol indicating the position of each searcher to the search map, and a transmission means for transmitting the search map or the search map after the symbol is added to at least the first terminal and the second terminal. The second terminal has a receiving means for receiving the search map transmitted by the management device, a display means for displaying the received search map, and a receiving means for receiving the search requirement transmitted by the management device. It has a display means for displaying the received search requirements and a communication means for the search commander and the searcher to talk between the third terminal, and the third terminal has a positioning means such as GPS and its positioning. It has a transmission means for transmitting the position information obtained by the means to the management device, and a communication means for the search commander and the searcher to talk with the second terminal.

Further, Patent Document 2 (Japanese Unexamined Patent Publication No. 2018-63707) relates to, for example, a dental panoramic X-ray image of a found body and dentistry of a person whose identity is to be confirmed before life as an aid to identification work utilizing dental information. An image analysis system that makes it possible to easily obtain identity verification information by collating with information, and further, in order to improve the accuracy of image analysis processing, an image analysis system with high reliability regarding tooth number information. Is disclosed.

In the image analysis system described in Patent Document 2, at least in an image analysis system including a management device server and a management system having a management device database, and an analysis system having an analysis device processor and an analysis device database, a management device server and / or A means for inputting the first information related to the image type to be image analysis processed by the analyzer processor, and the first information by the management device server and the management device database and / or the analyzer processor and the analyzer database. Input the means for storing as the first storage in the management device server and / or the analysis device database, and the second information related to the tooth number assigned to the image according to the image type by the management device server and / or the analysis device processor. And / or means to store the second information in the management device server and / or the management device database as a second storage by the management device server and the management device database and / or by the analysis device processor and the analysis device database. , A means for inputting a third information relating to the tooth condition for each tooth number by the management device server and / or the analysis device processor, and the third information by the management device server and the management device database and / or with the analysis device processor. It is provided with a means for storing as a third storage in the management device server and / or the analyzer database by the analyzer database.

Further, Patent Document 3 (Japanese Unexamined Patent Publication No. 2017-216757) discloses a system monitoring device and a program for predicting the next state of the target system by reflecting information obtained from the outside of the target system.

The system monitoring device described in Patent Document 3 includes a storage unit that stores state transition information including a plurality of states of the target system and a path of state transition from one state among the plurality of states to another state. The current state of the target system is determined from the input unit that accepts the input of the measurement information that is the information measured by the sensor provided in the target system and the peripheral information related to the target system, and the measurement information received by the input unit. From the current state based on the state detection unit to be detected, the current state of the target system detected by the state detection unit, the state transition information stored in the storage unit, and the peripheral information received by the input unit. A transition probability calculation unit that calculates the transition probability to the state of, a transition state determination unit that determines the next state that transitions from the current state based on the transition probability, and an output unit that outputs the determination result of the transition state determination unit. And.

Further, Patent Document 4 (Japanese Unexamined Patent Publication No. 2017-181870) discloses an information processing apparatus that realizes more accurate dynamic update of map information in response to changes in the real world.

The information processing apparatus described in Patent Document 4 is based on an acquisition unit that acquires observation information related to a real-world unit space from one or a plurality of sensors, and information related to inconsistency between a reference map and a real-world unit space. It also includes a communication unit that transmits the observation information acquired in the unit space.

Finally, Patent Document 5 (Japanese Unexamined Patent Publication No. 2017-135545) discloses a network management system and a network management method that can quickly use services without separately preparing a network for the integrated management system.

The network management system described in Patent Document 5 is connected to a management system that manages an access node accessed by a user terminal and a service node that provides a service, and constructs a network between the access node and the service node. An integrated management system and a network management system that holds an authentication key for using services, and authenticates the user terminal when the access node receives a service setting request including the authentication key from the user terminal. A communication management unit that is placed on the access node and sets a control session that is different from the service session on the access node after the authentication unit authenticates the user terminal, and a service of the user terminal that the authentication unit authenticates. In the control session formed by the communication management unit, the setting request unit that sends the setting request from the access node to either the integrated management system or the management system, and the service setting request sent by the setting request unit are received and combined with the user terminal. It has an internal request receiving unit that builds a service that forms a service session that enables communication with the service node in the access node.

Patent Document 6 (Japanese Unexamined Patent Publication No. 2018-17103) discloses an information processing apparatus, an information processing method, and a program for comprehensively determining the state of a surface such as a building.

The information processing apparatus described in Patent Document 6 is an image in which a learning unit that performs deep learning and a region including a surface are imaged based on teaching data indicating the state of the surface in an image obtained by capturing the surface. A state determination for determining the state of a surface in an image acquired by the image acquisition unit based on the learning results of the image acquisition unit that acquires an image associated with the position information indicating the captured position and the learning unit. It is equipped with a part.

Japanese Patent No. 5760155 Japanese Unexamined Patent Publication No. 2018-63707 JP-A-2017-216757 Japanese Unexamined Patent Publication No. 2017-181870 JP-A-2017-135545 Japanese Unexamined Patent Publication No. 2018-17103

However, in the techniques of Patent Documents 1 to 6, although there is a description of deep learning (deep learning) technique and disaster map creation technique, deep learning requires a large amount of data and many disasters do not occur frequently. There was a problem that it could not be dealt with.

In other words, there is a problem that it takes a lot of time (disasters in the time of decades to hundreds of years) and a lot of labor to collect a large number of both the type of disaster and the data immediately after the occurrence. , Was difficult to achieve.

A main object of the present invention is to provide a disaster situation determination system and a disaster determination flight system for determining a disaster state from a disaster image after a disaster in real time.

(1)
The disaster situation determination system that follows one aspect consists of a recording unit that records an artificially created disaster state, a deep learning unit that learns the disaster state using the disaster image recorded in the recording unit, and a deep unit. It includes a display unit that determines the disaster state of the disaster image after the disaster by the learning unit and displays a disaster situation map.

In this case, the disaster situation can be analyzed in real time and the disaster situation map can be displayed. That is, in the conventional deep learning (deep learning), an actual disaster disaster image as teacher data is required, but many disaster disaster images cannot be obtained. Therefore, the present inventor has found that by artificially creating a disaster image showing a disaster state and using the disaster image as teacher data, the disaster state can be determined from the disaster image after the disaster in real time. ..

(2)
In the disaster situation determination system according to the second invention, the disaster image showing the disaster state artificially created in the disaster situation determination system according to one aspect may be formed in three dimensions.

In this case, since the artificially created disaster image showing the disaster state is composed of three dimensions, it is possible to accurately determine the actual disaster image after the disaster.

(3)
The disaster situation determination system according to the third invention further includes a route guidance instruction unit in the disaster situation determination system according to the first aspect or the second invention, and the route guidance instruction unit can pass based on the disaster situation map. The route may be displayed on the display unit.

In this case, since the route guidance instruction unit can display a passable route based on the disaster situation map, it is possible to safely move relief supplies, rescue, and the like.

(4)
The disaster situation determination system according to the fourth invention is from one aspect to the disaster situation determination system according to the third invention. The artificially created disaster image showing the disaster state is a collapsed building, a collapsed bridge, or a collapsed. It may include at least one or more of mountains, collapsed embankments, collapsed roads, collapsed tunnels, earthquakes, tsunamis, fires, floods, cracks, and electrical line failures.

In this case, the artificially created disaster image shows the concrete collapsed building, collapsed bridge, collapsed mountain, collapsed embankment, collapsed road, collapsed tunnel, earthquake, tsunami, fire, flood damage. Since it includes at least one or a plurality of cracks, cracks in the ground, and defects in electric lines, the disaster situation can be easily and accurately determined.

(5)
The disaster situation determination system according to the fifth invention may display the probability of the disaster situation on the display unit in the disaster situation determination system according to the fourth aspect from the first aspect.

In this case, the deep learning unit can display the probability of a disaster situation, so it should display whether the building has completely collapsed or whether it may collapse further due to a secondary disaster in the future. Can be done. That is, the secondary disaster may occur again in the case of an earthquake, fire, flood, tsunami, or the like. Naturally, secondary disasters, tertiary disasters, etc. are included.

(6)
The disaster situation determination system according to the sixth invention further includes a weather display unit in the disaster situation determination system according to the fifth aspect from the first aspect, and the deep learning unit responds to the weather forecast information from the weather display unit. , The weather information of the disaster situation map may be displayed on the display unit.

In this case, since the weather information is also displayed on the display unit, it is possible to predict the deterioration of the disaster situation.

(7)
The disaster situation determination system according to the seventh invention is from one aspect to the disaster situation determination system according to the sixth invention. It may be divided into two, and after determining the disaster state of each image, it may be combined.

In this case, since the disaster video is divided into individual images, the image processing speed can be increased and the disaster situation determination accuracy can be improved.

(8)
The disaster determination flight system according to the other aspects includes the disaster situation determination system according to the first aspect to the seventh aspect, and a flying object equipped with a photographing device and capable of transmitting a disaster image.

In this case, the disaster determination flight system can transmit the disaster situation of the disaster image from the flying object, and the disaster situation determination system can easily display the disaster situation map.

It is a schematic diagram which shows an example of the whole structure of the disaster situation determination system which concerns on this invention. It is a schematic diagram which shows an example of a teacher data. It is a schematic diagram which shows an example of making a disaster model. It is a schematic diagram which shows an example of disaster map data. It is a schematic diagram which shows an example of a route map. It is a schematic diagram which shows an example of the whole structure of the disaster determination flight system which concerns on this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of them will not be repeated.

<Embodiment>
(Disaster situation judgment system 100)
FIG. 1 is a schematic diagram showing an example of the overall configuration of the disaster situation determination system 100 according to the present invention.

As shown in FIG. 1, the disaster situation determination system 100 includes a recording unit 110, a deep learning unit 120, and a display unit 130.
When a disaster occurs, the disaster information determination system 100 according to the present invention determines the actual disaster level SL from the disaster model 150 of deep learning described later, and creates the disaster map data 240.

The disaster in the present invention includes natural disasters and man-made disasters. Specifically, meteorological disasters, rain, concentrated heavy rains, floods, river floods, landslides, slope collapses, landslides, landslides, landslides, tornadoes, high tides, avalanches, snowstorms, lightning, hail, earthquakes, tsunamis, earthquake fires, Eruption, fountain, lava flow, pyroclastic flow, mud flow, large-scale accident, fire, train accident, aviation accident, marine accident, traffic accident, explosion accident, coal mine accident, oil spill accident, chemical substance pollution accident, nuclear accident, terrorism, It includes wars, war damages, and any other accidents.

Further, it is generally known that in order to construct a disaster model 150 for deep learning, at least tens of thousands of teacher data 140 for the disaster model 150 are required. Therefore, it is common knowledge that it will take hundreds of years, not just decades, to create a highly accurate disaster model 150.
In the present embodiment, the disaster model 150 may be learned using teacher data 140 of 500 types or more and 100,000 types or less for each disaster to be determined, and learning using teacher data 140 of 2000 types or more and 25,000 types or less. It is preferable to learn using the teacher data 140 of 8000 types or more and 12500 types or less. By learning the lower limit or more in this way, it is possible to determine the disaster with high accuracy. On the other hand, if the above upper limit is exceeded, the learning effect is saturated.

(Recording unit 110)
In the present embodiment, the recording unit 110 is composed of a general recording device. In the present embodiment, the recording device is used, but the present invention is not limited to this, and a recording device such as a cloud may be used.
Further, the recording unit 110 records a plurality of artificially created disaster data 200. Further, the recording unit 110 records the disaster model 150 deeply learned in the deep learning unit 120.
Further, the recording unit 110 records the map data 230 which is the basis for creating the disaster map. Further, the disaster map data 240 and / and the route map 250 created by the deep learning unit 120 are recorded.

(Deep learning unit 120)
The deep learning unit 120 performs deep learning (deep learning) based on the teacher data 140 recorded in the recording unit 110 by the multi-layer neural network. That is, the deep learning unit 120 creates a disaster model 150 for each type of disaster in advance as the teacher data 140 recorded in the recording unit 110.

As the deep learning unit 120 in the present embodiment, YOLO (Redmon, Joseph, et al. "YOLOv3: An Incremental Improvement" arXiv preprint arXiv: 1804.02767) can be used. YOLO divides the entire image into a grid in advance, classifies objects for each area (classification) and calculates the bounding box (Bounding Box Regression), and is constructed with one network, so it is highly accurate and high-speed. Can be processed.
In addition, the width of the disaster image input by YOLO is set to 320 pixels or more and 10,000 pixels or less, the height is set to 240 pixels or more and 5000 pixels or less, and the width and height of the image size before folding are both 104 pixels or more and 1664 pixels or less. It is more preferable to set with and set with 208 pixels or more and 832 pixels or less.
As a result, as will be described later, the determination using the disaster model 150 can be performed with high accuracy for the actual disaster image.

In the present embodiment, YOLO is used as the deep learning unit 120, but the present invention is not limited to this, and R-CNN, SPPnet, or any other engine (computer program) may be used.
Further, in the present embodiment, the disaster model 150 is recorded in the recording unit 110, but the present invention is not limited to this, and the disaster model 150 may be recorded in the deep learning unit 120.

(Display unit 130)
The display unit 130 includes a liquid crystal display unit or a plasma display. Further, the display unit 130 is a device having a communication unit 131 and capable of communicating with the deep learning unit 120.
The display unit 130 can display the disaster map data 240 obtained from the deep learning unit 120. Further, the display unit 130 can display the route map 250 obtained from the deep learning unit 120.
In the present embodiment, the display unit 130 is composed of a liquid crystal display unit or a plasma display, but the present invention is not limited to this, and any mobile terminal, mobile phone, smartphone, tablet terminal, head-mounted display, or the like can be used. The display unit 130 is included.

(Teacher data 140)
Next, FIG. 2 is a schematic diagram showing an example of teacher data 140.

As shown in FIG. 2, the teacher data 140 is composed of a plurality of artificially created teacher data 140. That is, various teacher data 140s are formed according to the state of each disaster. In the present embodiment, the teacher data 140 is composed of a three-dimensional image. Therefore, unlike a two-dimensional image, the amount of information can be increased.

For example, the teacher data 140 can be artificially created by using software such as OpenGL.
Further, in the present embodiment, the teacher data 140 is artificially created, but if a disaster actually occurs in the past and there is video data of the disaster, the past video is used. Data may be added to obtain teacher data 140.
Further, the teacher data 140 is composed of a plurality of artificially created teacher data 140, but the individual teacher data 140 may be a two-dimensional image in which the video data of the disaster may be separated in a short time. May be good. For example, in the case of the teacher data 140 in which the video time of the teacher data 140 is 30 seconds, the teacher data 140 may be divided every 5 seconds.

(Disaster model 150)
FIG. 3 is a schematic diagram showing an example of creating a disaster model 150.
As shown in FIG. 3, the learning is performed using a multi-layered neural network (deep neural network). The deep learning model is an expression showing the structure of the deep neural network. The deep learning unit 120 uses the teacher data 140 to generate at least a part of the components of the deep learning model (structure of the deep neural network) without human intervention, and the deep learning model including the generated components. The disaster model 150 is output.
Therefore, the deep learning model is automatically constructed. In the present invention, the actual disaster image is analyzed using the neural network thus obtained. Therefore, unlike the conventional machine learning method, it does not require steps such as region search and feature extraction, so that processing can be performed at higher speed.

Further, the disaster model 150 may be ranked according to the level of the disaster. Specifically, the disaster model 150 may be divided into a plurality of ranks such as the disaster level SL1 to the disaster level SL5.
Specifically, due to a disaster caused by an earthquake, the disaster level SL1 of one building is in a state where the wall is cracked, the disaster level SL2 is in a state where the window glass is broken, and the disaster level SL3 is in a state of collapse. The probability is less than 50%, the disaster level SL4 is the state where the possibility of collapse is 70% or more, and the disaster level SL5 is the state where the collapse is done.

Although the explanation has been given from the disaster level SL1 to the disaster level SL5, the number of disaster levels is not limited to this, and the number of disaster levels may be divided into arbitrary ranks such as 2, 3, 4, 10, and 100. For example, when the disaster level is divided into 100, it can be displayed with a probability of a percentage, and when the disaster level is divided into 10, it can be displayed as a percentage.

(Disaster map data 240)
Next, FIG. 4 is a schematic diagram showing an example of the disaster map data 240.
When the actual disaster image 300 shown in FIG. 1 is given to the disaster situation determination system 100, the deep learning unit 120 creates the disaster map data 240 based on the map data 230 and displays it on the display unit 130.

As shown in FIG. 4, when the actual disaster is a flood of a river, the bridge B1 shows that it is being washed away, and the bridge B2 shows that there is no damage. Further, the building B3 indicates that the building is submerged to the 1st floor, and the house B4 indicates that a fire has occurred. In addition, the square B5 indicates that there is no damage.
Further, as shown in FIG. 4, the disaster map data 240 may be a plane map or map data composed of a three-dimensional image.

Further, as shown in FIG. 4, the disaster map data 240 may be displayed by adding future weather forecast information. In this case, since the disaster map data 240 is the current information, for example, in the case of a flood disaster, an earthquake disaster, a heavy rain disaster, etc., the weather forecast information, particularly the weather forecast, the Amedas forecast, and the like are important.
Further, the disaster map data 240 may display the predicted change with time series. As a result, it is possible to predict the disaster situation several hours and several days from the present time.

(Route map 250)
Subsequently, FIG. 5 is a schematic diagram showing an example of the route diagram 250.
The deep learning unit 120 displays the route unit 250 showing the route map to the plaza B5 on the display unit 130 based on the disaster map data 240. That is, the deep learning unit 120 shows the route to the plaza 5 via the bridge B2 on the display unit 130 as a route map 250 while avoiding the area around the building B3 and the area around the house B4.

(Other embodiments)
FIG. 6 is a schematic diagram showing an example of the overall configuration of the disaster determination flight system 700 according to the present invention.
As shown in FIG. 6, the disaster determination flight system 700 includes a flight moving object 500 and a disaster situation determination system 100.
The disaster situation determination system 100 is the same as that shown in FIG.

The flight moving body 500 is equipped with an image pickup device 510 including a transmission unit that captures an actual disaster image 300 and transmits it to the deep learning unit 120 of the disaster situation determination system 100, and a flight body 520 that moves in flight equipped with the image pickup device 510. including.
Here, the image pickup apparatus 510 may be a camera, a video camera, or any other device that acquires an image, and may be an image. Further, the flying object 520 may be an airplane, an unmanned aerial vehicle, a drone, a helicopter, a kite, or the like.

The disaster determination flight system 700 according to the present invention can immediately acquire a disaster image 300 and give it to the disaster situation determination system 100 when an actual disaster occurs. Therefore, the municipality, residents, police, fire department, and self-defense force Information can be given appropriately to such things.

As described above, the disaster determination flight system 700 can transmit the disaster image 300 from the flying object 520, and the disaster situation determination system 100 can easily display the disaster map data 240 on the display unit 130 in real time.

[Correspondence between each part in the embodiment and each component of the claim]
The disaster situation determination system 100 in the present specification corresponds to the "disaster situation determination system", the disaster determination flight system 700 corresponds to the "disaster determination flight system", and the teacher data 140 indicates the "artificially created disaster state". The recording unit 110 corresponds to the "disaster image", the disaster map data 240 corresponds to the "disaster situation map", the route map 250 corresponds to the "passable route", and the deep learning unit. 120 corresponds to "route guidance instruction unit, weather display unit, deep learning unit", display unit 130 corresponds to "display unit", image pickup device 510 corresponds to "imaging device", and flying object 520 corresponds to "flying". The disaster image 300 corresponds to the "thing" and corresponds to the "disaster image".

100 Disaster situation judgment system 110 Recording unit 120 Deep learning unit 130 Display unit 140 Teacher data 240 Disaster map data 250 Route map 300 Disaster video 510 Imaging device (shooting device)
520 Aircraft 700 Disaster Judgment Flight System

Claims (7)

Multiple disaster images of collapsed buildings, collapsed bridges, collapsed mountains, collapsed embankments, collapsed roads, collapsed tunnels, fires, floods, and cracks, and disasters that show artificially created disaster conditions. A recording unit that records video and / or past disaster video ,
A deep learning unit that learns the disaster situation and the disaster location using the disaster video recorded in the recording unit as teacher data, and inputs the actual disaster video when a disaster occurs to identify the disaster situation and the disaster location .
A disaster situation determination system including a display unit for displaying a disaster situation map showing a disaster situation and a disaster location based on the disaster situation and the disaster location specified by the deep learning unit. The disaster situation determination system according to claim 1, wherein the artificially created disaster image showing a disaster state is formed by three dimensions. Including the route guidance instruction section
The disaster situation determination system according to claim 1 or 2, wherein the route guidance instruction unit displays a passable route on the display unit based on the disaster situation map. The disaster situation determination system according to any one of claims 1 to 3 , wherein the deep learning unit displays the probability of a disaster situation on the display unit. Including the weather display section
The disaster situation determination system according to any one of claims 1 to 4 , wherein the deep learning unit displays the weather information of the disaster situation map on the display unit according to the weather forecast information from the weather display unit. .. One of claims 1 to 5 , wherein the deep learning unit divides the disaster video after the disaster into one image, divides the image into a plurality of images, determines the disaster state of each image, and then synthesizes the images. The disaster situation judgment system described in item 1. At least one disaster situation determination system according to claims 1 to 6 ,
A disaster determination flight system that includes flying objects that are equipped with a shooting device and can transmit disaster images.

Images