JP7675469B2 - Road condition determination device, road condition determination program, and road condition determination method - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act (1) Snow and Ice Research Conference (2022, Sapporo), September 9, 2022, "Development of a Smartphone AI Road Surface Judgment System", Abstracts (booklet) (2) Snow and Ice Research Conference (2022, Sapporo), October 5, 2022, "Development of a Smartphone AI Road Surface Judgment System", Academic presentation (3) Snow and Ice Research Conference (2022, Sapporo), March 27, 2023, "Development of a Smartphone AI Road Surface Judgment System", Abstracts (website), https://www. jstage. jst. go. jp/article/jcsir/2022/0/2022＿126/＿article/-char/ja/ (4) The 38th Cold Region Technology Symposium, November 2022, "Development of an AI Road Surface Assessment System Using Smartphone Images", Collection of Cold Region Technology Papers and Reports (booklet) (5) The 38th Cold Region Technology Symposium, November 2022, "Development of an AI Road Surface Assessment System Using Smartphone Images", Collection of Cold Region Technology Papers and Reports (CD-ROM) (6) The 38th Cold Region Technology Symposium, December 1, 2022, "Development of an AI Road Surface Assessment System Using Smartphone Images", Conference presentation (7) National Research Institute for Earth Science and Disaster Resilience Snow and Ice Disaster Prevention Research Center website, December 21, 2022, "First report on the damage caused by the heavy snowfall that occurred in Niigata Prefecture from December 18 to 20, 2022 - A preliminary report on the on-site investigation conducted using a smartphone AI road surface judgment test system", https://yukibousai.bosai.go.jp/kaiseki/20221218Gousetsu/ShuuchuuGousetsuSokuhou1_20221221r2.pdf

(8) Website of the Snow and Ice Disaster Prevention Research Center of the National Research Institute for Earth Science and Disaster Resilience, January 4, 2023, “Fallen tree survey report on the large-scale power outage in Sado City caused by heavy snowfall from December 18 to 20, 2022”, https://yukibousai.bosai.go.jp/kaiseki/20221218Gousetsu/SadoToubokuChousa_20230105a.pdf (9) Website of the Snow and Ice Disaster Prevention Research Center of the National Research Institute for Earth Science and Disaster Resilience, January 27, 2023, “Survey report on the snowstorm that occurred in Yamagata Prefecture on January 25, 2023”, https://yukibousai.bosai.go.jp/kaiseki/20221218Gousetsu/SadoToubokuChousa_20230105a.pdf jp/kaiseki/20221218Gousetsu/YamagataFubuki20230125 .pdf (10) 2023 Annual General Meeting and Research Presentation and Product Announcement Meeting of the Hokushinetsu Branch of the Japan Society of Snow and Ice, May 29, 2023, "Survey of Snow and Ice Disasters Using a Smartphone AI Road Surface Judgment Test System", Proceedings (website), https://www.seppyo.org/hse/2023y17 (11) 2023 Annual General Meeting and Research Presentation and Product Announcement Meeting of the Hokushinetsu Branch of the Japan Society of Snow and Ice, June 3, 2023, "Survey of Snow and Ice Disasters Using a Smartphone AI Road Surface Judgment Test System", Academic Presentation

The present invention relates to a road condition determination device, a road condition determination program, and a road condition determination method.

The occurrence of snow damage on roads is one of the factors that greatly affect the safety of vehicles when traveling. For road operators, the occurrence of snow damage is an important criterion for determining, for example, whether or not to spray antifreeze during snowfall and the extent of the spraying, and whether or not to perform snow removal work to remove snow that has accumulated on the road and the extent of the snow removal. For this reason, a road condition detection device has been developed to determine the occurrence of snow damage, for example, a light wave transmitter/receiver attached to a gate-shaped frame installed across the road surface to detect the distance to the road surface and snow surface, thereby determining the road surface condition, snow banks, and effective width (see Patent Document 1).

Japanese Patent Application Publication No. 11-174161

The road surface condition detection device disclosed in Patent Document 1 uses a light-wave transmitter/receiver attached to a gate-shaped frame as described above, so the location where snow damage can be detected is limited to the location where the light-wave transmitter/receiver is installed. In addition, a gate-shaped frame is required to install the light-wave transmitter/receiver, which creates the problem of high equipment costs for installation and maintenance.

The present invention has been made in consideration of the above-mentioned problems, and aims to provide a road condition determination device, a road condition determination program, and a road condition determination method that make it possible to determine the snow damage situation at any point with a simple device configuration.

In order to achieve the above object, a road condition determination device according to one aspect of the present invention comprises:
a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The learning model is
Machine learning was performed using a learning dataset that includes multiple learning data consisting of image data of the target road and its surroundings, and correct answer labels that indicate the judgment results of the snow damage situation on the target road and its surroundings.

According to a road condition determination device according to one aspect of the present invention, a determination unit inputs image data of the road to be determined and its surroundings into a learning model, thereby determining the snow damage condition of the road to be determined and its surroundings. Therefore, since it is only necessary to obtain image data of the road to be determined and its surroundings, it is possible to determine the snow damage condition at any point with a simple device configuration.

Other issues, configurations and advantages will be made clear in the detailed description of the invention described below.

1 is an overall view showing an example of a road condition determination system 1. FIG. 2 is a block diagram showing an example of a terminal device 2. FIG. FIG. 2 is a block diagram showing an example of a road condition determination device 4. FIG. 4 is a data configuration diagram showing an example of a road condition database 401. FIG. 2 is a diagram showing an example of road surface divisions. FIG. 11 is a diagram showing an example of a side margin width. FIG. 13 is a diagram showing an example of snow bank height. FIG. 2 is a diagram illustrating an example of a visibility distance. FIG. 2 is a functional explanatory diagram showing an example of a road condition determination device 4. A schematic diagram showing an example of a first learning model 403A. A schematic diagram showing an example of a second learning model 403B. A schematic diagram showing an example of a third learning model 403C. A schematic diagram showing an example of a fourth learning model 403D. FIG. 2 is a block diagram showing an example of an administrator device 5. 4 is a flowchart showing an example of the operation of the road condition determination system 1. 15 is a flowchart showing an example of the operation of the road condition determination system 1; 17 is a flowchart (continuation of FIG. 16) showing an example of the operation of the road condition determination system 1. FIG. 11 is a diagram showing an example of a judgment result display screen 11. FIG. 12 is a diagram showing an example of a tally result display screen 12. FIG. 2 is a diagram showing an example of an image display screen 13. FIG. 13 is a diagram showing an example of a details display screen 14. FIG. 13 is a diagram showing an example of the number of passable lanes. FIG. 13 is a diagram showing an example of a degree of caution regarding stacking. FIG. 4 is a diagram showing an example of packed snow thickness. FIG. 11 is a diagram showing an example of roof snow depth. FIG. 13 is a diagram showing an example of a snow cornice caution level. FIG. 11 is a diagram showing an example of a degree of caution regarding fallen trees. FIG. 13 is a diagram showing an example of rain/snow classification. FIG. 11 is a functional explanatory diagram showing a modified example of the road condition determination device 4. A schematic diagram showing an example of a fifth learning model 403E. A schematic diagram showing an example of a sixth learning model 403F. A schematic diagram showing an example of a seventh learning model 403G. A schematic diagram showing an example of an eighth learning model 403H. A schematic diagram showing an example of a ninth learning model 403I. A schematic diagram showing an example of a 10th learning model 403J. A schematic diagram showing an example of an 11th learning model 403K.

The following describes an embodiment of the present invention with reference to the drawings. The following shows a schematic view of the scope of the description necessary to achieve the object of the present invention, and mainly describes the scope of the description necessary to explain the relevant parts of the present invention. The parts that are omitted are based on publicly known technology.

(Configuration of road condition determination system 1)
1 is an overall view showing an example of a road condition determination system 1. The road condition determination system 1 is a system that determines the road conditions of a road and its surroundings based on sensor data acquired by a vehicle 10 traveling on a road and weather data provided by a weather information providing device 3, and provides the determination results to the driver of each vehicle 10, a road management company, etc.

The road condition determination system 1 comprises, as its main components, a terminal device 2, a weather information providing device 3, a road condition determination device 4, and an administrator device 5. Each of the devices 2 to 5 is connected to a wired or wireless network 6 and is configured to be able to transmit and receive various data to and from each other (in FIG. 1, the transmission and reception of some data is indicated by dashed arrows). Note that the number of each of the devices 2 to 5 may be changed as appropriate, and the configuration of the network 6 is not limited to the example in FIG. 1.

The terminal device 2 is composed of, for example, a mobile terminal device such as a smartphone or tablet terminal that can be carried into the vehicle 10, and an in-vehicle device such as a drive recorder, a car navigation system, a collision prevention system, or a parking assistance system that is installed in the vehicle 10. In this embodiment, the terminal device 2 is composed of a smartphone carried by the driver, and the description will focus on a case where the terminal device 2 is used while installed on a fixed stand (not shown) on the dashboard. The vehicle 10 may be of any type or purpose, and may be, for example, a household vehicle, a commercial vehicle such as a taxi, a bus, or a truck, a vehicle managed by a road management company, or a two-wheeled vehicle such as a motorcycle or an electric motorcycle.

The terminal device 2 is equipped with a sensor group 26 including a position sensor 260, a camera 261, a microphone 262, an acceleration sensor 263, etc. The terminal device 2 transmits sensor data such as position data, image data, environmental sound data, and acceleration data recorded by the sensor group 26 at a predetermined recording interval (e.g., one-second intervals) to the road condition determination device 4 at a predetermined transmission interval (e.g., one-minute intervals) while the vehicle 10 is traveling. The terminal device 2 receives road condition determination information and road condition summary information from the road condition determination device 4, and displays a display screen based on the received information.

The position sensor 260 receives satellite positioning signals and, by combining with the acceleration sensor 263 as necessary, measures the current position (latitude, longitude) and traveling direction of the terminal device 2 (vehicle 10) as position data. When the camera 261 is used while being installed on a fixed stand in a position and orientation that allows it to capture an image of the road ahead of the vehicle 10, it captures an image of the road ahead of the vehicle 10 and its surroundings, and generates image data. When the microphone 262 is used in the vehicle 10, it records the environmental sounds (running sounds) of the vehicle 10 while it is running as environmental sound data. The acceleration sensor 263 is, for example, composed of a three-axis acceleration sensor (which may further be combined with a three-axis angular velocity sensor), and when used in the vehicle 10, it records the acceleration of the vehicle 10 while it is running as acceleration data.

The weather information providing device 3 is operated by the Japan Meteorological Agency, a weather information provider, etc., and provides (transmits) weather mesh information to other devices (e.g., the terminal device 2, the road condition determination device 4, the administrator device 5, etc.). The weather information providing device 3 is composed of, for example, a general-purpose or dedicated computer that functions as a server or cloud.

The weather information providing device 3 provides, as weather mesh information, weather data for the current time or past time, and weather forecast data for a future time a predetermined time after the current time (e.g., 1 hour, 2 hours, etc.). The weather mesh information includes weather, temperature, humidity, precipitation, accumulated precipitation, snowfall (dry snow, wet snow), accumulated snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as meteorological elements for each area divided into a mesh shape. The weather mesh information may be provided in either a pull or push manner, and the date, time period, area, etc. to be provided may be specified.

The road condition determination device 4 receives sensor data (position data, image data, environmental sound data, acceleration data, etc.) from the terminal device 2 and weather mesh information from the weather information providing device 3, and determines the road conditions of the road and its surroundings at each point and each time based on these data. Specifically, the road condition determination device 4 determines the snow damage conditions of the road and its surroundings as the road conditions of the road and its surroundings. In this case, the snow damage conditions of the road include, for example, those related to the roadway, and the snow damage conditions of the surroundings of the road include, for example, those related to the side strip, road shoulder, median strip, sidewalk, guardrail, boundary block, lane division sign, buildings along the road, fences, trees, etc. In addition, the road condition determination device 4 provides (transmits) road condition determination information indicating the judgment result of the road condition and road condition summary information indicating the summary result of the road condition judgment result to other devices (for example, the terminal device 2, the administrator device 5, etc.). The method by which the road condition determination device 4 provides various information may be either pull-type or push-type, and the date, time period, area, etc. to which the information is to be provided may be specified.

The administrator device 5 is used by road management businesses, etc., receives road condition judgment information and road condition summary information from the road condition judgment device 4, and displays a display screen based on the received information. The administrator device 5 may be configured, for example, as a stationary computer, or, like the terminal device 2, may be configured as a mobile terminal device or an in-vehicle device.

The administrator device 5 may function as a road traffic information providing device that provides (transmits) road traffic information based on road condition judgment information and road condition summary information to other devices (e.g., terminal device 2, etc.). In this case, the administrator device 5 provides, as road traffic information, road actual condition data at the current time and road forecast data for a future time a predetermined time after the current time (e.g., one hour, two hours, etc.). The road traffic information includes the judgment results and summary results of the road conditions, as well as traffic conditions related to congestion, accidents, construction, etc., and may be provided in a state where the road traffic information is superimposed on a map. The method by which the administrator device 5 provides various information may be either a pull method or a push method, and the date, time, period, area, etc. to be provided may be specified.

(Configuration of Terminal Device 2)
2 is a block diagram showing an example of the terminal device 2. The terminal device 2 includes a storage unit 20 including an HDD, an SSD, a memory, etc., a control unit 21 including a processor such as a CPU, a GPU, an MPU, etc., an input unit 22 including a button, a touch panel, etc., a display unit 23 including a display, a touch panel, etc., a communication unit 24 which is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I/F) unit 25 which is an interface with external devices such as a printer, a scanner, a USB memory, etc., and a sensor group 26 including a position sensor 260, a camera 261, a microphone 262, an acceleration sensor 263, etc.

The storage unit 20 stores an operating system (OS), which is a basic program, various application programs such as a sensor data recording program 200, a road condition display program 201, and a web browser, as well as various data used by these programs. Although the various data are basically stored in the storage unit 20, these programs and data may be obtained from an external storage device via the communication unit 24 or the external device I/F unit 25, and may be updated as appropriate.

The control unit 21 functions as a sensor data recording processing unit 210 by executing the sensor data recording program 200. The control unit 21 functions as a road condition display processing unit 211 by executing the road condition display program 201.

For example, when the sensor data recording processing unit 210 receives an input operation to start recording via the input unit 22 while the terminal device 2 is installed in the vehicle 10, the sensor data recording processing unit 210 records sensor data using the sensor group 26 at a predetermined recording interval (e.g., one-second intervals) and transmits the recorded sensor data to the road condition determination device 4 at a predetermined transmission interval (e.g., one-minute intervals). The sensor data recording processing unit 210 performs the above process (sensor data recording process) until an input operation to end recording is received via the input unit 22.

When the road condition display processing unit 211 receives road condition judgment information and road condition summary information from the road condition judgment device 4, it displays a display screen based on the road condition judgment information and road condition summary information on the display unit 23.

(Configuration of road condition determination device 4)
Fig. 3 is a block diagram showing an example of the road condition determination device 4. Fig. 4 is a data configuration diagram showing an example of the road condition database 401. Fig. 5 is a diagram showing an example of road surface divisions. Fig. 6 is a diagram showing an example of a lateral margin width. Fig. 7 is a diagram showing an example of a snow bank height. Fig. 8 is a diagram showing an example of a visibility distance.

The road condition determination device 4 includes a storage unit 40 including a HDD, SSD, memory, etc.; a control unit 41 including a processor such as a CPU, GPU, MPU, etc.; an input unit 42 including a keyboard, mouse, touch panel, etc.; a display unit 43 including a display, touch panel, etc.; a communication unit 44 that is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless); an external device interface (I/F) unit 45 that is an interface with external devices such as a printer, scanner, USB memory, etc.; and a media input/output unit 46 that is an interface with storage media such as CDs and DVDs. The input unit 42, the display unit 43, the external device I/F unit 45, and the media input/output unit 46 may be omitted as appropriate.

The storage unit 40 stores an operating system (OS), which is a basic program, and various application programs, such as a road condition determination program 400 that controls the operation of the road condition determination device 4. The storage unit 40 also stores various data used by these programs, such as a road condition database 401, a weather database 402, and learning models 403A to 403D. Note that while the various data are basically stored in the storage unit 40, these programs and data may be acquired from an external storage device via the communication unit 44 or the external device I/F unit 45, or may be acquired from a storage medium via the media input/output unit 46, and may be updated as appropriate.

The road condition database 401 is a database that stores sensor data received from the terminal device 2, road condition judgment results, and the like. As shown in FIG. 4, the road condition database 401 has multiple fields for each record, and each field stores information related to the driver, time, position data (point), image data, environmental sound data, acceleration data, road surface classification, lateral margin width, snow bank height, and visibility distance. The road surface classification, lateral margin width, snow bank height, and visibility distance are the judgment results of the road condition by the road condition judgment device 4, and are defined as shown in FIG. 5 to FIG. 8, respectively.

The weather database 402 is a database that stores the weather mesh information received from the weather information providing device 3. In the weather database 402, multiple weather elements for each region and each time (past time, current time, future time) are accumulated by registering weather mesh information from time to time. As described above, the weather mesh information includes weather, temperature, humidity, precipitation, snowfall (dry snow, wet snow), solar radiation, snow depth, wind speed, etc. as weather elements for each region divided into meshes. The size of one mesh may be, for example, 1 km square, 5 km square, 20 km square, etc., and may be different for each weather element. In addition, the value of the weather element may be an instantaneous value, or may be an integrated value if the weather element can be integrated over time (for example, precipitation, snowfall, etc.).

By executing the road condition determination program 400, the control unit 41 functions as a data acquisition unit 410, a determination unit 411, and an information provision unit 412, as shown in FIG. 3.

Figure 9 is a functional explanatory diagram showing an example of the road condition determination device 4. Figure 10 is a schematic diagram showing an example of the first learning model 403A. Figure 11 is a schematic diagram showing an example of the second learning model 403B. Figure 12 is a schematic diagram showing an example of the third learning model 403C. Figure 13 is a schematic diagram showing an example of the fourth learning model 403D. The road condition determination device 4 operates as the subject that implements the road condition determination method, and the processing contents performed by each part 410 to 412 of the control unit 41 correspond to each step of the road condition determination method (data acquisition step, determination step, information provision step).

The data acquisition unit 410 acquires road data including image data of the road to be judged and its surroundings (hereinafter referred to as the "road to be judged") photographed by the vehicle 10 while in motion, and correction data related to the location (hereinafter referred to as the "location to be judged") and time (hereinafter referred to as the "time to be judged") at which the image data was photographed.

The image data to be judged is data of the road to be judged (the road to be judged and its surroundings) photographed by the camera 261 of the terminal device 2 through the windshield of the vehicle 10 while it is traveling. The shooting range photographed in the image data may be any range that allows the road conditions to be judged based on the image, and is appropriately set to include not only the road (mainly the road surface) but also the surroundings of the road. The image data may be any of a color image, a grayscale image, and an image photographed with infrared rays or the like, and may be any of a two-dimensional image and a three-dimensional image. The image data may be any of a still image and a moving image. When the image data is a still image photographed by the camera 261 at a predetermined recording interval, the data acquisition unit 410 may acquire the image data one by one, or may acquire multiple pieces of image data collectively. When the image data is a moving image photographed by the camera 261, the data acquisition unit 410 may acquire the divided image data when the moving image is divided at a predetermined shooting period one by one, or may acquire multiple pieces of image data collectively. When the image data is a moving image, the image data contains dynamic information, such as the condition of splashing water, rainfall, or snowfall, which can further improve the accuracy of determining road conditions.

The correction data is, for example, weather data, position data, environmental sound data, acceleration data, etc. In this embodiment, the correction data is described as being of these four types, but the correction data may be of at least one of these types, or may include other correction data as long as it is related to the determination target location and determination target time. If the correction data is data that can be acquired by a sensor built into the terminal device 2 such as a smartphone, as in this embodiment, there is no need to install a new sensor, and therefore the introduction cost of the system can be reduced.

The weather data includes a target location (or an area including the target location) and multiple meteorological elements (e.g., temperature, amount of rainfall, amount of snowfall, etc.) for a target period from a time before the target time to the target time. The weather data is obtained from the weather database 402, for example, by referring to meteorological mesh information stored in the meteorological database 402, and using as extraction conditions the target location based on the position data obtained when the image data was captured, and the target period based on the time when the image data was captured. Note that the weather data may be obtained directly from the weather information providing device 3.

The position data is data that indicates the current position (target point) of the vehicle 10 at the target time. When the image data is a moving image, the position data records the driving trajectory, and it is preferable that the recording period of the driving trajectory is the same time period as the shooting period when the moving image was captured.

The environmental sound data is data recorded by the microphone 262 while the vehicle 10 is traveling at the determination target location and determination target time, and is time-series data having a predetermined recording period including before and after the determination target time. Note that, if the image data is a moving image, it is preferable that the recording period of the environmental sound data is the same time period as the shooting period during which the moving image was captured.

The acceleration data is data recorded by the acceleration sensor 263 while the vehicle 10 is traveling at the determination target point and determination target time, and is time-series data having a predetermined recording period including before and after the determination target time. Note that, if the image data is a moving image, it is preferable that the recording period of the acceleration data is the same time period as the shooting period during which the moving image was captured.

For example, the data acquisition unit 410 receives image data of the object to be judged and position data, environmental sound data, and acceleration data as correction data from the terminal device 2, and registers them in the road condition database 401. The data acquisition unit 410 also receives weather mesh information from the weather information providing device 3, and registers it in the weather database 402. Then, the data acquisition unit 410 acquires weather data as correction data by, for example, referring to the weather database 402 using the judgment target point (= position data) and judgment target period for the image data of the object to be judged received from the terminal device 2 as extraction conditions. As a result, the data acquisition unit 410 acquires road data including the image data of the object to be judged and correction data (weather data, position data, environmental sound data, and acceleration data).

The determination unit 411 determines the road conditions of the road to be determined at the determination point and determination time when the image data to be determined, which is included in the road data, was captured based on the road data acquired by the data acquisition unit 410. The determination unit 411 then registers the road conditions determination results in the road conditions database 401, and the road conditions determination results are accumulated in the road conditions database 401.

In this embodiment, the determination unit 411 uses the machine-learned learning models 403A to 403D (FIGS. 10 to 13) to determine the snow damage status, which is the occurrence status of snow damage, as the road status of the road to be determined.
(A) the condition of the road surface of the road to be judged;
(B) The condition of the lateral clearance of the road to be evaluated;
(C) The state of snow banks formed on the road to be assessed;
(D) The visibility condition of the road subject to the judgment due to snowfall or snowstorm;
That is, the determination unit 411 is a subject that executes the inference phase of machine learning, and inputs the image data to be determined into the learning models 403A to 403D, respectively, to determine the snow damage status for the above determination items (A) to (D), respectively.

The learning models 403A to 403D used in the judgment unit 411 correspond to the above judgment items (A) to (D), respectively, and are trained learning models that have undergone machine learning using a learning dataset. The learning dataset includes multiple pieces of learning data for supervised learning. The learning data is data used as training data, validation data, and test data in supervised learning. The learning model 403A is, for example, configured with a convolutional neural network (CNN).

As shown in Figs. 10 to 13, the learning models 403A to 403D perform machine learning using learning data consisting of learning target image data of a learning target road and its surroundings (hereinafter referred to as "learning target road") and a correct answer label indicating the judgment result of the snow damage situation of the learning target road (the learning target road and its surroundings). That is, in the learning phase of the machine learning, the learning target image data constituting the learning data is input as input data to the learning models 403A to 403D, and the judgment result of the snow damage situation (output data) outputted by the input data is compared with the judgment result of the snow damage situation (correct answer label) constituting the learning data, and the weighting parameters of the convolutional neural network are adjusted based on the comparison result, thereby performing machine learning. Note that the learning target image data may be subjected to a predetermined image adjustment process (e.g., image format, image size, image filter, image mask, etc.). In addition, data expansion process (rotation, shift, inversion, shear transformation, etc.) that generates multiple image data from the learning target image data may be performed. This makes it possible to homogenize the learning dataset.

The snow damage situation output as output data by the first learning model 403A is the classification result of the road surface classification when the judgment item (A) "road surface condition" is classified into multiple road surface classifications. The multiple road surface classifications include at least multiple stages of wet snow depths in which the depth of wet snow is classified in stages, and multiple stages of dry snow depths in which the depth of dry snow is classified in stages. In this embodiment, the road surface classification is defined by 17 classes consisting of dry, wet, puddle, flooded, 5 stages of wet snow depth, frozen, 5 stages of dry snow depth, compacted snow, and light snow, as shown in Figures 5 and 10. The first learning model 403A functions as a multi-class classifier (17 classes in this embodiment) and outputs the accuracy for each class when the road surface condition is classified into each of the multiple road surface classification classes. The accuracy for each class is output as a numerical value in a predetermined range (for example, 0 to 1). The classification method and number of classifications of road surface categories are not limited to the 17 types shown in the examples of Figures 5 and 10.

The snow damage situation output as output data by the second learning model 403B is the classification result of the side margin width when the "side margin condition", which is the judgment item (B), is classified into multiple stages of side margin width. In this embodiment, the side margin condition is defined by four stages of side margin width and five classes that are not judged, as shown in Figures 6 and 11. The second learning model 403B, like the first learning model 403A, functions as a multi-class classifier (five classes in this embodiment) and outputs the accuracy when the side margin condition is classified into each class for each class. Note that the classification method and number of classes of side margin are not limited to the five-class example shown in Figures 6 and 11.

The snow damage situation output as output data by the third learning model 403C is the classification result of snow bank height when the judgment item (C) "snow bank situation" is classified into multiple levels of snow bank height. In this embodiment, the snow bank situation is defined by four levels of snow bank height and five classes that are not judged, as shown in Figures 7 and 12. The third learning model 403C, like the first learning model 403A, functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the snow bank situation is classified into each class. Note that the classification method and number of classes for snow banks are not limited to the five-class example shown in Figures 7 and 12.

The snow damage situation output as output data by the fourth learning model 403D is the classification result of the visibility distance when the judgment item (D) "visibility condition" is classified into multiple stages of visibility distance. In this embodiment, the visibility condition is defined as five classes consisting of five stages of visibility distance, as shown in Figures 8 and 13. The fourth learning model 403D functions as a multi-class classifier (five classes in this embodiment) like the first learning model 403A, and outputs the accuracy when the visibility condition is classified into each class for each class. Note that the visibility classification method and number of classes are not limited to the five-class example shown in Figures 8 and 13.

The determination unit 411 is the entity that executes the inference phase of machine learning, and as shown in FIG. 9, inputs image data of the road to be determined, which is a photograph of the road to be determined, into the trained learning models 403A to 403D, respectively, to determine the snow damage situation for each of the above determination items (A) to (D). The determination unit 411 then outputs the classification results of the road surface classification, side margin width, snow bank height, and visibility distance for the road to be determined as the determination result. Specifically, the determination unit 411 outputs the class with the highest accuracy among the accuracy for each class for each determination item (A) to (D), for example, the road surface classification is "wet snow 1 cm to 3 cm" (see FIG. 10), the side margin width is "narrow" (see FIG. 11), the snow bank height is "slightly high" (see FIG. 12), and the visibility distance is "slightly poor" (see FIG. 13). In this case, the determination unit 411 may output the accuracy for each class as is, or instead of or in addition to the accuracy for each class, it may output the accuracy ranking when the accuracy for each class is sorted in descending order of value.

The determination unit 411 may perform a predetermined image adjustment process (e.g., image format, image size, image filter, image mask, etc.) on the image data to be determined as a preprocessing step when inputting the image data to be determined into the learning models 403A to 403D.

In addition, for example, when determining the side margin width and snow bank height, the determination unit 411 may determine the snow damage situation on the left side of the road to be determined and the snow damage situation on the right side of the road to be determined. On roads with a median strip, the snow damage situation on the right side of the road to be determined is determined by the side margin width and snow bank height on the median strip side. On roads without a median strip, the snow damage situation on the right side of the road to be determined is determined by the side margin width and snow bank height beyond the oncoming lane.

In this case, the determination unit 411 cuts out the left image area and the right image area from the image data to be determined to generate left image data and right image data. The determination unit 411 then inputs one side of the left image data and the right image data to the learning models 403B and 403C to determine the snow damage situation (side margin width and snow bank height) on one side of the road to be determined, and inputs the other side of the left image data and the right image data in a state in which they are inverted left and right to the learning models 403B and 403C to determine the snow damage situation (side margin width and snow bank height) on the other side of the road to be determined. The learning models 403B and 403C used here are machine-learned using a learning dataset that includes multiple learning data consisting of image data on one side of the road to be determined and a correct answer label indicating the determination result of the snow damage situation on one side of the road to be determined. As a result, for example, the left side image data is input as is to the left side learning models 403B and 403C, which have been machine-learned using learning data composed of image data including the image area of the left side of the road, and the right side image data is input after being inverted left and right. This allows the left side learning models 403B and 403C to determine the snow damage situation not only on the left side of the road but also on the right side of the road. In other words, the snow damage situation can be determined separately for the left and right sides of the road using the learning models 403B and 403C that are common to both sides of the road. Note that the snow damage situation may also be determined separately for the left and right sides of the road for determination items other than the lateral margin width and snow bank height. In addition, the image data of the learning target that constitutes the learning data may be image data in which the image area on the other side of the learning target road is inverted left and right.

Furthermore, as a process for correcting snow damage conditions, the determination unit 411 may determine whether or not a snow damage condition is likely to occur based on the correction data included in the road data, and correct the determination result of the snow damage condition (road condition) by the learning models 403A to 403D based on the presence or absence of the possibility of occurrence. The following describes a case where the determination unit 411 corrects the classification result of the road surface category.

In the first example, when weather data is acquired as correction data included in the road data, the determination unit 411 analyzes the weather data to determine the possibility of occurrence for each road surface classification. For example, when the air temperature at the determination target location and determination target time is 5°C or higher, it is determined that there is no possibility of occurrence of the "frozen" class of road surface classification. Then, the determination unit 411 outputs the road surface classification with the highest probability among the remaining road surface classifications excluding "frozen" as the road surface condition at the determination target location and determination target time.

In the second example, when position data is acquired as correction data included in the road data, the determination unit 411 determines the presence or absence of rainfall or snowfall by analyzing whether the position data is in a tunnel, and determines the presence or absence of rainfall or snowfall for each road surface classification according to the presence or absence of rainfall or snowfall. For example, when the position data is in a tunnel, the determination unit 411 determines that the following road surface classifications are not likely to occur: "0-1 cm wet snow", "1-3 cm wet snow", "3-5 cm wet snow", "5-10 cm wet snow", "10 cm or more wet snow", "0-1 cm dry snow", "1-3 cm dry snow", "3-5 cm dry snow", "5-10 cm dry snow", "10 cm or more dry snow", and "packed snow". The determination unit 411 then outputs the road surface classification with the highest probability among the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

In a third example, when environmental sound data is acquired as correction data included in the road data, the determination unit 411 analyzes the environmental sound data to determine the presence or absence of splashing water, and determines the possibility of occurrence of each road surface classification depending on the presence or absence of splashing water. For example, when the sound of splashing water is detected, the determination unit 411 determines that there is no possibility of occurrence of the road surface classifications of "dry", "wet", "frozen", and "packed snow". The determination unit 411 then outputs the road surface classification with the highest probability among the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

In a fourth example, when acceleration data is acquired as correction data included in the road data, the determination unit 411 detects the presence or absence of a slip component by analyzing the acceleration data, and determines the possibility of occurrence of each road surface classification depending on the presence or absence of the slip component. For example, when a slip component is detected in the acceleration data, the determination unit 411 determines that there is no possibility of occurrence of the road surface classifications of "dry", "wet", "puddle", and "flooded". The determination unit 411 then outputs the road surface classification with the highest probability among the remaining road surface classifications as the road surface condition at the determination target point and determination target time.

When the data acquisition unit 410 acquires some of the weather data, position data, environmental sound data, and acceleration data as correction data, the determination unit 411 may perform correction processing based on the part of the data acquired by the data acquisition unit 410. The determination unit 411 may not perform the above correction processing, in which case the road data acquired by the data acquisition unit 410 may not include correction data.

The information providing unit 412 generates road condition determination information indicating the result of the determination of the road condition (snow damage condition) determined by the determination unit 411 for the image data to be determined, and provides (transmits) the information to the terminal device 2 that captured the image data to be determined.

The information providing unit 412 also tally up the road condition (snow damage condition) judgment results determined by the judgment unit 411 for the image data to be judged for each road section or time section. A road section is, for example, each road section when a road is divided into a predetermined distance (e.g., 50 m). A time section is, for example, each time section when time is divided into a predetermined division interval (e.g., 1 minute or 1 hour). The information providing unit 412 then generates road condition summary information for superimposing the road condition (snow damage condition) summary results on the road map on the display screen by road section or time section, and provides the information to the terminal device 2, the administrator device 5, etc.

For example, the information providing unit 412 may tally up the road condition (snow damage condition) judgment results by road section or time section by referring to the road condition database 401. Therefore, when image data collected by the running of multiple vehicles 10 is registered in the road condition database 401, the road condition summary information is generated by tallying up the road condition judgment results for the collected image data. In this case, when multiple road condition judgment results are registered for a specific road section, the final road condition judgment result may be determined by majority vote, or the latest judgment result may be prioritized. In addition, when multiple road condition judgment results are registered for a specific time section, the final road condition judgment result may be determined by majority vote, or the latest judgment result may be prioritized. Furthermore, the information providing unit 412 may tally up the road condition judgment results separately for the left and right sides of the road, or may tally up by traffic direction or by lane. The information providing unit 412 may provide the road condition summary information in response to an output request from the terminal device 2, the administrator device 5, etc., or may provide the road condition summary information at a predetermined interval.

Furthermore, when the information providing unit 412 receives the designation of a specific point or specific time on a road where the tally results are superimposed on the road map on the display screen, the information providing unit 412 generates image display information for displaying on the display screen image data of the determination target taken at the determination target point or determination target time corresponding to the specific point or specific time, and provides this to the terminal device 2, the administrator device 5, etc. At that time, the information providing unit 412 may provide image display information for displaying on the display screen image data of multiple determination targets in chronological order or reverse chronological order based on the image data of the determination target taken at the determination target point corresponding to the specific point or the determination target time corresponding to the specific time.

The information providing unit 412 also provides additional display information for superimposing additional information related to the judgment target point and the judgment target time on the road map. The additional information is, for example, weather mesh information including meteorological elements for each area divided into a mesh, or disaster mesh information including disaster risk for each area divided into a mesh. The additional information may be any information that can be superimposed and displayed on the road map, and is not limited to the weather mesh information and disaster mesh information described above.

The various information provided by the information providing unit 412 is information that can be displayed by the road condition display program 201 executed on the terminal device 2 and the road condition display program 500 executed on the administrator device 5, but may also be generated in a format that can be displayed by any program, such as a web browser.

(Configuration of Administrator Device 5)
14 is a block diagram showing an example of the administrator device 5. The administrator device 5 includes a storage unit 50 including a HDD, SSD, memory, etc., a control unit 51 including a processor such as a CPU, GPU, MPU, etc., an input unit 52 including a keyboard, mouse, touch panel, etc., a display unit 53 including a display, touch panel, etc., a communication unit 54 that is an interface with the network 6 based on a predetermined communication standard (which may be either wired or wireless), an external device interface (I/F) unit 55 that is an interface with external devices such as a printer, scanner, USB memory, etc., and a media input/output unit 56 that is an interface with storage media such as a CD or DVD. The external device I/F unit 55 and the media input/output unit 56 may be omitted as appropriate.

The storage unit 50 stores an operating system (OS) which is a basic program, a road condition display program 500, various application programs such as a web browser, and various data used by these programs. Although the various data are basically stored in the storage unit 50, these programs and data may be obtained from an external storage device via the communication unit 54 or the external device I/F unit 55, or may be obtained from a storage medium via the media input/output unit 56, and may be updated as appropriate.

The control unit 51 functions as a road condition display processing unit 510 by executing the road condition display program 500.

When the road condition display processing unit 510 receives road condition judgment information and road condition summary information from the road condition judgment device 4, it displays a display screen based on the road condition judgment information and road condition summary information on the display unit 53.

(Operation of the road condition determination system)
15 to 17 are flowcharts showing an example of the operation of the road condition determination system 1. Below, an explanation will be given of the operation in the case where the terminal device 2 installed on a fixed stand of the vehicle 10 records sensor data (position data, image data, environmental sound data, and acceleration data) at a predetermined recording interval while the vehicle 10 is traveling on a road, and transmits the sensor data to the road condition determination device 4 at a predetermined transmission interval.

First, in step S100, the sensor data recording processing unit 210 of the terminal device 2 transmits the sensor data (position data, image data, environmental sound data, and acceleration data) at a given time (target time of determination) to the road condition determination device 4 each time a predetermined transmission interval arrives. Note that if the recording interval of the sensor data is shorter than the transmission interval of the sensor data, the terminal device 2 may transmit the sensor data for multiple times (multiple target times of determination) together.

Next, in step S101, the data acquisition unit 410 of the road condition determination device 4 receives the sensor data (position data, image data, environmental sound data, and acceleration data) transmitted in step S100, and acquires the image data to be determined, and the position data, environmental sound data, and acceleration data as correction data. The data acquisition unit 410 then registers these data in the road condition database 401.

In step S110, the weather information providing device 3 transmits weather mesh information to the road condition determination device 4. Next, in step S111, the data acquisition unit 410 receives the weather mesh information transmitted in step S110 and registers it in the weather database 402. Then, in step S112, the data acquisition unit 410 acquires weather data as correction data by referring to the weather database 402 using as extraction conditions the determination target point based on the position data received in step S101 and the determination target period based on the shooting time when the image data received in step S101 was taken. As a result of the above steps S101 and S112, in step S120, the data acquisition unit 410 acquires road data including the image data to be determined and the weather data, position data, environmental sound data, and acceleration data as correction data.

Next, in step S130 (S131 to S135), the determination unit 411 determines the snow damage condition of the road to be determined at the location (target location) and time (target time) when the image data to be determined, which is included in the road data, was captured based on the road data acquired in step S120, and registers the snow damage condition in the road condition database 401.

Specifically, in step S131, the determination unit 411 determines the road surface condition by inputting the image data of the determination target into the first learning model 403A, and outputs the classification result of the road surface category. In step S132, the determination unit 411 determines the side margin condition by inputting the image data of the determination target into the second learning model 403B, and outputs the classification result of the side margin width. In step S133, the determination unit 411 determines the snow bank condition by inputting the image data of the determination target into the third learning model 403C, and outputs the classification result of the snow bank height. In step S134, the determination unit 411 determines the visibility condition by inputting the image data of the determination target into the fourth learning model 403D, and outputs the classification result of the visibility distance.

Next, in step S135, the determination unit 411 determines whether or not a snow damage situation is likely to occur based on the correction data (weather data, position data, environmental sound data, and acceleration data), and corrects the determination results of the snow damage situation determined in steps S131 to S134 based on the presence or absence of the possibility of occurrence. Note that step S135 may be omitted, in which case the process of acquiring the correction data (steps S101, S111, S112, etc.) may also be omitted.

Next, in step S140, the information providing unit 412 transmits road condition determination information indicating the result of the snow damage determination made in step S130 to the terminal device 2 that is the sender of the sensor data received in step S101. Then, in step S141, the road condition display processing unit 211 of the terminal device 2 receives the road condition determination information transmitted in step S140, and displays the determination result display screen 11 (FIG. 18) based on the road condition determination information.

Figure 18 is a diagram showing an example of the judgment result display screen 11. The judgment result display screen 11 has an image display area 110 that displays the image data of the judgment target, and a judgment result display area 111 that displays the judgment result of the snow damage situation for the image data of the judgment target based on the road condition judgment information.

The judgment result display area 111 shown in FIG. 18 displays the judgment results when the road surface classification is judged to be "wet snow 0 cm to 1 cm", the side margin width is "slightly narrow", the snow bank height is "low", and the visibility distance is "good". Note that the judgment result display screen 11 may additionally display information based on the judgment results of the snow damage situation, for example, a warning message or icon that calls the driver's attention may be displayed.

In addition, in step S150, the information providing unit 412 repeatedly performs the above steps S110 to S130 in response to sequentially receiving image data, etc. transmitted at a predetermined transmission interval from the terminal device 2 (or multiple terminal devices 2), thereby tallying up the snow damage situation judgment results accumulated in the road condition database 401 for each road section and each time section. At that time, the road data extracted from the road condition database 401 as the target for tallying is extracted based on, for example, driver conditions for the driver, area conditions for the location, and date and time conditions for the time. Then, road condition tally information is generated based on the tallying results, and is transmitted to the terminal device 2 and the administrator device 5.

Next, in step S151, the road condition display processing unit 211 of the terminal device 2 receives the road condition summary information sent in step S150, and displays the summary result display screen 12 based on the road condition summary information. Also, in step S152, the road condition display processing unit 510 of the administrator device 5 receives the road condition summary information sent in step S150, and displays the summary result display screen 12 (FIG. 19) based on the road condition summary information. The following mainly describes the case where the summary result display screen 12 is displayed by the administrator device 5 and various input operations are performed by the road management company, but the same operation is performed when the summary result display screen 12 is displayed on the terminal device 2.

Figure 19 is a diagram showing an example of the tally result display screen 12. The tally result display screen 12 includes a tally condition specification area 120 for specifying extraction conditions for road data to be tallied, a map display area 121 for displaying the tally result of snow damage conditions superimposed on a road map, a judgment item specification area 122 for specifying judgment items for the snow damage conditions to be displayed in the map display area 121, and a legend display area 123 for displaying a legend showing the display mode of the snow damage conditions to be displayed in the map display area 121. The tally result display screen 12 may also include an input interface (e.g., a slider) for displaying the change over time in the tally result of snow damage conditions.

The tally condition specification area 120 has an input interface that allows specification of, for example, a driver condition that specifies all drivers or an individual driver, and a date and time condition that specifies a date and time range. The tally result display screen 12 shown in FIG. 19 is displayed when "all drivers" is specified as the driver condition, and "February 3, 2023, 10:00" to "February 3, 2023, 11:00" is specified as the date and time condition. Note that the date and time condition may be specified based on the current time.

In the map display area 121, a road map is displayed, and for each road displayed on the road map, the aggregated results of snow damage conditions for the judgment items specified in the judgment item specification area 122 (in FIG. 19, lateral margin width) are superimposed and displayed according to the legend displayed in the legend display area 123. Note that the map display area 121 may simultaneously display the aggregated results of snow damage conditions for multiple judgment items, such as road surface classification and snow bank height.

The map display area 121 also includes an input interface 121a for moving and zooming in and out of the road map, and is configured to be able to change the display range of the road map displayed in the map display area 121. When the display range of the map display area 121 is changed, the area conditions for the location are treated as being specified as the extraction conditions for the road data to be tabulated. Therefore, when the driver conditions or date and time conditions in the tabulation condition specification area 120 are changed, or when the display range (area conditions) of the map display area 121 is changed, the information providing unit 412 resends the road condition tabulation information according to the changed extraction conditions, thereby updating the tabulation result display screen 12. When the information providing unit 412 sends additional display information, the road condition display processing unit 211 may display weather mesh information and disaster mesh information superimposed on the road map displayed in the map display area 121 based on the additional display information.

Then, in step S160, when the road condition display processing unit 510 of the administrator device 5 receives an input operation to designate a specific point on the road on which the snow damage situation summary results are superimposed on the summary result display screen 12 displayed in step S152, the road condition display processing unit 510 transmits specific point information indicating the specific point to the road condition determination device 4. On the summary result display screen 12, for example, the specification of the specific point is accepted by placing the cursor 121b on an arbitrary point on the road and clicking it.

Next, in step S161, the information providing unit 412 receives the specific location information transmitted in step S160, and when it accepts the designation of a specific location based on the specific reception information, transmits image display information to the administrator device 5 for displaying on the display screen the image data of the determination target taken at the location corresponding to the specific location. In this embodiment, the information providing unit 412 transmits image display information to the administrator device 5 for displaying on the display screen the image data of multiple determination targets in chronological order or reverse chronological order based on the image data of the determination target taken at the location corresponding to the specific location.

Next, in step S162, the road condition display processing unit 510 of the administrator device 5 receives the image display information transmitted in step S161, and displays the image display screen 13 (Figure 20) based on the image display information.

Figure 20 is a diagram showing an example of the image display screen 13. The image display screen 13 includes an image display area 130 that displays the image data of the subject of judgment, a summary list area 131 that shows in list form a list of the road sections or time sections when the snow damage conditions were summarized, an image designation area 132 that specifies whether the image data of the subject of judgment displayed in the image display area 130 should be changed to chronological order or reverse chronological order, a summary result display area 133 that displays the summary results of the snow damage conditions in the road section or time section including the image data of the subject of judgment displayed in the image display area 130, and a details confirmation button 134 for confirming the details of the judgment results of the snow damage conditions for the image data of the subject of judgment.

In the initial state, the image display area 130 displays the image data to be judged as a reference. The image display screen 13 shown in FIG. 20 is displayed when image data captured at one-second intervals is tallied every minute (every time period), i.e., when 60 pieces of image data are tallied every minute, and the image display area 130 displays the image data captured at 10:03:10.

When an input operation to select a specific time using the selection frame 131a is received in the tally list area 131, the image display area 130 displays image data taken at that specific time. When an input operation to the play button, reverse play button, or slider 132a is received in the image designation area 132, multiple image data are displayed in chronological order or reverse chronological order in response to the input operation in the image display area 130. Therefore, when an input operation to the tally list area 131 or the image designation area 132 is received, the information providing unit 412 resends image display information in response to the input operation, thereby updating the image display screen 13. This allows a road management company (or a driver) to simply designate a specific point on a road map to check the snow damage situation at that specific point in an image, as well as the snow damage situation on the roads before and after the point in an image.

The tally result display area 133 displays the tally result of snow damage conditions in a specific road section or time section (1 minute in FIG. 20). The image display screen 13 shown in FIG. 20 is displayed when the classification results of the side margin width for 60 pieces of image data include "52 pieces" of image data that are judged to have a "slightly narrow" side margin width, "5 pieces" of image data that are judged to have a "narrow" side margin width, and "3 pieces" of image data that are judged to have a "wide" side margin width. In this case, if a majority vote is used as the tally result for the road section or time section, the tally result of the side margin width for the road section or time section in which 60 pieces of image data were captured is judged to be "slightly narrow" as shown by the thick frame in FIG. 20. Such tally results are obtained for each road section or time section, and the tally result of snow damage conditions is displayed for each road section or time section on the tally result display screen 12 shown in FIG. 19.

Then, in step S170, when the road condition display processing unit 510 of the administrator device 5 receives an input operation of pressing the details confirmation button 134 on the image display screen 13 displayed in step S162, it transmits specific time information indicating the specific time selected in the selection frame 131a at that time to the road condition determination device 4.

Next, in step S171, the information providing unit 412 receives the specific time information transmitted in step S170, and when it accepts the designation of a specific time based on the specific time information, transmits image display information to the administrator device 5 for displaying on the display screen the image data of the judgment target taken at the time corresponding to the specific time. In this embodiment, the information providing unit 412 transmits image display information to the administrator device 5 for displaying on the display screen the image data of multiple judgment targets in chronological order or reverse chronological order based on the image data of the judgment target taken at the time corresponding to the specific time.

Next, in step S172, the road condition display processing unit 510 of the administrator device 5 receives the image display information transmitted in step S171, and displays the detailed display screen 14 (FIG. 20) based on the image display information.

Figure 21 is a diagram showing an example of the detailed display screen 14. The detailed display screen 14 has a detailed result list area 140 in which the image data to be judged and the judgment results of the snow damage situation for that image data are arranged in chronological order or reverse chronological order in list format.

The detailed display screen 14 shown in FIG. 21 is displayed when "10:03:00" is specified as a specific time using the selection frame 131a as shown in FIG. 20.

The series of operations described above is performed mainly by the road condition determination device 4, which determines the road conditions at each location and each time, and provides the road condition determination results and the tabulated results to the terminal device 2 and the administrator device 5.

Therefore, according to the road condition determination device 4 and road condition determination method of this embodiment, the determination unit 411 determines the snow damage condition of the road to be determined by inputting image data of the road to be determined to the learning models 403A to 403D. Therefore, since it is only necessary to obtain image data of the road to be determined, it is possible to determine the snow damage condition at any point with a simple device configuration.

Other Embodiments
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and scope of the present invention. All such modifications are included in the technical concept of the present invention.

(Modification of Judgment Items)
In the above embodiment, the determination unit 411 uses the trained learning models 403A to 403D to determine the snow damage status of the road to be determined for the four determination items (A) to (D).
(E) The number of lanes available on the road to be assessed;
(F) The likelihood of a road being stacked on the road being evaluated;
(G) The condition of packed snow on the road to be evaluated;
(H) The condition of snow accumulated on the roofs of buildings along the road to be evaluated;
(I) The condition of snow cornices formed on the roofs of buildings along the road to be evaluated;
(J) The status of fallen trees on the road to be assessed;
(K) rainfall or snowfall conditions on the road to be judged;
In this case, the determination unit 411 may determine the snow damage state of the road to be determined for at least one of the determination items (A) to (K).

Fig. 22 is a diagram showing an example of the number of passable lanes. Fig. 23 is a diagram showing an example of the degree of caution for getting stuck. Fig. 24 is a diagram showing an example of packed snow thickness. Fig. 25 is a diagram showing an example of roof snow depth. Fig. 26 is a diagram showing an example of the degree of caution for snow cornices. Fig. 27 is a diagram showing an example of the degree of caution for fallen trees. Fig. 28 is a diagram showing an example of rain/snow classification.

Figure 29 is a functional explanatory diagram showing a modified example of the road condition determination device 4. Figure 30 is a schematic diagram showing an example of the fifth learning model 403E. Figure 31 is a schematic diagram showing an example of the sixth learning model 403F. Figure 32 is a schematic diagram showing an example of the seventh learning model 403G. Figure 33 is a schematic diagram showing an example of the eighth learning model 403H. Figure 34 is a schematic diagram showing an example of the ninth learning model 403I. Figure 35 is a schematic diagram showing an example of the tenth learning model 403J. Figure 36 is a schematic diagram showing an example of the eleventh learning model 403K.

In the road condition determination device 4 shown in FIG. 29, the determination unit 411, as in the above embodiment, inputs the image data to be determined into the learning models 403E to 403K, respectively, to determine the snow damage conditions for the above determination items (E) to (K). The determination results of the snow damage conditions for the determination items (E) to (K) are defined as shown in FIGS. 22 to 28, respectively, and are registered in each field of the road condition database 401 (FIG. 4) in the same manner as the determination items (A) to (D). Furthermore, the determination results of the snow damage conditions for the determination items (E) to (K) are displayed on the determination result display screen 11 (FIG. 18), the counting result display screen 12 (FIG. 19), the image display screen 13 (FIG. 20), and the details display screen 14 (FIG. 21) in the same manner as the determination items (A) to (D). At that time, for example,

Since the configurations and processing contents of the data acquisition unit 410 and the information provision unit 412 are similar to those in the above embodiment, the following description will focus on the learning models 403E to 403 and the determination unit 411.

The learning models 403E to 403K used in the judgment unit 411 correspond to the judgment items (E) to (K) described above, respectively, and, like the learning models 403A to 403D, are stored in the storage unit 40 as trained learning models for which machine learning has been performed using the learning datasets shown in Figures 30 to 36. Note that the configurations and machine learning techniques of the learning models 403E to 403K are similar to those of the learning models 403A to 403D according to the above embodiment, and therefore will not be described here.

The snow damage situation output as output data by the fifth learning model 403E is the classification result of the number of passable lanes when the judgment item (E) "status of the number of passable lanes on the road" is classified into multiple stages of the number of passable lanes. In the example of Figures 22 and 30, the status of the number of passable lanes is defined by six stages of the number of passable lanes and seven classes consisting of non-judgment targets. The fifth learning model 403E functions as a multi-class (seven classes in this embodiment) classifier and outputs the accuracy when the status of the number of passable lanes on the road is classified into each class for each class. Note that the classification method and number of classifications of the number of passable lanes are not limited to the seven-class example shown in Figures 22 and 30.

The snow damage situation output as output data by the sixth learning model 403F is the classification result of the stack caution level when the judgment item (F) "situation of likelihood of getting stuck" is classified into multiple stages of stack caution level. The situation of likelihood of getting stuck varies, for example, depending on the type and weight of the vehicle. In addition, the situation of likelihood of getting stuck varies depending on whether or not depressions are formed, whether or not the surface is crunchy, and the degree of depression or crunchy when snow or ice on the road melts or is crushed by the passage of vehicles. In the example of FIG. 23 and FIG. 31, the situation of likelihood of getting stuck is defined by four stages of stack caution level and five classes that are not subject to judgment. The sixth learning model 403F functions as a multi-class (five classes in this embodiment) classifier and outputs the accuracy when the situation of likelihood of getting stuck is classified into each class for each class. Note that the classification method and number of classifications of stack caution levels are not limited to the five-class example shown in FIG. 23 and FIG. 31.

The snow damage condition output as output data by the seventh learning model 403G is the classification result of the packed snow thickness when the judgment item (G) "packed snow condition" is classified into multiple levels of packed snow thickness. In the example of Figures 24 and 32, the packed snow condition is defined by four levels of packed snow thickness and five classes consisting of non-judgment targets. The seventh learning model 403G functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the packed snow condition is classified into each class. Note that the classification method and number of classes of packed snow thickness are not limited to the five-class example shown in Figures 24 and 32.

The snow damage situation output as output data by the eighth learning model 403H is the classification result of roof snow depth when the judgment item (H) "roof snow situation" is classified into multiple levels of roof snow depth. In the example of Figures 25 and 33, the roof snow situation is defined by five levels of roof snow depth and six classes consisting of non-judgment targets. The eighth learning model 403H functions as a multi-class (six classes in this embodiment) classifier, and outputs the accuracy for each class when the roof snow situation is classified into each class. Note that the classification method and number of classes for roof snow depth are not limited to the six-class example shown in Figures 25 and 33.

The snow damage situation output as output data by the ninth learning model 403I is the classification result of the snow cornice warning level when the judgment item (I) "snow cornice situation" is classified into multiple levels of snow cornice warning level. In the examples of Figures 26 and 34, the snow cornice situation is defined by four levels of snow cornice warning level and five classes consisting of non-judgment targets. The ninth learning model 403I functions as a multi-class (in this embodiment, five classes) classifier, and outputs the probability for each class when the snow cornice situation is classified into each class. Note that the classification method and number of classes for snow cornice warning level are not limited to the five-class example shown in Figures 26 and 34.

The snow damage situation output as output data by the tenth learning model 403J is the classification result of the degree of caution for fallen trees when the judgment item (J) "fallen tree situation" is classified into multiple levels of fallen tree caution. In the examples of Figures 27 and 35, the fallen tree situation is defined by four levels of fallen tree caution and five classes that are not subject to judgment. The tenth learning model 403J functions as a multi-class (five classes in this embodiment) classifier, and outputs the accuracy for each class when the fallen tree situation is classified into each class. Note that the classification method and number of classes for the degree of caution for fallen trees are not limited to the five-class example shown in Figures 27 and 35.

The snow damage situation output as output data by the eleventh learning model 403K is the classification result of the rain/snow classification when the judgment item (K) "rainfall or snowfall situation" is classified into multiple rain/snow classifications. In the examples of Figures 28 and 36, the rain/snow situation is defined by seven rain/snow classifications and eight classes that are not subject to judgment. The eleventh learning model 403K functions as a multi-class (eight classes in this embodiment) classifier, and outputs the accuracy for each class when the rain/snow situation is classified into each class. Note that the classification method and number of classifications of rain/snow classifications are not limited to the eight-class example shown in Figures 28 and 36.

In addition, when determining the roof snow depth and snow cornice caution level, the determination unit 411 may generate left side image data and right side image data from the image data to be determined as in the above embodiment, and input one side of the left side image data and the right side image data to the learning models 403H and 403I to determine the snow damage situation (roof snow depth and snow cornice caution level) on one side of the road to be determined, and input the other side of the left side image data and the right side image data in a state of being inverted left and right to the learning models 403H and 403I to determine the snow damage situation (roof snow depth and snow cornice caution level) on the other side of the road to be determined. The learning models 403H and 403I used here are machine-learned using a learning dataset that includes multiple learning data consisting of image data on one side of the road to be determined and a correct answer label indicating the judgment result of the snow damage situation on one side of the road to be determined. In addition, as a process for correcting snow damage conditions, the determination unit 411 may determine whether or not a snow damage condition is likely to occur based on correction data included in the road data, and correct the determination result of the snow damage condition (road condition) by the learning models 403E to 403K based on the presence or absence of the possibility of occurrence. The other configurations and processing contents of the determination unit 411 are the same as those of the above embodiment, so a description thereof will be omitted.

Furthermore, the determination unit 411 may determine the snow damage status of the road to be determined for other snow damage status determination items, such as stuck vehicles, stranded vehicles, accident vehicles, snowdrifts, and avalanches. The determination unit 411 may determine the road status of the road to be determined, and may determine road status other than snow damage status. Examples of road status of the road to be determined include, but are not limited to, damage to the pavement due to melting or freezing (e.g., potholes, etc.), flooding due to flooding or tsunami (which may be flooding itself or traces of flooding), occurrence of earthquake disasters such as fissures and cracks, occurrence of landslides such as cliff collapses, mudslides, and landslides, etc.

In the above embodiment, the data acquisition unit 410 of the road condition determination device 4 acquires image data captured by the camera 261 (terminal device 2) on the vehicle 10. In contrast, the data acquisition unit 410 may acquire image data of the road to be determined, and may acquire image data captured by a device other than the camera 261 on the vehicle 10. For example, the data acquisition unit 410 may acquire image data captured by a road monitoring camera equipped in a road monitoring device installed on the road, or may acquire image data captured by a drone camera equipped in an unmanned aerial vehicle such as a drone. In this case, the data acquisition unit 410 may further acquire correction data detected by a group of sensors equipped in the road monitoring device or the unmanned aerial vehicle. The data acquisition unit 410 may also acquire image data captured on a bicycle or on foot using a terminal device 2 such as a smartphone.

In the above embodiment and the modified example of the judgment item, the case where a convolutional neural network is adopted as a specific method of machine learning using the learning models 403A to 403K has been described. In contrast, the learning models 403A to 403K may adopt any other machine learning method. Examples of other machine learning methods include tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, other neural net types (including deep learning) such as recurrent neural networks, clustering types such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors and k-means, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, and support vector machines. In addition, the learning models 403A to 403K may be re-learned by batch learning or online learning. In that case, the learning data may be, for example, image data in which the judgment result of the road condition (snow damage condition) has been corrected based on the correction data. The re-learning process may be performed by the road condition judgment device 4 or by another device.

In the above embodiment and modified examples of the judgment items, the first to eleventh learning models 403A to 403K have been described as judging each of the judgment items (A) to (K) of the snow damage situation. In contrast, one learning model may judge multiple judgment items of the snow damage situation. Also, multiple learning models may be used for one judgment item. For example, two learning models in which machine learning has been performed separately for daytime and nighttime may be used, or two learning models in which machine learning has been performed separately for general roads and expressways may be used.

In the above embodiment and modified examples of the judgment items, the judgment unit 411 uses the trained learning models 403A-403K to judge the snow damage situation of the road to be judged for 11 judgment items (A)-(K). In contrast, instead of using the trained learning models 403A-403K, the judgment unit 411 may judge the road conditions of the road to be judged by performing image processing to detect feature points (e.g., edges and corners) contained in the image data or image processing to detect differences from image data captured under normal circumstances. In that case, the information provision unit 412 may provide various information based on the judgment results of the road conditions judged by the judgment unit 411.

In the above embodiment and the modified example of the judgment item, the judgment unit 411 has been described as correcting the judgment result of the snow damage situation (road situation) by the learning models 403A to 403K with the correction data. In contrast, if there is a judgment item among the judgment items (A) to (K) of the snow damage situation that can judge the snow damage situation only with the correction data, the judgment unit 411 may judge the snow damage situation based on the correction data without inputting the image data of the judgment target to the learning model corresponding to that judgment item. For example, if position data is acquired as the correction data included in the road data, the judgment unit 411 judges whether the position data is inside a tunnel. If it is inside a tunnel, the judgment unit 411 may judge the side margin width as "wide" and the snow bank height as "none" for the judgment items (B) and (C) of the snow damage situation, respectively, without inputting the image data of the judgment target to the second and third learning models 403B and 403C. Furthermore, when inside a tunnel, the judgment unit 411 may judge the roof snow depth to be "none," the snow cornice warning level to be "none," and the fallen tree warning level to be "none" for the snow damage situation judgment items (H) to (J) without inputting the image data of the judgment target to the eighth, ninth, and tenth learning models 403H to 403J.

In the above embodiment, the control unit 41 of the road condition determination device 4 executes the road condition determination program 400 to function as the data acquisition unit 410, the determination unit 411, and the information provision unit 412. In contrast, the control unit of each device, such as the terminal device 2, the administrator device 5, the vehicle-mounted device, the vehicle (vehicle control device), the road monitoring device, or the unmanned aerial vehicle, may execute the road condition determination program 400 to function in the same manner as the road condition determination device 4, or may function as part of the data acquisition unit 410, the determination unit 411, and the information provision unit 412. In this case, the road condition database 401, the weather database 402, and the learning model 403A may be stored in the storage unit of each device, or may be stored in an external device. In addition, the road condition determination program 400 may include all or part of the functions realized by the sensor data recording program 200, the road condition display program 201, and the road condition display program 500.

In the above embodiment, each program (sensor data recording program 200, road condition display program 201, road condition judgment program 400, road condition display program 500) has been described as being stored in the storage unit of each device. In contrast, each program may be provided by being recorded in an installable or executable file format on a computer-readable storage medium such as a CD-ROM or DVD. In addition, each program may be provided by being downloaded from an external device via the network 6.

1...road condition determination system, 2...terminal device, 3...weather information providing device,
4...road condition determination device, 5...administrator device, 6...network,
10: vehicle; 11: judgment result display screen; 12: compilation result display screen;
13...Image display screen, 14...Details display screen,
40: storage unit, 41: control unit, 42: input unit, 43: display unit, 44: communication unit,
45: external device I/F unit, 46: media input/output unit,
200: Sensor data recording program, 201: Road condition display program,
210: sensor data recording processing unit; 211: road condition display processing unit;
400...road condition judgment program, 401...road condition database,
403A: first learning model, 403B: second learning model,
403C...third learning model, 403D...fourth learning model,
403E...fifth learning model, 403F...sixth learning model,
403G... seventh learning model, 403H... eighth learning model,
403I... 9th learning model, 403J... 10th learning model,
403K...11th learning model,
410: data acquisition unit, 411: determination unit, 412: information provision unit,
500: Road condition display program, 510: Road condition display processing unit

Claims (15)

a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes.
Road condition determination device. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
a classification result of the stack caution level when the situation of the road where the vehicle is likely to become stuck is classified into a plurality of stages of stack caution levels according to the type or weight of the vehicle;
Road condition determination device. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
A classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level.
Road condition determination device. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
a classification result of the degree of caution for fallen trees when the state of fallen trees with respect to the road and its road boundary is classified into a plurality of levels of the degree of caution for fallen trees according to the extent to which the fallen trees exist with respect to the road and its road boundary;
Road condition determination device. a data acquisition unit that acquires road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination unit that determines a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined included in the road data acquired by the data acquisition unit into a learning model,
The image data to be determined is
an image of a front view of a vehicle traveling on a road to be determined is taken from the vehicle,
The determination unit is
A left image region and a right image region are cut out from the image data to be determined to generate left image data and right image data;
The left side image data and the right side image data are input to the learning model to determine the snow damage situation on the one side of the road to be determined;
The other side of the left side image data and the right side image data is input to the learning model in a state where the other side is inverted in the left-right direction, thereby determining the snow damage situation on the other side of the road to be determined;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data sets each including image data of a learning target road and its surroundings, the image data being of the one side of the learning target road and a correct answer label indicating a determination result of the snow damage situation on the one side of the learning target road.
Road condition determination device. The snow damage situation is as follows:
a classification result of the road surface classification when the road surface conditions are classified into a plurality of road surface classifications;
a classification result of the side margin width when the side margin condition of the road is classified into a plurality of stages of side margin width;
a classification result of the snow bank height when the state of the snow bank formed on the road is classified into a plurality of stages of snow bank height;
a classification result of the visibility distance when the visibility condition of the road due to snowfall or snowstorm is classified into a plurality of visibility distance stages;
A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes;
a classification result of the stack caution level when the state of the road where stacks are likely to occur is classified into a plurality of stack caution levels;
a classification result of the packed snow thickness when the packed snow condition on the road is classified into a plurality of stages of packed snow thickness;
a classification result of the roof snow depth when the state of roof snow accumulated on the roof of the building along the road is classified into a plurality of levels of roof snow depth;
a classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level;
A classification result of the degree of caution for fallen trees when the situation of fallen trees on the road is classified into a plurality of levels of the degree of caution for fallen trees, or
A classification result of the rain/snow classification when the rainfall or snowfall condition on the road is classified into a plurality of stages of rain/snow classifications.
The road condition determining device according to claim 5. an information providing unit that provides road condition summary information for tallying up the snow damage status judgment results determined by the judging unit for each road section or each time section, and superimposing and displaying the snow damage status summary results on a road map on a display screen;
The information providing unit is
When a designation of a specific point or a specific time on a road on which the tabulation results are superimposed on the road map is accepted, image display information is provided for displaying, on the display screen, in chronological order or in reverse chronological order, image data of the object to be determined that was taken while the object was traveling on the road before and after the point at which the image data of the object to be determined was taken, based on image data of the object to be determined that was taken at the point corresponding to the specific point or at the time corresponding to the specific time.
The road condition determination device according to any one of claims 1 to 6 . A method for causing a computer to function as each unit included in the road condition determination device according to any one of claims 1 to 6 ,
Road condition judgment program. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes.
Method for determining road conditions. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
a classification result of the stack caution level when the state of the road where the vehicle is likely to become stuck is classified into a plurality of stages of stack caution levels according to the type or weight of the vehicle;
Method for determining road conditions. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
A classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level.
Method for determining road conditions. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
a determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data consisting of image data of a learning target road and its surroundings and a correct answer label indicating a judgment result of the snow damage situation of the learning target road and its surroundings,
The snow damage situation is as follows:
a classification result of the degree of caution for fallen trees when the state of fallen trees with respect to the road and its road boundary is classified into a plurality of levels of the degree of caution for fallen trees according to the extent to which the fallen trees exist with respect to the road and its road boundary;
Method for determining road conditions. A data acquisition step of acquiring road data including image data of a road to be determined and its surroundings captured at a predetermined point and time;
A determination step of determining a snow damage situation of the road to be determined and its surroundings by inputting image data of the road to be determined, which is included in the road data acquired by the data acquisition step, into a learning model,
The image data to be determined is
an image of a front view of a vehicle traveling on a road to be determined is taken from the vehicle,
The determining step includes:
A left image region and a right image region are cut out from the image data to be determined to generate left image data and right image data;
The left side image data and the right side image data are input to the learning model to determine the snow damage situation on the one side of the road to be determined;
The other side of the left side image data and the right side image data is input to the learning model in a state where the other side is inverted in the left-right direction, thereby determining the snow damage situation on the other side of the road to be determined;
The learning model is
Machine learning is performed using a learning dataset including a plurality of learning data sets each including image data of a learning target road and its surroundings, the image data being of the one side of the learning target road and a correct answer label indicating a determination result of the snow damage situation on the one side of the learning target road.
Method for determining road conditions. The snow damage situation is as follows:
a classification result of the road surface classification when the road surface conditions are classified into a plurality of road surface classifications;
a classification result of the side margin width when the side margin condition of the road is classified into a plurality of stages of side margin width;
a classification result of the snow bank height when the state of the snow bank formed on the road is classified into a plurality of stages of snow bank height;
a classification result of the visibility distance when the visibility condition of the road due to snowfall or snowstorm is classified into a plurality of visibility distance stages;
A classification result of the number of passable lanes when the situation of the number of passable lanes of the road is classified into a plurality of stages of the number of passable lanes;
a classification result of the stack caution level when the state of the road where stacks are likely to occur is classified into a plurality of stack caution levels;
a classification result of the packed snow thickness when the packed snow condition on the road is classified into a plurality of stages of packed snow thickness;
a classification result of the roof snow depth when the state of roof snow accumulated on the roof of the building along the road is classified into a plurality of levels of roof snow depth;
a classification result of the snow cornice warning level when the state of the snow cornice formed on the roof of the building along the road is classified into a plurality of stages of the snow cornice warning level;
A classification result of the degree of caution for fallen trees when the situation of fallen trees on the road is classified into a plurality of levels of the degree of caution for fallen trees, or
A classification result of the rain/snow classification when the rainfall or snowfall condition on the road is classified into a plurality of stages of rain/snow classifications.
The method for determining road conditions according to claim 13. an information providing step of providing road condition summary information for tallying up the snow damage status judgment results determined by the judgment step for each road section or each time section, and superimposing and displaying the snow damage status summary results on a road map on a display screen;
The information providing step includes:
When a designation of a specific point or a specific time on a road on which the tabulation results are superimposed on the road map is accepted, image display information is provided for displaying, on the display screen, in chronological order or in reverse chronological order, image data of the object to be determined that was taken while the object was traveling on the road before and after the point at which the image data of the object to be determined was taken, based on image data of the object to be determined that was taken at the point corresponding to the specific point or at the time corresponding to the specific time.
The method for determining road conditions according to any one of claims 9 to 14 .

Images