JP7541232B2 - PERIOD ESTIMATION DEVICE, PERIOD ESTIMATION METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a period estimation device, a period estimation method, and a program.

The following Patent Document 1 describes a recognition method and device that can recognize the wear state of a tire from an image of the tire using a convolutional neural network.

JP 2019-35626 A

The technology described in Patent Document 1 can recognize the wear state of tires as consumables, but cannot estimate the usable period, which indicates how long the tire can be used for. It is difficult for general users who do not have specialized knowledge of the consumables to estimate the usable period of the consumables from the wear state of the consumables.

The present invention has been made in consideration of the above-mentioned circumstances, and its purpose is to provide a period estimation device, a period estimation method, and a program that can estimate the usable period of a consumable item.

To solve the above problem, the period estimation device according to the present invention includes an acquisition means for acquiring multiple images showing changes in a consumable over a predetermined period of time, and a period estimation means for estimating the usable period of the consumable based on the output when multiple degrees of wear based on the multiple images acquired by the acquisition means are input into a trained machine learning model.

In one aspect of the present invention, the acquisition means acquires, as the multiple images, images of the consumable taken by an image capture means provided at a predetermined location, or images generated based on the captured images.

In one aspect of the present invention, the device further includes a wear level estimation means for estimating multiple wear levels of the consumable during the specified period based on the output when the multiple images acquired by the acquisition means are input into a first machine learning model, and the period estimation means estimates the usable period of the consumable based on the output when the multiple wear levels estimated by the wear level estimation means are input into a second machine learning model, which is the trained machine learning model.

In one aspect of the present invention, the first machine learning model is a machine learning model trained by first training data consisting of an image of the consumable and the degree of consumption of the consumable, and the second machine learning model is a machine learning model trained by second training data consisting of multiple degrees of consumption of the consumable during the specified period and the usable period.

In one aspect of the present invention, each of the multiple images shows a wear surface of the consumable that wears down with use, the wear degree estimation means estimates the wear degree of the wear surface as the wear degree, and the period estimation means estimates the usable period of the consumable based on the output when the multiple wear degrees estimated by the wear degree estimation means are input to the second machine learning model.

In one aspect of the present invention, the wear surface is the tread surface of a tire, the wear degree estimation means estimates the groove depth in the tread surface as the wear degree, and the period estimation means estimates the usable period of the tire based on the output when information indicating the multiple groove depths estimated by the wear degree estimation means is input to the second machine learning model.

In one embodiment of the present invention, the specified period is one year or longer.

In one aspect of the present invention, the device further includes an output means for outputting the expiration date of the consumable based on the usable period estimated by the period estimation means.

The period estimation method according to the present invention also includes the steps of acquiring multiple images showing changes in a consumable over a predetermined period of time, and estimating the usable period of the consumable based on the output of a trained machine learning model that inputs multiple degrees of wear based on the multiple images acquired in the acquisition step.

The program of the present invention also causes a computer to execute an acquisition procedure for acquiring multiple images showing changes in a consumable over a predetermined period of time, and an estimation procedure for estimating the usable period of the consumable based on the output when multiple degrees of wear based on the multiple images acquired by the acquisition procedure are input into a trained machine learning model.

FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a left side view of the vehicle. FIG. 2 is a diagram illustrating a top surface of the vehicle. FIG. 13 is a diagram showing an example of a side image. FIG. 13 is a diagram showing an example of a wear surface image. FIG. 13 is a diagram showing another example of the wear surface image. FIG. 2 is a functional block diagram showing an example of functions of an information processing device. FIG. 13 is a diagram showing an example of a display of an expiration date. 13A and 13B are diagrams showing other examples of display of the expiration date. FIG. 10 is a diagram illustrating an example of time-series data. A figure showing an example of second training data used in machine learning of a second machine learning model. FIG. 13 is a diagram showing an example of recorded data relating to wear levels. FIG. 11 is a flow diagram showing an example of the flow of an estimation process executed in an information processing device. FIG. 13 is a diagram showing another example of recorded data relating to wear level.

Below, an embodiment of the present disclosure will be described with reference to the drawings. In this embodiment, a plurality of pieces of information based on a plurality of images showing changes in a consumable item are input to a machine learning model, and the usable period of the consumable item is estimated. Below, an example will be described in which the consumable item is a tire mounted on a vehicle, but the consumable item is not limited to tires and may be an item whose surface condition changes (wears) with use. The consumable item may be any item whose degree of wear with use can be observed from the outside.

[1. Hardware Configuration]
Fig. 1 is a diagram showing a configuration of an information processing device (period estimation device) 10 according to an example of an embodiment of the present invention. The information processing device 10 according to this embodiment is a computer such as a personal computer, and includes a processor 12, a storage unit 14, a communication unit 16, a display unit 18, an operation unit 20, a first imager 22a, and a second imager 22b, as shown in Fig. 1. The information processing device 10 may also include an optical disk drive that reads optical disks such as DVD-ROMs and Blu-ray (registered trademark) disks, a USB (Universal Serial Bus) port, and the like.

The processor 12 is, for example, a program-controlled device such as a CPU that operates according to a program installed in the information processing device 10. The memory unit 14 is, for example, a storage element such as a ROM or RAM, or a hard disk drive. The memory unit 14 stores programs executed by the processor 12, etc. The communication unit 16 is, for example, a communication interface such as a network board. The display unit 18 is a display device such as a liquid crystal display, and displays various images according to instructions from the processor 12. The operation unit 20 is a user interface such as a keyboard or mouse, which accepts operation input from the user and outputs a signal indicating the content to the processor 12.

The first image capturing unit 22a and the second image capturing unit 22b are image capturing devices that output images generated by capturing to the processor 12. The first image capturing unit 22a may be a digital camera or the like. It is preferable that the second image capturing unit 22b be a stereo camera capable of capturing three-dimensional images or a depth camera capable of capturing depth images. In this case, the second image capturing unit 22b outputs the three-dimensional image and depth image generated by capturing to the processor 12.

Figure 2 is a schematic diagram of the left side of the vehicle 90. Figure 3 is a schematic diagram of the top of the vehicle 90. The vehicle 90 is fitted with tires, which are consumable items, and is parked in a predetermined location (such as a parking lot). As shown in Figures 2 and 3, the first photographing unit 22a and the second photographing unit 22b are disposed in the predetermined location (such as a parking lot) where the vehicle 90 is parked. The first photographing unit 22a is disposed in a position where it can photograph the left side of the left rear wheel 92 of the vehicle 90, and the second photographing unit 22b is disposed in a position where it can photograph the tread surface of the left rear wheel 92 of the vehicle 90.

Figure 4 is a diagram showing an example of a side image 110, which is an image of the left side of a tire, captured by the first imaging unit 22a. As shown in Figure 4, the side image 110 shows multiple side slip signs 94 stamped on the side (shoulder) of the tire. As shown in Figure 4, the multiple side slip signs 94 are stamped on the side of the tire at equal intervals along the outer circumference of the tire. Generally, 4 to 9 side slip signs 94 are stamped on the side of the tire at equal intervals along the outer circumference of the tire. The tire shown in Figure 4 has six side slip signs 94 stamped on it.

5A and 5B are diagrams showing an example of a tread surface image 120, which is an image of the tread surface of a tire, captured by the second image capture unit 22b. As shown in FIGS. 5A and 5B, the tread surface image 120 shows a main groove 96, which is a groove formed on the tread surface, and a tread surface slip sign 98 is shown inside the main groove 96. Note that the tread surface image 120 may also show a periodic structure of the tread pattern. The tread pattern is omitted from the tread surface image 120 shown in FIGS. 5A and 5B.

The tread surface image 120 shown in FIG. 5B was taken on a day after the tread surface image 120 shown in FIG. 5A was taken. The tread surface image 120 shown in FIG. 5B shows the tread surface of a tire that has reached its limit of use. In this tire, the depth of the main grooves 96 has decreased due to wear of the tread surface, and the boundary between the main grooves 96 and the tread surface slip sign 98 has become unclear. It is desirable to replace the tires mounted on the vehicle 90 with new tires before they reach this state.

When a user of the vehicle 90 (such as an individual owner or manager) determines when to replace a tire based on the amount of wear (amount of wear on the tread surface) of the tire, the user typically measures the depth of the main groove 96. However, since manually measuring the depth of the main groove 96 using a caliper or the like is time-consuming, the user of the vehicle 90 rarely has the opportunity to actually check the amount of wear on the tire. For this reason, the user of the vehicle 90 may not realize that the tire will soon reach its limit of use, or that it has already reached its limit of use.

In addition, it may be desirable to output the expiration date of a consumable item to a user in advance, before the consumable item reaches its usage limit, not limited to tires. In this regard, as described in detail below, the information processing device 10 estimates the usable period of a consumable item such as a tire, and outputs the expiration date to the display unit 18 or the display unit of another information processing device (for example, a mobile terminal such as a smartphone). This allows the user of the consumable item to obtain useful information regarding the purchase of a new consumable item, refilling, or determining when to next replace the item.

2. Functional Blocks
The information processing device 10 estimates the usable period of a consumable (such as a tire). In this embodiment, the usable period of the tire of the left rear wheel 92 is estimated based on multiple images captured by the second imaging unit 22b, and the expiration date of the tire is output. In addition, two trained machine learning models (a first machine learning model 40 and a second machine learning model 42 shown in FIG. 6 ) are used until the usable period of the consumable is estimated from the multiple images. The estimation of the usable period of the consumable by the information processing device 10 will be described below.

Figure 6 is a functional block diagram showing an example of functions implemented in information processing device 10. Note that not all of the functions shown in Figure 6 need to be implemented in information processing device 10, and functions other than the functions shown in Figure 6 may be implemented.

As shown in FIG. 6, the information processing device 10 functionally includes a data storage unit 30, an image acquisition unit 32, a wear level estimation unit 34, a period estimation unit 36, an expiration date output unit 38, a first machine learning model 40, and a second machine learning model 42. The data storage unit 30 is implemented primarily in the memory unit 14. The image acquisition unit 32, the wear level estimation unit 34, the period estimation unit 36, and the expiration date output unit 38 are implemented primarily in the processor 12. The first and second learning models 40, 42 are implemented primarily in the processor 12 and the memory unit 14.

[2-1. Expiration date output section]
The expiration date output unit 38 outputs the expiration date of the consumables (such as tires) based on the usable period estimated by the period estimation unit 36, which will be described later in detail. In this embodiment, the expiration date output unit 38 The expiration date of the consumable item is output to the display unit 18 or to a display unit of another information processing device via the communication unit 16 or the like.

Figure 7A is a diagram showing an example of the display of the expiration date. As shown in image 130A in Figure 7A, the expiration date output unit 38 may output the number of days of the usable period estimated by the period estimation unit 36 described below. Furthermore, the expiration date output unit 38 may output to the display unit 18, etc., not limited to the number of days of the usable period, but also a numerical value indicating hours, minutes, and seconds, the number of months, the number of years, etc.

Figure 7B is a diagram showing another example of the display of the expiration date. As shown in image 130B in Figure 7B, the expiration date output unit 38 may output the expiration date based on the usable period estimated by the period estimation unit 36 described below. In this case, the expiration date output unit 38 may output the date obtained by adding the usable period estimated by the period estimation unit 36 to the current date as the usable date of the consumable.

[2-2. Data storage unit]
6, the data storage unit 30 stores time-series data D1 of the degree of wear estimated by the wear-out degree estimation unit 34. In addition, for example, the data storage unit 30 may store various data such as image data captured by the first imaging unit 22a and the second imaging unit 22b, second training data used for machine learning of the second machine learning model 42 described later, and recorded data related to the degree of wear-out for generating the second training data. The time-series data D1 will be described in detail later.

[2-3. Image Acquisition Unit]
The image acquisition unit 32 acquires multiple images showing changes in the consumables (such as tires) over a predetermined period (e.g., three months) as images to be input to the wear level estimation unit 34, which will be described later in detail. The image acquisition unit 32 may acquire, as multiple images, images taken periodically over a predetermined period by an imaging means provided at a predetermined location, or images generated based on the captured images.

Each of the multiple images showing changes in a consumable may, for example, show a wear surface of the consumable that wears down with use. Alternatively, for example, each of the multiple images may show the remaining amount of ink or the like that is consumed with use.

In this embodiment, the image acquisition unit 32 acquires images generated based on tread surface images 120 captured periodically (e.g., once a week) by the second image capture unit 22b located in the parking lot over a predetermined period (e.g., three months) as multiple images showing changes during the predetermined period. Note that the image acquisition unit 32 may acquire the tread surface images 120 captured by the second image capture unit 22b themselves, or images captured by the second image capture unit 22b and then stored in the storage unit 14 as the multiple images.

"Images generated based on the tread surface image 120" refers to images generated by applying a given image processing process (e.g., a trimming process, a zoom-in/zoom-out process, or an edge extraction process) to the tread surface image 120. In this embodiment, the image acquisition unit 32 acquires, as the above-mentioned multiple images, images that have been trimmed in the tread surface image 120 at a region of interest 122 (see Figures 5A and 5B), which is an area that includes the tread surface slip sign 98.

The area of interest 122 in the tread surface image 120 may be identified based on the side image 110 (see FIG. 4) captured by the first image capture unit 22a at the same time that the second image capture unit 22b captures the tread surface image 120. As described above, the side image 110 captures the tire side slip sign 94. In the tire, the position of the side slip sign 94 and the position of the tread surface slip sign 98 correspond one-to-one. Therefore, the position of the area of interest 122 in the tread surface image 120 can be identified based on the position of the side slip sign 94 in the side image 110. The position of the area of interest 122 in the tread surface image 120 can be identified, for example, based on the angle a between a reference line (a line in the right direction in the example of FIG. 4) passing through the center of the left rear wheel 92 in the side image 110 and a line connecting the center and the side slip sign 94.

The multiple tread surface images 120 captured periodically may include images captured at a time slightly shifted from the period (e.g., one week). The multiple tread surface images 120 captured in this manner can also be called multiple images showing changes in the tire over a specified period (e.g., three months). In addition, among the multiple times at which images are captured periodically, there may be times when no images are captured due to reasons such as the vehicle 90 not being present in the location where the second imaging unit 22b is located.

[2-4. Wear level estimation unit (first machine learning model)]
The wear degree estimation unit 34 estimates the wear degree of a consumable (such as a tire) based on an image of the consumable acquired by the image acquisition unit 32. More specifically, the wear degree estimation unit 34 estimates a plurality of wear degrees of the consumable during a predetermined period (e.g., three months) based on an output when a plurality of images showing changes in the consumable during the predetermined period, which are acquired by the image acquisition unit 32, are input to the first machine learning model 40.

"Wear level" is an index value indicating the degree of wear of a consumable. In the case of a consumable whose surface condition wears away with use, as in this embodiment, the wear level can be rephrased as "wear level." In the case of a consumable being a tire, the wear level (wear level) of the tire may be data indicating the groove depth (remaining groove amount) of the main groove 96 on the tread surface. The wear level may also be an index value indicating the amount of wear from when the consumable was not in use.

The first machine learning model 40 is a machine learning model that implements machine learning techniques such as AdaBoost, random forest, neural network, support vector machine (SVM), and nearest neighbor classifier.

The first machine learning model 40 is a machine learning model that learns from training data (hereinafter, also referred to as first training data to distinguish it from the second training data described below) consisting of an image of a consumable and the degree of consumption of the consumable (teacher data). The first machine learning model 40 is, for example, a regression model that receives an image of a consumable and outputs data indicating the degree of consumption of the consumable.

The first training data consists of an image showing the wear surface of the consumable (for example, a tread surface image 120 showing the tread surface of a tire) and the degree of wear of the wear surface. The first machine learning model 40 trained using such first training data becomes a regression model that outputs data showing the degree of wear of the consumable when an image showing the wear surface of the consumable is input. In this case, the wear degree estimation unit 34 estimates the degree of wear of the wear surface of the consumable as the degree of wear based on the output when an image showing the wear surface is input to the first machine learning model 40.

In this embodiment, the first training data consists of an image generated based on the tread surface image 120 and data such as a numerical value indicating the groove depth of the main groove 96 measured in advance with a caliper or the like in the tire shown in the tread surface image 120. Therefore, in this embodiment, the first machine learning model 40 is a regression model that outputs data indicating the groove depth of the main groove 96 in response to the input of the tread surface image 120. Therefore, in this embodiment, the wear degree estimation unit 34 estimates the depth of the main groove 96 in the tread surface as the wear degree (wear degree) of the tire based on the output when the image generated based on the tread surface image 120 is input to the first machine learning model 40.

In this embodiment, the degree of wear (wear) of a tire is estimated by inputting an image of the tread surface image 120 trimmed in a region of interest 122 including the tread surface slip sign 98 to the first machine learning model 40. The tread surface slip sign 98 exists regardless of the type of tire, and it is considered that the difference in the image features near the tread surface slip sign 98 due to the difference in the degree of wear of the tire is more significant than the difference in the image features near the tread surface slip sign 98 due to the difference in the type of tire. Therefore, by using the image trimmed in the region of interest 122 as input to the first machine learning model 40, it becomes possible to accurately estimate the degree of wear of various types of tires even if the amount of data used for learning is small.

Although not shown in the figures, the information processing device 10 may have a first training data generation means for generating first training data via the second imaging unit 22b or the operation unit 20, and a first learning means for executing learning of the first machine learning model 40 using the first training data. In this case, the first learning means may, for example, identify the difference between the output when an image of a consumable included in the first training data is input to the first machine learning model 40 and the degree of consumption included in the training data, and update the parameters of the first training data based on the identified difference.

[2-5. Time series data]
The wear-out degree estimation unit 34 stores the time-series data of the wear-out degrees estimated as described above in the data storage unit 30. In the data storage unit 30, a predetermined number (e.g., 15) of wear-out degrees estimated by the wear-out degree estimation unit 34 are accumulated as time-series data D1 as a plurality of wear-out degrees for a predetermined period (e.g., three months).

Figure 8 is a diagram showing an example of time-series data D1 stored in the data storage unit 30. As shown in Figure 8, the time-series data D1 is a one-dimensional array of values indicating the degree of wear (e.g., the groove depth of the main groove 96). In the example shown in Figure 8, "2.5" is the wear value most recently estimated by the wear estimation unit 34, "2.6" is the wear value estimated at the timing just before that, and "2.7" is the wear value estimated at the timing just before that.

The number of wear-out data items (the number of elements in a one-dimensional array) stored in the time-series data D1 may be a number determined according to the above-mentioned predetermined period. For example, if the wear-out estimation unit 34 estimates the wear-out level every seven days (one week) and the number of wear-out data items stored in the time-series data D1 is 15, data indicating the wear-out level for 99 days (7 days * 14 weeks + 1 day (* is a multiplication operator), i.e., approximately three months) will be stored in the time-series data D1.

As mentioned above, the multiple tread surface images 120 captured periodically may include images captured slightly off-set from the period (e.g., one week). The time-series data D1 may include the degree of wear at a time slightly off-set from the regular period.

In addition, if there is a time when no image is captured due to the absence of the vehicle 90 among the multiple times when image capture is performed periodically, the time series data D1 may include a wear level value that is complemented based on the wear level values estimated before and after that time (e.g., the same value as the previous time or the average value of the values before and after). In this case, the information processing device 10 may have a means for complementing the wear level value in the time series data D1 based on the time series data D1 stored in the data storage unit 30.

[2-6. Period Estimation Unit (Second Machine Learning Model)]
The period estimation unit 36 estimates the usable period of the consumable (such as a tire) based on an output when a plurality of degrees of wear based on a plurality of images showing changes in the consumable over a predetermined period (e.g., three months) acquired by the image acquisition unit 32 are input to the trained second machine learning model 42. More specifically, the period estimation unit 36 estimates the usable period of the consumable based on an output when a plurality of degrees of wear estimated by the wear degree estimation unit 34 are input to the second machine learning model 42.

In this embodiment, the period estimation unit 36 estimates the usable period of the consumable based on the output when the multiple wear degrees estimated by the wear degree estimation unit 34 are input to the second machine learning model 42. In this embodiment, the consumable is a tire, and the period estimation unit 36 estimates the usable period of the tire based on the output when the groove depth of the main groove 96 estimated by the wear degree estimation unit 34 is input to the second machine learning model 42. The period estimation unit 36 estimates the usable period of the tire, which is a consumable, based on the output when the time series data D1 stored in the data storage unit 30 is input to the second machine learning model 42.

The second machine learning model 42 is a machine learning model in which a given machine learning is implemented, similar to the first machine learning model 40. For example, the second machine learning model 42 may be implemented with a deep neural network (DNN) as the machine learning.

The second machine learning model 42 is a machine learning model that learns from second training data consisting of multiple wear levels of consumables (such as tires) over a predetermined period (e.g., three months) and the usable period (teacher data) of the consumables that has been measured in advance. The second machine learning model 42 is a regression model that receives multiple wear levels as input and outputs data indicating the usable period of the consumables.

Figure 9 is a diagram showing an example of second training data used in the machine learning of the second machine learning model 42. As shown in Figure 9, the second training data is, for example, data that accumulates multiple combinations of multiple wear levels and expiration dates over a specified period of time.

The second training data shown in FIG. 9 indicates that for wear levels estimated in the order of "6.4, 6.3, 6.1, 5.9, 5.8," the usable life of the consumable at the time "5.8" was estimated was "350" days. Also, the second training data shown in FIG. 9 indicates that for wear levels estimated in the order of "6.3, 6.1, 5.9, 5.8, 5.7," the usable life of the consumable at the time "5.7" was estimated was "330" days.

Figure 10 is a diagram for explaining a method for creating the second training data, and shows an example of recorded data related to the degree of wear (abrasion) when a tire mounted on a vehicle 90 is used to its limit. The recorded data shown in Figure 10 may be recorded, for example, by one of multiple trial users who are users of the vehicle 90 and who cooperate in creating the second training data.

In the example shown in FIG. 10, the horizontal axis indicates the passage of time, and the vertical axis indicates the groove depth of the main groove 96, which is the degree of wear (abrasion) of the tire. The recorded data shown in FIG. 10 shows that groove depth measurements have been taken periodically (e.g., once a week) since early January 2017, and that the groove depth has decreased over time, i.e., the tread surface of the tire is gradually wearing away with use. The recorded data shown in FIG. 10 also shows that in mid-March 2020, the tire's groove depth fell below the specified value (2.0 mm or less), indicating that the tire had reached its expiration date.

The second training data can be created, for example, as follows. First, in the recorded data shown in FIG. 10, multiple (e.g., 15) tread depths measured during a period M1 from the first measurement of the tire tread depth (early January 2017) until a predetermined period (e.g., three months) has passed are obtained, and the period from the last day of the period M1 (the end of March 2017) until the expiration date (mid-March 2020) is obtained as the usable period. Then, data of the combinations of the multiple tread depths and usable periods thus obtained is added to the second training data. In other words, the second training data is updated so that the combinations of the multiple tread depths and usable periods obtained become one of the multiple combinations included in the second training data.

Next, in period M2, which is a predetermined period shifted from period M1 by one recording cycle (e.g., one week), multiple groove depths and usable periods from the last day of period M2 are similarly obtained, and the combinations of the obtained multiple groove depths and usable periods are added to the second training data. Furthermore, in period M3, which is a predetermined period shifted from period M2 by one recording cycle, multiple groove depths and usable periods are similarly obtained and included in the second training data.

By repeating the above process, it is possible to obtain multiple combinations of teacher data from one recorded data about tire wear (abrasion) and add them to the second training data. Furthermore, by obtaining multiple recorded data from users of multiple vehicles 90 as trial users, it is possible to accumulate a large amount of teacher data in the second training data.

In addition, by machine learning the second machine learning model 42 using the second training data that includes a large amount of teacher data, the period estimation unit 36 can use the second machine learning model 42 to estimate a usable period with a high likelihood as the usable period of the tire.

Although not shown in the figures, the information processing device 10 may have a second training data generation means for generating second training data, and a learning means for executing learning of the second machine learning model 42 using the second training data. In this case, the information processing device 10 may have a means for generating the recorded data shown in FIG. 10 based on multiple past wear levels that are periodically estimated by the period estimation unit 36 and stored in the data storage unit 30, and the second training data generation means may generate the second training data based on the generated recorded data.

3. Flowchart
Fig. 11 is a flow diagram showing an example of the flow of the estimation process executed by the information processing device 10. The estimation process shown in Fig. 11 is executed periodically (for example, once a week).

First, the image acquisition unit 32 acquires an image of the consumable item in which the degree of wear of the consumable item can be observed (step S101). If the consumable item has a wear surface that wears with use, the image acquisition unit 32 may acquire an image showing the wear surface. If the consumable item is a tire, the image acquisition unit 32 may acquire a tread surface image 120 as exemplified in FIGS. 5A and 5B. In this embodiment, the image acquisition unit 32 acquires an image of the tread surface image 120 cropped to a region of interest 122 that is an area including the tread surface slip sign 98.

Next, the wear degree estimation unit 34 inputs the image of the consumable obtained in step S101 into the first machine learning model 40, and estimates the wear degree of the consumable based on the output from the first machine learning model 40 at that time (step S102). In this embodiment, the wear degree estimation unit 34 estimates the groove depth of the main groove 96 as the wear degree (wear degree) of the tire.

The wear degree estimation unit 34 also records the wear degree estimated in step S102 in the time series data D1 stored in the data storage unit 30 (step S103). In the time series data, a predetermined number (e.g., 15) of wear degree data (in this embodiment, numerical data indicating the groove depth of the main groove 96) are recorded as data for a predetermined period (e.g., three months) (see FIG. 8).

Next, the period estimation unit 36 inputs the time series data D1 stored in the data storage unit 30 into the second machine learning model 42, and estimates the usable period of the consumable based on the output from the second machine learning model 42 at that time (step S104).

Finally, the expiration date output unit 38 outputs the expiration date of the consumable to the display unit 18 or to a display unit of another information processing device based on the usable period estimated in step S104 (step S105), and the process ends. The expiration date output unit 38 may output, for example, an image 130A indicating the number of days of the usable period as shown in FIG. 7A, or an image 130B indicating the date of the expiration date as shown in FIG. 7B, as an image indicating the expiration date of the consumable.

In step S103, if the number of wear data recorded in the time series data D1 has not reached a predetermined number (e.g., 15) (if a predetermined period (e.g., three months) has not elapsed since recording in the time series data D1 began), the processor 12 of the information processing device 10 may not execute the processes of steps S104 and S105. In addition, if the number of wear data in the time series data D1 has not reached the predetermined number, the period estimation unit 36 may complement the time series data D1 so that the number of wear data becomes the predetermined number, and input this to the second machine learning model 42 to estimate the provisional usable period of the consumable.

[4. Summary]
As described above, in this embodiment, the image acquisition unit 32 acquires an image showing the region of interest 122 in the tread surface image 120 captured by the second image capture unit 22b as a plurality of images showing the change in the consumable (such as a tire) over a predetermined period (e.g., three months). The wear degree estimation unit 34 estimates a plurality of wear degrees by inputting the plurality of images into the first machine learning model 40. Then, the period estimation unit 36 estimates the usable period of the consumable based on the output when the plurality of estimated wear degrees are input into the second machine learning model. In this manner, the information processing device 10 can estimate the usable period of the consumable. In addition, the expiration date output unit 38 outputs the expiration date of the consumable on the display unit 18 or the like, so that the user of the consumable can grasp information useful for making decisions such as purchasing the consumable.

In the above explanation, the usable period of the tire on the left rear wheel 92 of the vehicle 90 has been estimated, but it is possible to estimate the usable period of tires other than the tire on the left rear wheel 92 of the vehicle 90 using a similar method. In the example of tires, it is often the case that all of the multiple tires mounted on the vehicle 90 are replaced at the same time. For this reason, it is not necessary to estimate the usable period of all of the multiple tires mounted on the vehicle 90, and it is often sufficient to estimate the usable period of one tire.

5. Modifications
The present invention is not limited to the above embodiments.

For example, in this embodiment, a case has been described in which the predetermined period is three months, and wear-out data for three months is recorded in the time-series data D1, and the second machine learning model 42 is machine-learned using second training data consisting of wear-out data for three months. However, the predetermined period may be one year or longer.

For example, one year's worth of wear-out data may be recorded in the time-series data D1, and the second machine learning model 42 may perform machine learning using second training data consisting of one year's worth of wear-out data. In this case, the time-series data D1 in which one year's worth of wear-out data is recorded is input to the second machine learning model 42.

Figure 12 shows another example of recorded data regarding the degree of wear when a tire mounted on a vehicle 90 is used to its limit. In the example of Figure 12, similar to the example of Figure 10, the horizontal axis indicates the passage of time, and the vertical axis indicates the groove depth of the main groove 96, which is the degree of wear (degree of wear) of the tire.

In the example shown in FIG. 12, as shown in the area surrounded by the dashed line, a significant decrease in tire tread depth is observed from early to mid-March 2019 during the one-year period M4. In other words, the frequency of use of vehicle 90 increased during the period March 2019.

Depending on the situation of the user of the vehicle 90, the frequency of use of the vehicle 90 may be high in March as shown in FIG. 12, or may be high at times other than March. In addition, the frequency of use of the vehicle 90 may be high not only at one time in a year, but at multiple times, or may be high for multiple consecutive months.

Therefore, the specified period may be one year or more, and the period estimation unit 36 may estimate the usable period of the consumable based on the output when multiple wear levels based on multiple images showing changes in the consumable (such as tires) over a specified period of one year or more (i.e., time-series data D1 in which wear levels for one year or more are recorded) are input to the second machine learning model 42. In this way, the usable period can be estimated based on the frequency of use of the consumable, which changes at each time point in the year. For example, when a period of high wear of the consumable arrives, it becomes possible to estimate the usable period of the consumable based on the fact that the consumable will be worn out even more from now on.

The specified period is not limited to one year, but may be one year or more (e.g., two years or three years). In the embodiment, the period during which the tread surface image 120 is captured by the second image capture unit 22b is one week as the period during which the image of the consumable item is acquired, but the period does not have to be one week.

In the embodiment, an example in which the consumable is a tire has been described, but the consumable may be anything whose surface wears away with use, such as the soles of shoes at the entrance of a house, or railway tracks or pantographs. The consumable may also be anything whose degree of wear due to use can be observed from the outside, such as a marker whose remaining ink level can be checked from the outside. The present invention can also be applied to estimating the usable period of such consumables.

10 Information processing device, 12 Processor, 14 Memory unit, 16 Communication unit, 18 Display unit, 20 Operation unit, 22a First imaging unit, 22b Second imaging unit, 30 Data memory unit, D1 Time series data, 32 Image acquisition unit, 34 Wear degree estimation unit, 36 Period estimation unit, 38 Expiration date output unit, 40 First machine learning model, 42 Second machine learning model, 90 Vehicle, 92 Left rear wheel, 94 Side slip sign, 96 Main groove, 98 Tread surface slip sign, 110 Side image, 120 Tread surface image, 122 Area of interest, 130A, 130B Images.

Claims (9)

an acquisition means for acquiring a predetermined number of images (two or more) showing a change in the consumable over a predetermined length of time;
a consumption degree estimation means for estimating a consumption degree of the predetermined number of the consumables in the predetermined length of time based on an output when the predetermined number of images acquired by the acquisition means are input into a first machine learning model;
and a period estimation means for estimating a usable period of the consumable based on an output when the predetermined number of degrees of consumption estimated by the degree of consumption estimation means are input to a trained second machine learning model,
A period estimation device, wherein the second machine learning model is a model learned using training data including a plurality of combinations of the predetermined number of levels of consumption for a second consumable that is the same type of consumable as the consumable during the period of the predetermined length and the usable period of the second consumable from the period, the plurality of combinations being obtained by shifting the period. The period estimation device according to claim 1 , wherein the acquisition means acquires, as the predetermined number of images, images of the consumable taken by an imaging means provided at a predetermined location, or images generated based on the taken images. The period estimation device according to claim 1 or 2, wherein the first machine learning model is a machine learning model trained by first training data including an image of the consumable and a consumption degree of the consumable. each of the predetermined number of images represents a wear surface of the consumable product that wears away with use;
The wear degree estimation means estimates a wear degree of the wear surface as the wear degree,
The period estimation device according to claim 1 , wherein the period estimation means estimates the usable period of the consumable based on an output when the predetermined number of the wear levels estimated by the wear level estimation means are input into the second machine learning model. the wear surface being a tread surface of a tire;
The wear degree estimation means estimates a depth of a groove on the tread surface as the wear degree,
The period estimation device according to claim 4, wherein the period estimation means estimates the usable period of the tire based on an output when information indicating the depth of the predetermined number of grooves estimated by the wear degree estimation means is input to the second machine learning model. The period estimation device according to claim 1 , wherein the predetermined length of time is one year or more. The period estimation device according to claim 1 , further comprising an output unit that outputs an expiration date of the consumable item based on the usable period estimated by the period estimation unit. An acquisition step in which an acquisition means acquires a predetermined number of images, which are two or more, showing a change in the consumable over a predetermined length of time;
a consumption degree estimation step in which a consumption degree estimation means estimates a consumption degree of the predetermined number of the consumables in the predetermined length of time period based on an output when the predetermined number of images acquired in the acquisition step are input to a first machine learning model;
a period estimation step in which a period estimation means estimates the usable period of the consumable based on an output when the predetermined number of degrees of consumption estimated in the degree of consumption estimation step are inputted into a second machine learning model trained with training data including a plurality of combinations of the predetermined number of degrees of consumption in the predetermined period for a second consumable that is the same type of consumable as the consumable and the usable period of the second consumable from the period, the plurality of combinations being obtained by shifting the period. an acquisition step of acquiring a predetermined number of images, two or more, showing a change in the consumable over a predetermined length of time;
a consumption degree estimation step of estimating consumption degrees of the predetermined number of the consumables in the predetermined length of time period based on an output when the predetermined number of images acquired by the acquisition step are input to a first machine learning model; and
A program that causes a computer to execute an estimation procedure for estimating the usable period of the consumable based on the output when the specified number of degrees of consumption estimated in the consumption degree estimation procedure are input into a second machine learning model that has been trained using training data that includes multiple combinations of the specified number of degrees of consumption in the specified period for a second consumable that is the same type of consumable as the consumable, and the usable period of the second consumable from that period, the multiple combinations being obtained by shifting the period.

Images