JP7699996B2 - Aggregate determination device and aggregate determination method - Google Patents

Description

The present invention relates to an aggregate judgment device and an aggregate judgment method that judge the suitability of aggregate fed into a conveyor.

Conventionally, concrete plants are equipped with multiple silos for storing aggregate, the raw material for concrete. Aggregates transported by truck from a quarry are transported by conveyors within the concrete plant and stored in designated silos for each type of aggregate. Conventional concrete plants are described, for example, in Patent Document 1.

Japanese Patent Application Publication No. 08-091580

Aggregates are classified into several types according to characteristics such as particle size, color, and place of origin. In concrete plants, a silo is designated for storing each type of aggregate. When aggregates are dumped from a truck onto the conveyor, the truck driver or a designated worker inputs the type of aggregate into the operation panel. This sets the destination for the aggregates to be transported by the conveyor to the silo that corresponds to that type of aggregate.

If the wrong aggregate is put on the conveyor, or if the wrong type of aggregate is entered on the control panel, the put-in aggregate will be transported to the wrong silo. This will result in the silo being mixed with aggregate of a different type. In this case, it will not be possible to properly control the quality of the ready-mix concrete being produced, and it will become necessary to remove all of the aggregate from the silo where the mixing has occurred. This will result in a great deal of work and damage.

The present invention has been developed in consideration of these circumstances, and aims to provide an aggregate determination device and an aggregate determination method that can determine whether the aggregate type fed into the conveyor matches the aggregate type input into the operation panel.

In order to solve the above problems, the first invention of the present application is an aggregate judgment device that judges whether aggregate fed into a conveyor is suitable, and includes a camera that photographs the aggregate transported by the conveyor, a control panel to which information on the type of aggregate is input, and an information processing unit that is communicatively connected to the camera and the control panel. The information processing unit has an estimation unit that inputs images of the aggregate captured by the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model, and a judgment unit that judges whether the aggregate type input into the control panel matches the estimation result.

The second invention of the present application is the aggregate determination device of the first invention, in which the camera captures a video of the aggregate being transported by the conveyor, the video including a plurality of frame images having a blur in the transport direction of the aggregate, and the information processing unit inputs the frame images included in the video captured by the camera to the learning model.

The third invention of the present application is the aggregate determination device of the second invention, and the information processing unit further has a learning unit that generates the learning model by performing machine learning using multiple frame images included in a video of a sample aggregate prepared for learning and the known aggregate type of the sample aggregate as training data.

The fourth invention of the present application is the aggregate determination device of the third invention, in which the learning unit performs image processing other than rotation on the frame image and then performs the machine learning.

The fifth invention of the present application is an aggregate judgment device according to the third or fourth invention, in which the operation panel is capable of inputting correct/incorrect information indicating whether the judgment result by the judgment unit is correct or not, and the learning unit updates the learning model by performing additional machine learning using the image of the aggregate obtained by the camera, the estimation result, and the correct/incorrect information as training data.

The sixth invention of the present application is an aggregate determination device according to any one of the first to fifth inventions, further comprising a light source that irradiates light toward the imaging range of the camera.

The seventh invention of the present application is the aggregate determination device of the sixth invention, further comprising a hood that covers a portion of the upper surface of the conveyor, and the camera and the light source are disposed within the hood.

The eighth invention of the present application is an aggregate determination device according to any one of the first to seventh inventions, further comprising a result output unit that outputs the determination result of the determination unit.

The ninth invention of the present application is an aggregate determination device according to any one of the first to eighth inventions, further comprising a moisture sensor that measures the moisture content of the surface of the aggregate transported by the conveyor, and the information processing unit has a first learning model generated by performing the machine learning on aggregate whose surface moisture content is less than a predetermined value, and a second learning model generated by performing the machine learning on aggregate whose surface moisture content is equal to or greater than the predetermined value, and the estimation unit inputs an image of the aggregate taken by the camera to the first learning model when the moisture content measured by the moisture sensor is less than the predetermined value, and outputs an estimation result of the aggregate type from the first learning model, and inputs an image of the aggregate taken by the camera to the second learning model when the moisture content measured by the moisture sensor is equal to or greater than the predetermined value, and outputs an estimation result of the aggregate type from the second learning model.

The tenth invention of the present application is an aggregate determination device according to any one of the first to ninth inventions, in which the estimation unit inputs an odd number of images obtained from the camera to the learning model and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model, and the determination unit compares the aggregate type input to the operation panel with the odd number of estimation results and makes a determination by majority vote of the comparison results.

The eleventh invention of the present application is an aggregate determination device according to any one of the first to tenth inventions, in which the information processing unit further has a control unit that restricts the transport of aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimated result.

The twelfth invention of the present application is an aggregate judgment method for judging the suitability of aggregate fed into a conveyor, comprising the steps of: a) inputting information on the aggregate type of the aggregate fed into the conveyor into an operation panel; b) photographing the aggregate being transported by the conveyor; c) inputting the image of the aggregate obtained by photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; and d) judging whether the aggregate type input in step a) matches the estimation result.

According to the first to twelfth inventions of the present application, it is possible to determine whether the aggregate fed into the conveyor matches the aggregate type input into the operation panel.

In particular, according to the second invention of the present application, a video of the aggregate is captured, and frame images with blur in the transport direction are input to the learning model. The camera that captures the aggregate does not need to be a high-speed camera with a high frame rate that can capture frame images without blur in the transport direction, or a high-performance camera with a large number of pixels; instead, an easily available, inexpensive camera is sufficient. This makes it possible to reduce the cost of the camera. Furthermore, by using an easily available, inexpensive camera, the replacement cost in the event of a camera failure can also be reduced. Therefore, an aggregate judgment device with excellent maintainability can be realized.

In particular, according to the third invention of the present application, many frame images included in a video are used as learning images. Therefore, many aggregate images required to generate a learning model can be acquired in a relatively short time. Furthermore, a learning model with sufficient estimation accuracy can be created with blurred frame images. Furthermore, the storage capacity of the aggregate determination device can be reduced compared to when many fine images are prepared. In other words, a learning model with good estimation accuracy can be generated with an inexpensive equipment configuration.

In particular, according to the fifth aspect of the present invention, the learning model can be updated by performing additional machine learning. This allows the estimation accuracy of the aggregate type by the learning model to be maintained at a certain level. Furthermore, by storing images of the aggregate, estimation results, and correct/incorrect information, additional machine learning can be performed quickly.

In particular, according to the sixth aspect of the present invention, the variation in brightness can be suppressed, allowing for more accurate estimation of aggregate type.

In particular, according to the seventh aspect of the present invention, the influence of external ambient light is suppressed, allowing for more accurate estimation of aggregate type.

In particular, according to the eighth aspect of the present invention, the user can recognize the judgment result of the judgment unit.

In particular, according to the ninth aspect of the present invention, different learning models are used when the moisture content of the surface of the aggregate is less than a predetermined value and when it is equal to or greater than the predetermined value. This allows for more accurate estimation of the aggregate type.

In particular, according to the tenth aspect of the present invention, more accurate estimation results can be obtained based on an odd number of images.

In particular, according to the eleventh invention of the present application, when there is a possibility that inappropriate aggregate has been added, the transport of the aggregate to the silo can be restricted.

1 is a diagram showing a configuration of an aggregate transport device according to a first embodiment. FIG. FIG. 2 is a block diagram conceptually showing functions realized in an information processing unit of the first embodiment. 4 is a flowchart showing a flow of a learning process in the first embodiment. FIG. 13 is a diagram showing an example of learning images extracted from a video. 4 is a flowchart showing a flow of a determination process in the first embodiment. FIG. 11 is a diagram showing a configuration of an aggregate transport device according to a second embodiment. 10 is a flowchart showing a flow of a learning process in the second embodiment. 10 is a flowchart showing a flow of a determination process in a second embodiment. 13 is a flowchart showing a flow of a determination process in the third embodiment. 13 is a flowchart showing the processing flow when performing additional machine learning in the fourth embodiment.

A preferred embodiment of the present invention will be described below with reference to the drawings.

1. First embodiment
<1-1. Configuration of aggregate transport device>
Fig. 1 is a diagram showing the configuration of an aggregate transporting device 1 including an aggregate determination device 50. This aggregate transporting device 1 is a device that receives aggregate 9 as a raw material in a concrete plant and transports the aggregate 9 to a plurality of silos 40. As shown in Fig. 1, the aggregate transporting device 1 includes a receiving hopper 10, a receiving conveyor 20, a shuttle conveyor 30, four silos 40, and an aggregate determination device 50.

The receiving hopper 10 is positioned above the conveying start position of the receiving conveyor 20. The receiving hopper 10 is a funnel-shaped hopper whose inner diameter gradually decreases as it approaches the bottom. The aggregate 9 is transported from the quarry by truck 90 and dumped into the upper opening of the receiving hopper 10. The dumped aggregate 9 is temporarily stored in the receiving hopper 10 and is supplied to the upper surface of the receiving conveyor 20 from the lower opening of the receiving hopper 10.

The receiving conveyor 20 is a mechanism that transports the aggregate 9 that is put into the receiving hopper 10 diagonally upward. The receiving conveyor 20 has a start pulley 21, a terminal pulley 22, and a circular belt 23 that is stretched between the pulleys 21, 22. The start pulley 21 is located below the receiving hopper 10. The terminal pulley 22 is located above the shuttle conveyor 30, which will be described later. The terminal pulley 22 is positioned diagonally upward away from the start pulley 21.

At least one of the start pulley 21 and the end pulley 22 rotates by the power of a motor (not shown). This causes the belt 23 to rotate in the direction of the arrow in FIG. 1 between the start pulley 21 and the end pulley 22. The moving speed of the belt 23 is adjusted to any speed, for example, 50 m/min or more. The aggregate 9 fed into the receiving hopper 10 is supplied to the upper surface of the belt 23 near the start pulley 21. Then, as the belt 23 rotates, the aggregate 9 placed on the belt 23 moves diagonally upward from a position below the receiving hopper 10 toward the end pulley 22.

The belt 23 of the receiving conveyor 20 moves continuously at a constant speed. Therefore, the aggregate 9 drawn from the bottom of the receiving hopper 10 is transported while resting on the upper surface of the belt 23 at a substantially constant height. Then, when the aggregate 9 is transported to the position of the terminal pulley 22, the belt 23 reverses and the aggregate 9 falls onto the shuttle conveyor 30.

The shuttle conveyor 30 is a mechanism for distributing the aggregate 9 transported from the receiving conveyor 20 to the four silos 40. The shuttle conveyor 30 is located below the terminal pulley 22 of the receiving conveyor 20. The shuttle conveyor 30 has a first pulley 31, a second pulley 32, and a circular belt 33 stretched between the pulleys 31 and 32.

At least one of the first pulley 31 and the second pulley 32 rotates due to the power of a motor (not shown). This causes the belt 33 to rotate between the first pulley 31 and the second pulley 32. The aggregate 9 that falls from the receiving conveyor 20 and is placed on the belt 33 of the shuttle conveyor 30 is transported to one of the four silos 40 by the rotation of the belt 33.

The shuttle conveyor 30 can be moved along a horizontal plane by a drive mechanism (not shown). This allows the destination of the aggregate 9 by the shuttle conveyor 30 to be switched to a specified one of the four silos 40.

The four silos 40 are storage tanks for aggregates 9. The aggregates 9 are classified into a number of aggregate types according to characteristics such as particle size, color, and place of origin. In this aggregate transport device 1, the silo 40 in which the aggregates should be stored is determined for each aggregate type. In the example of FIG. 1, aggregates 9 of four aggregate types A to D are sorted and stored in the four silos 40. Note that the number of silos 40 provided in the aggregate transport device 1 is not limited to four.

<1-2. About the aggregate judgment device>
The aggregate determination device 50 is a device for determining whether the aggregate 9 fed onto the receiving conveyor 20 is suitable. The aggregate determination device 50 photographs the aggregate 9 transported by the receiving conveyor 20 and estimates the aggregate type based on the obtained image. The aggregate determination device 50 then determines whether the estimated aggregate type matches a specified aggregate type. As shown in Fig. 1, the aggregate determination device 50 includes an operation panel 51, a display unit 52, a camera 53, a light source 54, a hood 55, and an information processing unit 56.

The operation panel 51 is disposed near the receiving hopper 10. The operation panel 51 is electrically connected to the information processing unit 56. The operation panel 51 has a number of keys 511, 512. The operator (the driver of the truck 90 or a worker at the concrete plant) presses the key 511 indicating the type of aggregate 9 to be put into the receiving hopper 10. This causes information on the aggregate type to be input from the operation panel 51 to the information processing unit 56. In addition, when the information on the aggregate type is input, the shuttle conveyor 30 changes its position so that the silo 40 for that aggregate type becomes the destination.

After the aggregate 9 is loaded into the receiving hopper 10, the operator presses the key 512 to command the start of transport. This starts the operation of the receiving conveyor 20 and the shuttle conveyor 30. As a result, the loaded aggregate 9 is transported by the receiving conveyor 20 and the shuttle conveyor 30 to the specified silo 40.

The operation panel 51 may be configured with a general keyboard, mouse, or touch panel.

The display unit 52 is disposed near the receiving hopper 10 together with the operation panel 51. The display unit 52 is electrically connected to the information processing unit 56. For example, a liquid crystal display is used for the display unit 52. The display unit 52 displays various information related to the aggregate determination device 50. For example, the display unit 52 displays the determination results output from a determination unit 63 (described later) of the information processing unit 56. In other words, the display unit 52 is an example of a "result output unit" in the present invention.

The camera 53 is an imaging device that captures images of the aggregates 9 transported by the receiving conveyor 20. The camera 53 is positioned downstream of the receiving hopper 10 on the transport path so as to face the upper surface of the belt 23 of the receiving conveyor 20. In this embodiment, the camera 53 captures video of the aggregates 9 transported by the receiving conveyor 20. After the receiving conveyor 20 starts operating, a predetermined time has elapsed, and when the leading aggregate 9 passes below the camera 53, the camera 53 begins capturing video. The camera 53 then inputs the obtained video to the information processing unit 56.

In addition, a sensor that detects aggregate 9 may be provided midway along the receiving conveyor 20, and video recording by the camera 53 may be started based on the detection signal of the sensor.

The light source 54 is a part for emitting light toward the shooting range of the camera 53. For example, an LED lamp is used as the light source 54. However, the light source 54 may also be a fluorescent lamp or an incandescent lamp. The light source 54 is always turned on when shooting with the camera 53.

The hood 55 is a box-shaped member that covers a portion of the upper surface of the receiving conveyor 20. The hood 55 is formed from a material that has light-blocking properties. The hood 55 is disposed midway along the transport path of the aggregate 9 by the receiving conveyor 20. The hood 55 has an entrance opening 551 and an exit opening 552. The aggregate 9 transported by the receiving conveyor 20 is transported into the hood 55 through the entrance opening 551 and is transported out of the hood 55 through the exit opening 552.

The camera 53 and light source 54 are disposed inside the hood 55. External ambient light is blocked by the hood 55. This suppresses the effects of ambient light when the camera 53 takes an image. Furthermore, by irradiating light from the light source 54 inside the hood 55, the aggregate 9 can be illuminated at a constant brightness. Therefore, even if the amount of sunlight changes depending on the season or weather, the aggregate 9 can be photographed at a constant brightness. It is also possible to suppress changes in the shadow of the aggregate 9 due to the position of the sun.

The information processing unit 56 is a device that performs the learning process and the judgment process described below. The information processing unit 56 is composed of a computer having a processor 561 such as a CPU or a GPU, a memory 562 such as a RAM, and a storage unit 563 such as a hard disk drive or an SSD. A computer program for executing the learning process and the judgment process described below is installed in the storage unit 563. The information processing unit 56 is also connected to the operation panel 51, the display unit 52, the camera 53, the receiving conveyor 20, and the shuttle conveyor 30 described above so as to be able to communicate with them.

Figure 2 is a block diagram conceptually illustrating the functions realized in the information processing unit 56. As shown in Figure 2, the information processing unit 56 has a learning unit 61, an estimation unit 62, a determination unit 63, and a control unit 64. The functions of each of these units are realized by the computer serving as the information processing unit 56 operating in accordance with a computer program stored in the memory unit 563.

The learning unit 61 is a processing unit that generates a learning model M for estimating the aggregate type based on an image of the aggregate 9. The learning unit 61 performs machine learning using images of sample aggregate 9 prepared for learning (hereinafter referred to as "learning images") and the known aggregate type of the sample aggregate 9 as training data. A convolutional neural network, which is a type of deep learning, is suitable as a machine learning algorithm. In an experiment conducted by the present inventors, when a convolutional neural network with a pooling layer that performs a blurring process on the image was used in the learning unit 61, a learning model M with sufficient estimation accuracy could be generated even for an image that was blurred in the transport direction.

However, the machine learning algorithm used by the learning unit 61 is not limited to a convolutional neural network, and may be other supervised machine learning algorithms such as a neural network, linear regression, decision tree, or support vector regression.

The learning unit 61 learns the relationship between the images of a large number of sample aggregates 9 and the aggregate types of the sample aggregates 9. This generates a learning model M that can output an estimation result of the aggregate type corresponding to the image of the aggregate 9. The learning unit 61 provides the generated learning model M to the estimation unit 62. Details of the learning process will be described later.

The estimation unit 62 is a processing unit that estimates the aggregate type from the image of the aggregate 9 using the above-mentioned learning model M. The estimation unit 62 cuts out an image from the video D captured by the camera 53 and inputs it to the learning model M. The learning model M then outputs an estimation result E of the aggregate type based on the input image of the aggregate 9. The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

The determination unit 63 acquires aggregate type information I input to the operation panel 51. The determination unit 63 also acquires the aggregate type estimation result E by the estimation unit 62. The determination unit 63 then judges whether or not the aggregate type information I input to the operation panel 51 matches the aggregate type estimation result E by the estimation unit 62. The determination unit 63 also outputs the judgment result R to the display unit 52. As a result, the display unit 52 displays the judgment result R indicating whether or not the aggregate type of the aggregate 9 transported by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51.

The control unit 64 controls the operation of the receiving conveyor 20 according to the judgment result R of the judgment unit 63. The control unit 64 continues the operation of the receiving conveyor 20 when the judgment unit 63 judges that the aggregate type information I input to the operation panel 51 matches the estimation result E of the estimation unit 62. The control unit 64 stops the operation of the receiving conveyor 20 when the judgment unit 63 judges that the aggregate type information I input to the operation panel 51 does not match the estimation result E of the estimation unit 62.

<1-3. Learning process>
Next, a learning process executed in the aggregate determination device 50 will be described. Fig. 3 is a flow chart showing the flow of the learning process. This learning process is executed in advance before the determination process of the aggregate 9, which will be described later, is executed.

When performing the learning process, first prepare sample aggregate 9 for learning (step S11). For example, if you want to learn about four aggregate types A to D, prepare sample aggregate 9 for those four aggregate types.

The operator places the prepared sample aggregates 9 one type at a time onto the receiving conveyor 20 and operates the receiving conveyor 20. Then, while shining light from the light source 54, the camera 53 takes a video D of the sample aggregates 9 (step S12). When the operator has completed taking a video of the sample aggregates 9 of one aggregate type, the operator places the sample aggregates 9 of the next aggregate type onto the receiving conveyor 20. Then, the receiving conveyor 20 is operated to take a video D of the sample aggregates 9 in the same manner as above. In this manner, the videos D of the sample aggregates 9 of all prepared aggregate types are taken in sequence.

The video D captured by the camera 53 is composed of many frame images. For example, when a camera 53 with a frame rate of 30 frames/second captures images for 5 minutes, the video D obtained by the capture is composed of 9,000 frame images. In step S12, such videos D (a collection of frame images) are captured for all prepared aggregate types. The captured video D is then transmitted from the camera 53 to the information processing unit 56.

Inside the information processing unit 56, many frame images included in the video D are stored in the memory unit 563 as learning images (step S13). As described above, the video D includes many frame images. Therefore, the learning unit 61 can prepare many learning images. Figure 4 shows an example of a frame image included in the video D. As shown in Figure 4, due to the relationship between the frame rate of the camera 53 and the moving speed of the belt 23, the frame image is an image that has a blur in the conveying direction.

The learning unit 61 classifies the many learning images into teacher images and validation images (step S14). For example, if 36,000 learning images are obtained by photographing four types of sample bone material 9 for five minutes each, 3/4 of the learning images, or 27,000 learning images, are used as teacher images, and the remaining 1/4, or 9,000 learning images, are used as validation images. However, the ratio of teacher images to validation images does not necessarily have to be 3:1.

The learning unit 61 sets a teacher image and the known aggregate type of the sample aggregate 9 shown in the teacher image as teacher data. The learning unit 61 also sets a validation image and the known aggregate type of the sample aggregate 9 shown in the validation image as validation data.

The learning unit 61 performs image processing such as blurring, left-right inversion, and up-down inversion on the teacher image (step S15). By varying the teacher image through image processing, the range of images to be learned is expanded. This can improve the accuracy of machine learning. However, it is preferable that the image processing does not include image rotation processing. As described above, the teacher image is an image that has blur in the transport direction, and if rotation processing is performed, the direction of this blur will change, which will actually decrease the accuracy of machine learning. Note that images that have not been subjected to image processing may be included among the many teacher images.

The learning unit 61 performs machine learning using a large number of training data after image processing (step S16). That is, the machine learning algorithm inputs a large number of training images into the learning model M one by one. Then, the parameters of the learning model M are adjusted so that the estimated result of the aggregate type output from the learning model M approaches the aggregate type of the training image that was input. The machine learning algorithm repeats this parameter adjustment process for the number of training data prepared.

Then, the machine learning algorithm uses multiple validation images to check the accuracy of the aggregate type estimation by the learning model M (step S17). Specifically, the validation images are input to the learning model M, and it is checked whether the aggregate type estimation result output from the learning model M matches the aggregate type of the input validation image. Then, if the aggregate type estimation accuracy has reached a predetermined level (e.g., an accuracy rate of 95% or more), the machine learning algorithm ends the learning process. This generates one learning model M that can output an aggregate type estimation result based on an image of aggregate 9.

If the accuracy of the aggregate type estimation does not reach the desired level, the processes in steps S12 to S17 described above may be repeated. After generating the learning model M, an image of a sample aggregate 9 other than the teacher image and validation image described above may be input to the learning model M, and it may be further confirmed whether the aggregate type estimation result output from the learning model M matches the aggregate type of the sample aggregate 9 shown in the image.

Even if the aggregate 9 is the same type of aggregate, the surface color will differ depending on the dry state, wet state, the location of crushing, etc. For this reason, in the above-mentioned step S12, a video D of the sample aggregate 9 in a dry state, a video D of the sample aggregate 9 in a wet state, and a video D of the sample aggregate 9 in a different location of crushing may be obtained by shooting multiple times for each type of aggregate.

In addition, the multiple learning images input to the learning model M may include images of sample aggregate 9 in a dry state, images of sample aggregate 9 in a wet state, and images of sample aggregate 9 in different crushing locations. For example, when 9,000 learning images are used for one aggregate type, images of multiple sample aggregates 9 in different dry and wet states and crushing locations may be mixed among the 9,000 images. This allows the learning unit 61 to learn various states such as dry and wet states and different crushing locations for each aggregate type. Therefore, the estimation accuracy of the aggregate type output from the learning model M can be further improved.

<1-4. Aggregate determination process>
Next, a process for determining the suitability of the aggregate 9 inputted into the receiving conveyor 20 using the learning model M when the aggregate 9 is actually received will be described. Fig. 5 is a flowchart showing the flow of the determination process.

When receiving aggregate 9, first, the aggregate 9 loaded on the truck 90 is dumped into the receiving hopper 10 (step S21). Next, the operator inputs information I on the aggregate type of the dumped aggregate 9 into the operation panel 51 (step S22). For example, the operator inputs the aggregate type information I by pressing key 511 on the operation panel 51. After that, the operator presses key 512 on the operation panel 51 to instruct the start of transport. This starts the operation of the receiving conveyor 20 (step S23).

The aggregate 9 supplied from the receiving hopper 10 to the upper surface of the belt 23 is transported diagonally upward toward the terminal pulley 22 as the belt 23 rotates. Then, after a predetermined time has elapsed since the receiving conveyor 20 started operating, the camera 53 takes a video D of the aggregate 9 as the aggregate 9 passes below the camera 53 (step S24). The camera 53 then transmits the obtained video D to the information processing unit 56.

Inside the information processing unit 56, the estimation unit 62 cuts out one image (frame image) from the video D (step S25). The estimation unit 62 then inputs the cut-out image to the learning model M. The learning model M then outputs an estimation result E of the aggregate type based on the input image of the aggregate 9 (step S26). The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

The determination unit 63 acquires the aggregate type information I input to the operation panel 51 in step S22 above. The determination unit 63 also acquires the estimation result E output from the learning model M in step S26 above. The determination unit 63 then determines whether the aggregate type information I input to the operation panel 51 matches the aggregate type estimation result E by the estimation unit 62 (step S27).

The judgment unit 63 outputs the judgment result R to the display unit 52. As a result, the judgment result R as to whether or not the aggregate type of the aggregate 9 transported by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed on the display unit 52 (step S28). The operator can recognize the judgment result R by checking the display unit 52.

The control unit 64 checks whether the judgment result R of the judgment unit 63 in step S27 above is "match" or "mismatch" (step S29). Then, if the judgment result R indicates that the aggregate type information I input to the operation panel 51 and the estimation result E of the estimation unit 62 match (step S29: Yes), the control unit 64 continues the operation of the receiving conveyor 20. On the other hand, if the judgment result R indicates that the aggregate type information I input to the operation panel 51 and the estimation result E of the estimation unit 62 do not match (step S29: No), the control unit 64 stops the operation of the receiving conveyor 20 (step S30).

If the judgment result R is "Mismatch," it is possible that an inappropriate type of aggregate 9 has been put into the receiving hopper 10, or that the aggregate type information I has been entered incorrectly into the operation panel 51. In either case, the aggregate 9 on the receiving hopper 10 may be transported to the wrong silo 40. In step S30 above, in such a case, the operation of the receiving conveyor 20 is stopped urgently. This stops the transport of the aggregate 9, preventing the aggregate 9 from being transported to the wrong silo 40.

As described above, the aggregate determination device 50 photographs the aggregate 9 being transported by the receiving conveyor 20, and estimates the aggregate type of the aggregate 9 being transported using the obtained image and a learning model M that has been generated in advance by machine learning. The aggregate determination device 50 then determines whether the aggregate type estimation result E matches the aggregate type information I input to the operation panel 51. This makes it possible to detect the possibility that the aggregate 9 may be transported to the wrong silo 40.

In particular, in the aggregate determination device 50 of this embodiment, the aggregate 9 is photographed without stopping the receiving conveyor 20. Therefore, the image input to the learning model M is not a high-definition image, but an image that has blur. For the camera 53 that photographs the aggregate 9, an easily available, inexpensive camera can be used, rather than a high-speed camera with a high frame rate that can obtain frame images without blur in the conveying direction, or a high-performance camera with a large number of pixels. This makes it possible to reduce the cost of the camera 53. Furthermore, by using an easily available, inexpensive camera 53, the replacement cost when the camera 53 breaks down can also be reduced. Therefore, an aggregate determination device 50 with excellent maintainability can be realized.

In addition, in the aggregate discrimination device 50 of this embodiment, a convolutional neural network that performs image blurring processing is used to generate the learning model M in the learning unit 61. For this reason, the image input to the learning model M does not necessarily need to be a high-definition image. In other words, even if an image has blurring in the conveying direction, a learning model M with high estimation accuracy can be obtained. Therefore, a learning model M with high estimation accuracy can be obtained using an inexpensive camera 53 instead of a high-performance camera. The camera 53 may be, for example, a web camera with a frame rate of about 30 frames/second. In addition, by suppressing the frame rate and the number of pixels, the memory capacity of the aggregate determination device 50 can be reduced. In other words, an aggregate determination device with sufficient determination accuracy can be realized while having an inexpensive equipment configuration.

When the frame rate of the camera 53 is 30 frames/sec, if the moving speed of the belt 23 of the receiving conveyor 20 is 50 m/min or more (moving distance per frame is 2.78 cm/frame), frame images with blur in the conveying direction can be obtained. If the moving speed of the belt 23 exceeds 120 m/min (moving distance per frame is 6.67 cm/frame), the frame images obtained by shooting will be blurred to such an extent that the outline of the aggregate cannot be distinguished. For this reason, it is advisable to adjust the relationship between the frame rate of the camera 53 and the moving speed of the belt 23 so that the moving distance per frame is in the range of 2.78 cm/frame or more and 6.67 cm/frame or less.

In addition, in the aggregate determination device 50 of this embodiment, within the hood 55, light is irradiated from the light source 54 while the camera 53 takes an image. This makes it possible to reduce the variation in brightness of the image input to the learning model M. Therefore, the aggregate type can be estimated with greater accuracy.

2. Second embodiment (moisture sensor)
Next, a second embodiment of the present invention will be described. Fig. 6 is a diagram showing the configuration of an aggregate transport device 1 according to the second embodiment.

This embodiment differs from the first embodiment in that the aggregate determination device 50 is equipped with a moisture sensor 57. The moisture sensor 57 is a sensor that measures the amount of moisture on the surface of the aggregate 9 transported by the receiving conveyor 20. The moisture sensor 57 is disposed above the receiving conveyor 20, upstream of the camera 53. The moisture sensor 57 is also disposed within the hood 55 together with the camera 53 and the light source 54.

For example, a near-infrared moisture meter is used as the moisture sensor 57. The near-infrared moisture meter measures the moisture content of the aggregate 9 by irradiating the aggregate 9 with near-infrared rays and measuring the absorbance at the surface of the aggregate 9 based on the amount of reflected light. However, the moisture sensor 57 may be a moisture meter of another type. The moisture sensor 57 is connected to the information processing unit 56 so as to be able to communicate with it. The measurement results of the moisture sensor 57 are transmitted to the information processing unit 56.

Figure 7 is a flowchart showing the flow of the learning process in the second embodiment. As shown in Figure 7, in this second embodiment, after preparing a sample aggregate 9, first, the same processes as steps S12 to S17 described above are performed on the dry sample aggregate 9 (steps S12A to S17A). This generates a first learning model M1 capable of outputting an aggregate type estimation result E based on an image of the dry aggregate 9. Next, the same processes as steps S12 to S17 described above are performed on the wet sample aggregate 9 (steps S12B to S17B). This generates a second learning model M2 capable of outputting an aggregate type estimation result E based on an image of the wet aggregate 9.

The term "dry state" refers to a state in which the moisture content of the surface of the aggregate 9 measured by the moisture sensor 57 is less than a predetermined value. The term "wet state" refers to a state in which the moisture content of the surface of the aggregate 9 measured by the moisture sensor 57 is equal to or greater than a predetermined value.

Figure 8 is a flowchart showing the flow of the determination process in the second embodiment. As shown in Figure 8, in this second embodiment, like the first embodiment, steps S21 to S25 are performed to acquire an image of the aggregate 9. However, just before the image of the aggregate 9 is captured by the camera 53, the moisture sensor 57 measures the moisture content on the surface of the aggregate 9 (step S24A).

Then, the estimation unit 62 judges whether the aggregate 9 transported by the receiving conveyor 20 is in a dry state or a wet state (step S31). If the moisture amount measured by the moisture sensor 57 is less than the above-mentioned predetermined value, the estimation unit 62 judges that the aggregate 9 is in a dry state (step S31: Yes). If the moisture amount measured by the moisture sensor 57 is equal to or greater than the above-mentioned predetermined value, the estimation unit 62 judges that the aggregate 9 is in a wet state (step S31: No).

When the aggregate 9 is dry, the estimation unit 62 selects the first learning model M1 (step S32). When the aggregate 9 is wet, the estimation unit 62 selects the second learning model M2 (step S33).

When the first learning model M1 is selected, the estimation unit 62 inputs an image of the aggregate 9 to the first learning model M1. The first learning model M1 then outputs an estimation result E of the aggregate type based on the input image (step S26). When the second learning model M2 is selected, the estimation unit 62 inputs an image of the aggregate 9 to the second learning model M2. The second learning model M2 then outputs an estimation result E of the aggregate type based on the input image (step S26).

Then, the aggregate determination device 50 performs steps S27 to S30 in the same manner as in the first embodiment.

As described above, in the second embodiment, the aggregate determination device 50 is equipped with a moisture sensor 57. When the aggregate 9 is dry, the first learning model M1, whose parameters are adjusted for the dry state, is used. When the aggregate 9 is wet, the second learning model M2, whose parameters are adjusted for the wet state, is used. In this way, by using different learning models M1 and M2 for the dry and wet states, the aggregate type can be estimated with greater accuracy.

3. Third embodiment (majority decision)
Next, a third embodiment of the present invention will be described. Fig. 9 is a flowchart showing the flow of the determination process in the third embodiment. As shown in Fig. 9, in the third embodiment, similarly to the first embodiment, the processes of steps S21 to S24 are performed to obtain a video D of the aggregate 9.

The estimation unit 62 cuts out an odd number of images (an odd number equal to or greater than three) from the video D (step S25). The estimation unit 62 then inputs each of the cut out odd images into the learning model M. The learning model M then outputs an odd number of estimation results E corresponding to each of the input odd number of images of aggregate 9 (step S26). The estimation unit 62 sends the odd number of estimation results E output from the learning model M to the determination unit 63.

Then, the judgment unit 63 compares whether or not each of the odd number of estimation results E matches the aggregate type information I input to the operation panel 51. The judgment unit 63 then makes a final judgment by majority vote of the comparison results (step S27). That is, if there are more "matches" than "mismatches" among the odd number of comparison results, the judgment unit 63 determines the final judgment result R to be "matches." Also, if there are more "mismatches" than "mismatches" among the odd number of comparison results, the judgment unit 63 determines the final judgment result R to be "mismatches."

Then, the aggregate determination device 50 performs steps S28 to S30 in the same manner as in the first embodiment.

As described above, in this third embodiment, an odd number of estimation results E are output from the learning model M based on an odd number of images of aggregate 9. Then, based on the odd number of estimation results E, it is determined by majority vote whether the aggregate 9 fed into the receiving conveyor 20 matches the aggregate type input to the operation panel 51. This makes it possible to obtain a more accurate determination result R.

The aggregate determination device 50 may have an odd number of cameras 53, i.e., three or more. An odd number of images may be obtained by cutting out one image from each of the videos D captured by the odd number of cameras 53.

<4. Fourth embodiment (additional learning)>
Next, a fourth embodiment of the present invention will be described. The fourth embodiment differs from the first embodiment in that additional machine learning is appropriately performed while performing the above-mentioned determination process using a learning model M once generated. Fig. 10 is a flowchart showing the flow of the process when performing additional machine learning.

In the example of FIG. 10, after the judgment result R is displayed on the display unit 52, the operator judges whether the displayed judgment result R is correct or not. Then, the operator inputs correct/incorrect information indicating whether the judgment result R is correct or not into the operation panel 51 (step S34).

For example, if the display unit 52 displays a judgment result R of "mismatch," the operator checks whether there is indeed a mismatch between the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 and the aggregate type information I input to the operation panel 51. If there is indeed a mismatch, the operator inputs to the operation panel 51 that the judgment result R by the judgment unit 63 is correct. If the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 and the aggregate type information I input to the operation panel 51 match, the operator inputs to the operation panel 51 that the judgment result R by the judgment unit 63 is incorrect.

The above-mentioned correct/incorrect information input to the operation panel 51 is sent to the information processing unit 56. The information processing unit 56 stores the set of the image cut out in step S25, the estimation result in step S26, and the correct/incorrect information input in step S34 in the memory unit 563 as teacher data for additional learning (step S35). The information processing unit 56 stores such teacher data each time aggregate 9 is received by the receiving conveyor 20.

In addition, in step S35, not only the images (frame images) cut out in step S25 but also a video D of a predetermined time (e.g., 5 minutes) including the images may be included in the teacher data for additional learning. In this way, many frame images included in the video D can be used as images for additional learning. Therefore, a large amount of teacher data can be accumulated in a short period of time.

Then, the learning unit 61 of the information processing unit 56 performs additional machine learning using the accumulated teacher data at a predetermined timing (step S36). The timing for performing additional machine learning may be, for example, the point in time when the rate at which the above-mentioned true/false information is "incorrect" becomes higher than a predetermined threshold. The learning unit 61 updates or recreates the learning model M using only the teacher data for additional learning, or the initial teacher data plus the teacher data for additional learning. This allows the estimation accuracy of the aggregate type by the learning model M to be maintained at a predetermined level.

In addition, when the moisture sensor 57 measures the moisture content on the surface of the aggregate 9 as in the second embodiment described above, the measured moisture content may be included in the training data for additional learning.

5. Modifications
Although the first to fourth embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments.

In the above embodiment, the learning unit 61 performs image processing such as blurring, left-right inversion, and up-down inversion on the frame images. However, the frame images may be used as learning images without performing image processing.

In the above embodiment, the camera 53 captures a video D of the aggregate 9 being transported by the receiving conveyor 20. However, the camera 53 may capture a still image of the aggregate 9 being transported by the receiving conveyor 20. Even when capturing a still image instead of a video D, an image including blur can be obtained by capturing the image without stopping the transport of the aggregate 9. In other words, the camera 53 is not limited to capturing a video D, and may capture a still image of the aggregate 9 being transported to obtain an image including blur as a learning image.

In the above embodiment, the receiving conveyor 20 is stopped when the judgment result R by the judgment unit 63 is "mismatch". However, for example, a cut gate may be provided at the bottom of the receiving hopper 10, and the cut gate may be closed when the judgment result R is "mismatch". A surge bin that can temporarily store the aggregate 9 may be provided on the transport path, and the aggregate 9 may be transported to the surge bin when the judgment result R is "mismatch". A gate may be provided somewhere on the transport path to the silo 40, and the gate may be closed when the judgment result R is "mismatch". In other words, the control unit 64 may use some method to restrict the transport of the aggregate 9 to the silo 40 when the judgment result R is "mismatch".

In addition, in the above embodiment, the receiving conveyor 20 is inclined. However, the receiving conveyor 20 does not have to be inclined. In other words, the receiving conveyor 20 may transport the aggregate 9 horizontally.

In the above embodiment, the computer serving as the information processing unit 56 is located within the concrete plant. However, some or all of the functions of the information processing unit 56 may be located outside the concrete plant. For example, the computer serving as the information processing unit 56 may be located in a remote location and be able to communicate with the operation panel 51, the display unit 52, and the camera 53 via the Internet.

In addition, in the above embodiment, the judgment result R of the judgment unit 63 is displayed on the display unit 52. However, when the judgment result R of "mismatch" is output from the judgment unit 63, a warning lamp may be turned on or a warning buzzer may be sounded. In other words, the "result output unit" in the present invention may be a warning lamp, a warning buzzer, or the like.

In the above embodiment, an example of determining the type of aggregate on the receiving conveyor 20 of a concrete plant has been described. However, the present invention can also be used in a quarry that ships and supplies aggregate 9 to a concrete plant. In other words, the aggregate determination device of the present invention may determine the type of aggregate on a conveyor installed in a quarry.

Furthermore, the elements appearing in the above embodiments and variations may be combined as appropriate to the extent that no contradictions arise.

The invention can be used to determine aggregate type in concrete plants or quarries.

REFERENCE SIGNS LIST 1 Aggregate transport device 9 Aggregate 10 Receiving hopper 20 Receiving conveyor 30 Shuttle conveyor 40 Silo 50 Aggregate determination device 51 Operation panel 52 Display unit 53 Camera 54 Light source 55 Hood 56 Information processing unit 57 Moisture sensor 61 Learning unit 62 Estimation unit 63 Determination unit 64 Control unit 90 Truck D Video E Estimation result I Aggregate type information M Learning model M1 First learning model M2 Second learning model R Determination result

Claims (11)

An aggregate judgment device that judges whether aggregate fed into a conveyor is suitable or not,
A camera for photographing the aggregate transported by the conveyor;
an operation panel into which information on aggregate types is input;
an information processing unit communicably connected to the camera and the operation panel;
Equipped with
The information processing unit includes:
an estimation unit that inputs an image of the aggregate captured by the camera into a learning model generated by machine learning, and outputs an estimation result of the aggregate type from the learning model;
a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimated result;
having
The camera captures video of the aggregate being transported by the conveyor;
the video includes a plurality of frame images having a blur in a conveying direction of the aggregate, the frame images being obtained by adjusting a moving speed of the conveyor so that a moving distance per frame is 2.78 cm or more and 6.67 cm or less;
The learning model is generated by performing machine learning using as training data a plurality of frame images having the blur contained in a video of a sample aggregate prepared for learning, in the aggregate determination device. The aggregate determination device according to claim 1 ,
The information processing unit includes:
The aggregate determination device further comprises a learning unit that generates the learning model by performing the machine learning. The aggregate determination device according to claim 2 ,
The learning unit performs image processing other than rotation on the frame image and then performs the machine learning. The aggregate determination device according to claim 2 or 3 ,
the operation panel is capable of inputting correct/incorrect information indicating whether the determination result by the determination unit is correct or not,
The learning unit updates the learning model by performing additional machine learning using the images of aggregate obtained from the camera, the estimation results, and the correct/incorrect information as training data. The aggregate determination device according to any one of claims 1 to 4 ,
The aggregate determination device further includes a light source that irradiates light toward the shooting range of the camera. The aggregate determination device according to claim 5 ,
a hood for covering a portion of an upper surface of the conveyor;
The aggregate determining device, wherein the camera and the light source are disposed within the hood. The aggregate determination device according to any one of claims 1 to 6 ,
The aggregate determination device further comprises a result output unit that outputs a determination result of the determination unit. An aggregate judgment device that judges whether aggregate fed into a conveyor is suitable or not,
A camera for photographing the aggregate transported by the conveyor;
an operation panel into which information on aggregate types is input;
an information processing unit communicably connected to the camera and the operation panel;
Equipped with
The information processing unit includes:
an estimation unit that inputs an image of the aggregate captured by the camera into a learning model generated by machine learning, and outputs an estimation result of the aggregate type from the learning model;
a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimated result;
having
A moisture sensor for measuring the moisture content on the surface of the aggregate transported by the conveyor.
Further equipped with
The information processing unit includes:
A first learning model generated by performing the machine learning on aggregate having a surface moisture content less than a predetermined value;
A second learning model generated by performing the machine learning on aggregates having a surface moisture content equal to or greater than the predetermined value;
having
The estimation unit is
When the moisture content measured by the moisture sensor is less than the predetermined value, an image of the aggregate captured by the camera is input to the first learning model, and an estimation result of the aggregate type is output from the first learning model;
An aggregate determination device that, when the moisture amount measured by the moisture sensor is greater than or equal to the specified value, inputs an image of the aggregate obtained from the camera into the second learning model and outputs an estimation result of the aggregate type from the second learning model. The aggregate determination device according to any one of claims 1 to 8 ,
the estimation unit inputs an odd number of images obtained from the camera to the learning model, and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model;
The determination unit compares the aggregate type input to the operation panel with the odd number of estimated results and makes a determination by majority vote of the comparison results. The aggregate determination device according to any one of claims 1 to 9 ,
The information processing unit includes:
An aggregate determination device further having a control unit that limits the transportation of aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimated result. An aggregate judgment method for judging suitability of aggregate fed into a conveyor, comprising the steps of:
a) inputting information on the type of aggregate of the aggregate fed into the conveyor into an operation panel;
b) taking an image of the aggregate being conveyed by the conveyor;
c) inputting the image of the aggregate obtained by the photographing in the step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model;
d) determining whether the aggregate type input in step a) matches the estimated result;
having
In the step b), a video of the aggregate being transported by the conveyor is taken,
the video includes a plurality of frame images having a blur in a conveying direction of the aggregate, the frame images being obtained by adjusting a moving speed of the conveyor so that a moving distance per frame is 2.78 cm or more and 6.67 cm or less;
The learning model is generated by performing machine learning using as training data a plurality of frame images having the blur contained in a video of a sample aggregate prepared for learning, in the aggregate determination method.

Images