JP7593632B2 - Judgment method and judgment system - Google Patents

Description

The present invention relates to a determination method and a determination system.

WO2017/158698 (Patent Document 1) discloses a monitoring device. This monitoring device is installed in a cowshed and monitors cows. This monitoring device calculates the amount of movement of the cows in the cowshed and judges whether the cows are showing signs of parturition based on this amount of movement (see Patent Document 1).

WO2017/158698

In the monitoring device disclosed in the above-mentioned Patent Document 1, the amount of movement of cows (one example of livestock) in the barn is used to determine signs of parturition. Incidentally, in addition to the amount of movement of the livestock, another parameter that correlates with signs of parturition in livestock is the number of times the livestock changes its posture (hereinafter also referred to as the "number of posture changes"). Postures of livestock include standing and sitting. In other words, a change in the posture of livestock is a change from standing to sitting, and a change from sitting to standing.

However, it is not necessarily easy to count the number of posture changes with high accuracy. When determining whether or not there is a sign of labor based on the number of posture changes, if the number of posture changes is counted with low accuracy, the accuracy of the determination regarding the sign of labor will also be low.

The present invention was made to solve these problems, and its purpose is to provide a method and system for determining signs of parturition that can be performed with greater accuracy when determining signs of parturition based on the number of posture changes in livestock.

A method for determining signs of parturition in livestock according to one aspect of the present invention is a method for determining signs of parturition in livestock. This method for determining signs of parturition in livestock includes the steps of acquiring video data related to livestock, inputting the video data into a trained model to output probability information indicating the probability that the livestock's posture is standing or sitting at a predetermined interval, counting the number of changes indicating the number of times the posture has changed from one of standing or sitting to the other, tabulating the number of changes in each of a plurality of intervals, and making a determination regarding signs of parturition in the livestock based on the results of the tabulation. After a first state, in which the probability that the posture is standing or sitting is equal to or greater than a predetermined probability, changes to a second state, in which the probability that the posture is the other of standing or sitting is equal to or greater than a predetermined probability, the number of changes is counted when the second state continues a predetermined number of times.

In this determination method, if the second state does not continue for a predetermined number of times after the first state changes to the second state, the number of changes is not counted. Therefore, even if the first state and the second state frequently switch during the process of changing the livestock's posture, the possibility of counting the number of changes more than necessary is suppressed. Therefore, according to this determination method, the number of changes in the livestock's posture can be counted with relatively high accuracy, and it is possible to more accurately determine the signs of parturition in livestock.

In the above determination method, the trained model may be constructed through machine learning using images containing livestock and posture information indicating the posture of the livestock contained in the images as training data.

In the above-mentioned determination method, the livestock may be housed in a farrowing pen, the video image data may be generated by a thermographic camera, and the thermographic camera may be installed in a position where the farrowing pen is within the imaging range of the thermographic camera.

In the above determination method, only one livestock may be housed in the farrowing pen.

A judgment system according to another aspect of the present invention judges the signs of parturition in livestock. This judgment system includes an acquisition unit, an output unit, a counting unit, a compilation unit, and a judgment unit. The acquisition unit acquires video data related to livestock. The output unit outputs probability information indicating the probability that the livestock's posture is standing or sitting at a predetermined interval by inputting the video data to the trained model. The counting unit counts the number of changes indicating the number of times the posture changes from one of standing or sitting to the other. The compilation unit compiles the number of changes in each of a plurality of intervals. The judgment unit judges the signs of parturition in livestock based on the compilation result. When a first state, in which the probability that the posture is standing or sitting is equal to or greater than a predetermined probability, changes to a second state, in which the probability that the posture is the other of standing or sitting is equal to or greater than a predetermined probability, and the second state continues a predetermined number of times, the counting unit counts the number of changes.

In this determination system, if the second state does not continue for a predetermined number of times after the first state has changed to the second state, the number of changes is not counted. Therefore, even if the first state and the second state frequently switch during the process of changing the livestock's posture, the possibility of counting the number of changes more than necessary is reduced. Therefore, with this determination system, the number of changes in the livestock's posture can be counted with relatively high accuracy, making it possible to more accurately determine the signs of parturition in livestock.

The present invention provides a method and system for determining signs of parturition in livestock with higher accuracy when determining signs of parturition based on the number of posture changes in the livestock.

FIG. 1 is a diagram illustrating a schematic configuration of a delivery sign determination system. FIG. 2 illustrates an example of a hardware configuration of a server. FIG. 2 is a diagram showing a configuration of software modules realized in the server. 11 is a diagram for explaining a method of counting the number of posture changes by a counting unit. FIG. FIG. 13 is a diagram showing a posture change frequency table managed in a parameter DB. FIG. 13 is a diagram illustrating an example of training data used when generating a trained model. 13 is a flowchart showing a procedure for counting the number of posture changes. 13 is a flowchart showing a procedure for determining the presence or absence of a labor symptom based on the number of posture changes.

The following describes in detail an embodiment of the present invention with reference to the drawings. Note that the same or corresponding parts in the drawings are given the same reference numerals and their description will not be repeated.

[1. Overview]
It is known that the probability of livestock birth accidents occurring can be significantly reduced by having a human present during livestock (e.g., cow) birth. In order for a human to be present during birth, a human needs to wait in the farrowing room before the livestock begins to give birth. If the accuracy of detecting signs of livestock birth is improved, the labor required to monitor birth can be reduced, for example, because a human only needs to wait in the farrowing room after the signs of birth are detected. In the birth sign determination system 10 according to the present embodiment, the accuracy of determining signs of livestock birth is improved.

Figure 1 is a diagram showing a schematic configuration of a labor sign determination system 10. As shown in Figure 1, the labor sign determination system 10 includes a thermography camera 100 and a server 200.

The thermographic camera 100 is installed in a farrowing pen 300 housing livestock (e.g., cows) 400. The thermographic camera 100 is configured to generate image data relating to the livestock 400. More specifically, the thermographic camera 100 generates image data showing the temperature distribution in the imaging range. The thermographic camera 100 generates moving image data showing the temperature distribution in the imaging range, for example, by continuously generating thermographic images. The thermographic camera 100 includes, for example, an infrared sensor. Because the image data is thermographic image data, the condition of the livestock 400 can be grasped based on the image data regardless of the surrounding brightness.

For example, only one livestock 400 is housed in the farrowing room 300. The thermographic camera 100 is installed in a position where the farrowing room 300 falls within the imaging range of the thermographic camera 100. The thermographic camera 100 may be installed, for example, near any of the four corners of the farrowing room 300. For example, the height position of the thermographic camera 100 in the farrowing room 300 may be a position where the livestock 400 falls within the imaging range of the thermographic camera 100. Also, for example, the orientation of the thermographic camera 100 may be such that the approximate center of the angle of view of the thermographic camera 100 is the center of the farrowing room 300.

The image data generated by the thermography camera 100 is transmitted to the server 200 via the network N1. In other words, the thermography camera 100 has a communication function. The server 200 determines whether the livestock 400 is showing signs of parturition based on the received image data.

The number of posture changes (posture change count) of the livestock 400 is a parameter that correlates with the signs of parturition of the livestock 400. In the server 200, a judgment regarding the signs of parturition of the livestock 400 is made based on the number of posture changes. However, it is not necessarily easy to count the number of posture changes with high accuracy based on the image data generated by the thermography camera 100. When making a judgment regarding the signs of parturition based on the number of posture changes, if the accuracy of the count of the number of posture changes is low, the accuracy of the judgment regarding the signs of parturition will be low. In the parturition signs judgment system 10 according to the present embodiment, the number of posture changes is counted with high accuracy, and as a result, the judgment regarding the signs of parturition of the livestock 400 is made with higher accuracy. Below, the configuration of the parturition signs judgment system 10 will be explained in order.

2. Configuration
<2-1. Hardware configuration>
Fig. 2 is a diagram showing an example of a hardware configuration of server 200. As shown in Fig. 2, server 200 includes a control unit 210, a storage unit 220, a communication module 230, and an input/output I/F 240. The components included in server 200 are electrically connected to each other via a bus.

The control unit 210 includes a CPU (Central Processing Unit) 211, a RAM (Random Access Memory) 212, and a ROM (Read Only Memory) 213, and is configured to control each component in accordance with information processing.

The storage unit 220 is, for example, a memory such as a flash memory. The storage unit 220 is configured to store, for example, a control program 221. The control program 221 is a control program of the server 200 that is executed by the control unit 210. When the control unit 210 executes the control program 221, the control program 221 is deployed in the RAM 212. The control unit 210 then controls each component by interpreting and executing the control program 221 deployed in the RAM 212 by the CPU 211.

The communication module 230 is configured to communicate with an external device, for example, with the thermography camera 100. The communication module 230 is configured, for example, with a wireless LAN (Local Area Network) module, a wired LAN module, or a communication module that complies with other communication standards such as LTE (Long Term Evolution).

The input/output I/F 240 is an input/output device in the server 200. The input/output I/F 240 is composed of, for example, a keyboard, a mouse, and a display.

<2-2. Software configuration>
Fig. 3 is a diagram showing a configuration of software modules realized in the server 200. As shown in Fig. 3, in the server 200, the control unit 210 executes the control program 221 to realize an acquisition unit 250, an output unit 252, a trained model 260, a counting unit 254, a counting unit 256, and a determination unit 258.

The acquisition unit 250 is configured to acquire video data generated by the thermography camera 100. The video data shows a video that includes livestock 400.

The output unit 252 is configured to input video data to the trained model 260, and output probability information indicating the probability that the posture of the livestock 400 is standing or sitting at a predetermined interval. The predetermined interval is a time shorter than the time required from the start of the change in posture of the livestock 400 to the end of the change, for example, 1 second. Note that in this specification, "sitting position" includes at least one of the general "sitting position" (posture with hind legs bent and lowered, and upper body raised), "prone" (posture with chest raised), and "lateral lying" (posture with legs thrown out and lying on side).

The trained model 260 is configured to use the thermographic image data acquired by the acquisition unit 250 as input data and to output to the output unit 252 probability information indicating the probability that the posture of the livestock 400 contained in the thermographic image is standing or sitting. For example, information such as "the probability of being standing is 90%", "the probability of being standing is 60%", "the probability of being sitting is 80%", or "the probability of being sitting is 70%" is output as probability information. The trained model 260 is generated, for example, by performing learning (machine learning) using training data described below. Note that various well-known methods such as deep learning, neural networks, decision tree learning, association rule learning, and Bayesian networks can be applied as machine learning.

The counting unit 254 is configured to count the number of posture changes indicating the number of times the posture of the livestock 400 changes from one of the standing and sitting positions to the other. The counting unit 254 counts the number of posture changes based on the probability information output by the output unit 252.

When the posture of the livestock 400 is changing, the probability information output by the output unit 252 changes drastically. That is, when the posture of the livestock 400 is changing, for example, a state in which the probability that the posture of the livestock 400 is standing exceeds a threshold value Th1 (hereinafter also referred to as the "standing position dominant state") and a state in which the probability that the posture of the livestock 400 is sitting exceeds a threshold value Th2 (hereinafter also referred to as the "sitting position dominant state") frequently switch. If the number of posture changes is counted each time the standing position dominant state switches to the sitting position dominant state, the number of posture changes counted will be greater than the actual number of posture changes. As a result, the accuracy of counting the number of posture changes decreases.

The counting unit 254 therefore considers the posture of the livestock 400 to be standing when, for example, it has been determined that the probability that the posture of the livestock 400 is standing is equal to or greater than the threshold value Th1 for a predetermined number of consecutive times. Also, the counting unit 254 considers the posture of the livestock 400 to be sitting when, for example, it has been determined that the probability that the posture of the livestock 400 is sitting is equal to or greater than the threshold value Th2 for a predetermined number of consecutive times.

That is, for example, the counting unit 254 counts the number of posture changes when, after a state in which the posture of the livestock 400 is considered to be standing changes to a state in which the probability of the posture of the livestock 400 being sitting is equal to or greater than the threshold value Th2, the determination that the probability of the posture of the livestock 400 being sitting is equal to or greater than the threshold value Th2 continues for a predetermined number of times. Also, the counting unit 254 counts the number of posture changes when, for example, a state in which the posture of the livestock 400 is considered to be sitting changes to a state in which the probability of the posture of the livestock 400 being standing is equal to or greater than the threshold value Th1, the determination that the probability of the posture of the livestock 400 being standing is equal to or greater than the threshold value Th1 continues for a predetermined number of times.

Figure 4 is a diagram for explaining a method for counting the number of posture changes by the counting unit 254. With reference to Figure 4, the horizontal axis indicates time, the vertical axis extending upward indicates the standing probability, and the vertical axis extending downward indicates the sitting probability. The broken line L1 indicates the progress of the probability information output by the output unit 252.

At time t0-t1, the probability information is equal to or greater than the threshold value Th1 of the probability of being in a standing position. For example, the predetermined number of judgments is performed in a time shorter than the time between time t0-t1. Therefore, at time t1, the posture of the livestock 400 is considered to be in a standing position. At time t1-t2, the probability information changes significantly. However, at time t2-t3, the probability information is not equal to or greater than the threshold value Th2 of the probability of being in a sitting position. Therefore, at time t2-t3, the posture of the livestock 400 remains considered to be in a standing position.

Between times t3 and t4, the probability information is equal to or greater than the threshold value Th2 for the sitting probability. For example, the predetermined number of determinations are made in a time that is shorter than the time between times t3 and t4. When the determination that the probability that the livestock 400 is in a sitting position is equal to or greater than the threshold value Th2 continues for a predetermined number of times, the livestock 400 is considered to be in a sitting position, and the number of posture changes is counted as one.

Between time t4 and t5, the probability information changes significantly. Between time t5 and t6, the probability information is equal to or greater than the standing probability threshold Th1. However, a predetermined number of determinations are not made between time t5 and t6. Therefore, the number of posture changes of livestock 400 is not counted between time t5 and t6. The posture of livestock 400 remains sitting at time t6. The same is true for times t5 and t6 for times t7 and t8, and the posture of livestock 400 remains sitting at time t8. At time t9, the probability information again becomes equal to or greater than the sitting probability threshold Th2. In this example, after the posture of livestock 400 is deemed to be sitting between time t3 and t4, the posture of livestock 400 remains deemed to be sitting.

In this way, in the server 200, for example, after a state in which the posture of the livestock 400 is considered to be standing changes to a state in which the probability of the posture of the livestock 400 being sitting is equal to or greater than the threshold value Th2 (sitting predominant state), the number of posture changes is not counted unless a predetermined number of times are continuously determined that the probability of the posture of the livestock 400 being sitting is equal to or greater than the threshold value Th2. Also, in the server 200, for example, after a state in which the posture of the livestock 400 is considered to be sitting changes to a state in which the probability of the posture of the livestock 400 being standing is equal to or greater than the threshold value Th1 (standing predominant state), the number of posture changes is not counted unless a predetermined number of times are continuously determined that the probability of the posture of the livestock 400 being standing is equal to or greater than the threshold value Th1.

Therefore, even if the livestock 400 frequently switches between the standing dominant state and the sitting dominant state during the process of changing its posture, the possibility of counting the number of posture changes more than necessary is suppressed. Therefore, according to the server 200, the number of posture changes of the livestock 400 can be counted with relatively high accuracy, and therefore it is possible to more accurately determine whether the livestock 400 is showing signs of parturition.

Referring again to FIG. 3, the tallying unit 256 is configured to tally up the number of posture changes in each of a plurality of sections. That is, the tallying unit 256 tally up the number of posture changes in each fixed period. For example, the tallying unit 256 tally up the number of posture changes in each predetermined time period (1 hour). The tallying unit 256 registers the tallying results in the parameter DB 270.

Figure 5 shows a posture change count table 271 managed in the parameter DB 270. In the posture change count table 271, the posture change counts tallied by the tallying unit 256 are managed in association with time periods.

Referring again to FIG. 3, the determination unit 258 makes a determination regarding the signs of parturition of the livestock 400 based on the information managed in the posture change frequency table 271. The determination unit 258 determines, for example, whether or not the livestock 400 has signs of parturition. For example, if it is determined that the probability that the livestock 400 will give birth within a predetermined time (e.g., 10 hours) is equal to or greater than a predetermined probability (e.g., 70%), it is determined that there are signs of parturition. On the other hand, if it is determined that the probability that the livestock 400 will give birth within a predetermined time is less than the predetermined probability, it is determined that there are no signs of parturition.

[3. Trained Model]
As described above, the trained model 260 is configured to determine the posture (standing or sitting) of the livestock 400 included in the thermography image. The trained model 260 is generated, for example, by performing machine learning using the following training data.

Figure 6 is a diagram showing an example of training data used when generating a trained model 260 for posture determination. As shown in Figure 6, training data 500 includes multiple (many) thermographic images 510 including livestock 400 in an upright position, and multiple (many) thermographic images 520 including livestock 400 in a sitting position. A label indicating "standing position" is associated with thermographic image 510, and a label indicating "sitting position" is associated with thermographic image 520.

The trained model 260 is generated by performing machine learning using such training data. As described above, various well-known methods such as deep learning, neural networks, decision tree learning, association rule learning, and Bayesian networks can be applied as machine learning. The trained model 260 generated through machine learning outputs probability information indicating the probability that the posture of the livestock 400 is standing or sitting by accepting input of thermographic image data generated by the thermographic camera 100.

4. Operation
7 is a flowchart showing a procedure for counting the number of posture changes. The process shown in this flowchart is executed by the control unit 210.

Referring to FIG. 7, the control unit 210 acquires thermographic image data (step S100). The control unit 210 detects a livestock area included in the thermographic image (step S105). The control unit 210 determines whether or not a livestock area is included in the thermographic image (step S110). A known technique is used to determine the livestock area.

If it is determined that a livestock area is included (YES in step S110), the control unit 210 outputs probability information regarding the posture of the livestock 400 (step S115). The trained model 260 is used to generate the probability information regarding the posture of the livestock 400. On the other hand, if it is determined that a livestock area is not included (NO in step S110), the control unit 210 sets the previous value as the probability information regarding the posture of the livestock 400 (step S120).

The control unit 210 refers to the probability information and determines whether the probability that the posture is different from the posture considered to be the current posture is equal to or greater than a threshold (step S125). If it is determined that the probability that the posture is different from the posture considered to be the current posture is not equal to or greater than the threshold (NO in step S125), the process proceeds to step S140.

On the other hand, if it is determined that the probability that the posture is different from the posture considered to be the current posture is equal to or greater than the threshold (YES in step S125), the control unit 210 determines whether the determination that the probability that the posture is different from the posture considered to be the current posture is equal to or greater than the threshold has continued for a predetermined number of times (step S130).

If it is determined that such a determination has not continued for the predetermined number of times (NO in step S130), the process proceeds to step S140. On the other hand, if it is determined that such a determination has continued for the predetermined number of times (YES in step S130), the control unit 210 counts (counts up) the number of posture changes (step S135).

Then, the control unit 210 determines whether or not a predetermined time T1 has elapsed (since the previous predetermined time T1 has elapsed) (step S140). In the example of FIG. 5, the predetermined time T1 is, for example, one hour.

If it is determined that the predetermined time T1 has not elapsed (NO in step S140), the process returns to step S100. On the other hand, if it is determined that the predetermined time T1 has elapsed (YES in step S140), the control unit 210 associates the counted number of posture changes with the time period and registers them in the parameter DB 270 (step S145). Thereafter, the control unit 210 resets the value of the number of posture changes to the initial value (0) (step S150).

Figure 8 is a flowchart showing the procedure for determining whether or not there are signs of labor based on the number of posture changes. The process shown in this flowchart is executed by the control unit 210, for example, every time the parameter DB 270 is updated.

Referring to FIG. 8, the control unit 210 determines whether the latest number of posture changes registered in the parameter DB 270 is equal to or greater than a predetermined number (step S200). If it is determined that the latest number of posture changes is equal to or greater than the predetermined number (YES in step S200), the control unit 210 determines that the livestock 400 is showing signs of parturition (step S210). On the other hand, if it is determined that the latest number of posture changes is not equal to or greater than the predetermined number (NO in step S200), the control unit 210 determines that the livestock 400 is showing no signs of parturition (step S220).

[5. Features]
As described above, in the server 200 according to this embodiment, the number of posture changes is not counted unless, for example, the posture of the livestock 400 changes from a state considered to be an upright posture to a sitting dominant posture and the livestock 400 is continuously determined to be in the sitting dominant posture for a predetermined number of times after the state changes from a state considered to be an upright posture to a sitting dominant posture and the livestock 400 is continuously determined to be in the standing dominant posture for a predetermined number of times.

Therefore, even if the livestock 400 frequently switches between the standing dominant state and the sitting dominant state during the process of changing its posture, the possibility of counting the number of posture changes more than necessary is suppressed. Therefore, according to the server 200, the number of posture changes of the livestock 400 can be counted with relatively high accuracy, and therefore it is possible to more accurately determine whether the livestock 400 is showing signs of parturition.

[6. Modifications]
Although the embodiment has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention.

In the above embodiment, various processes such as counting the number of posture changes are executed by the server 200. However, the entity that executes the various processes does not necessarily have to be the server 200. For example, a PC (Personal Computer) may be connected to the thermography camera 100, and the various processes may be executed on the PC. Also, for example, the various processes may be executed on the thermography camera 100.

In addition, in the above embodiment, the presence or absence of birth signs in the livestock 400 is determined based on whether the number of posture changes is equal to or greater than a predetermined number. However, the method of determining the presence or absence of birth signs in the livestock 400 is not limited to this. For example, the presence or absence of birth signs in the livestock 400 may be determined by using a trained model that uses the number of posture changes as input data and outputs the presence or absence of birth signs.

In addition, in the above embodiment, a thermographic camera 100 is used as a means for acquiring images of the livestock 400. However, the means for acquiring images of the livestock 400 does not necessarily have to be a thermographic camera. The means for acquiring images of the livestock 400 may be, for example, a normal digital camera capable of shooting video.

10 Parturition sign determination system, 100 Thermography camera, 200 Server, 210 Control unit, 211 CPU, 212 RAM, 213 ROM, 220 Memory unit, 221 Control program, 230 Communication module, 240 Input/output I/F, 250 Acquisition unit, 252 Output unit, 254 Counting unit, 256 Tabulation unit, 258 Determination unit, 260 Trained model, 270 Parameter DB, 271 Posture change frequency table, 300 Parturition room, 400 Livestock, 500 Teacher data, 510, 520 Thermography image, L1 Broken line, N1 Network.

Claims (5)

A method for determining signs of parturition in livestock, comprising:
acquiring video data relating to the livestock;
A step of inputting the video data into a trained model to output probability information indicating a probability that the posture of the livestock is standing or sitting at a predetermined interval;
counting a number of changes indicating the number of times the posture is changed from one of a standing posture or a sitting posture to the other;
counting the number of changes in each of a plurality of intervals;
and making a determination regarding a birth sign of the livestock based on the result of the counting,
A determination method in which, after a first state in which the probability that the posture is either standing or sitting is greater than or equal to a predetermined probability changes to a second state in which the probability that the posture is the other of standing or sitting is greater than or equal to a predetermined probability, if the second state continues a predetermined number of times, the number of changes is counted. The determination method according to claim 1, wherein the trained model is constructed through machine learning using images including the livestock and posture information indicating the posture of the livestock contained in the images as training data. The livestock are housed in a farrowing pen;
the video data is generated by a thermography camera;
The method according to claim 1 or 2, wherein the thermographic camera is installed at a position where the birthing chamber is within an imaging range of the thermographic camera. The method of claim 3, wherein only one livestock is housed in the farrowing pen. A judgment system for judging a sign of parturition in livestock, comprising:
An acquisition unit that acquires video data related to the livestock;
an output unit that outputs probability information indicating a probability that the posture of the livestock is standing or sitting at a predetermined interval by inputting the video data to a trained model;
A counting unit that counts the number of times the posture is changed from one of a standing posture and a sitting posture to the other;
a counting unit that counts the number of changes in each of a plurality of intervals;
a determination unit that determines whether the livestock is experiencing a parturition symptom based on the result of the counting,
A determination system in which, after a first state in which the probability that the posture is either standing or sitting is greater than or equal to a predetermined probability changes to a second state in which the probability that the posture is the other of standing or sitting is greater than or equal to a predetermined probability, when the second state continues a predetermined number of times, the counting unit counts the number of changes.