JP7002066B2 - Information processing systems, information processing equipment, and programs - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Law Published at the 60th Ibaraki Prefectural Livestock Research Association Symposium held at the Ibaraki Prefectural Livestock Center (1234 Negoya, Ishioka City, Ibaraki Prefecture) on June 27, 2017.

The present invention relates to an information processing system, an information processing device, and a program.

Traditionally, livestock raised by livestock farmers are killed by a few percent due to inability to stand or weakness before shipment. Due to labor shortages, livestock farmers cannot keep an eye on all the livestock in the stables 24 hours a day, and even if there are changes in the behavior of the livestock, it is difficult to detect them at an early stage. For this reason, livestock may die due to suffocation while they cannot stand up in the stable, or they may become debilitated due to illness or the like and die. Inexperienced new farmers, unlike skilled workers, cannot even notice changes in livestock behavior, further increasing the rate of livestock mortality.

There is one that discloses a reporting system that reports abnormalities in the health status of livestock. This notification system attaches a sensor that detects the number of steps to a cow, receives a step count signal from each sensor, and detects an abnormality in the health condition from the number of steps. If there is an abnormality in the health information, the user of the reporting system is notified to that effect (see Patent Document 1). In addition, there is also one in which a sensor for measuring body temperature is embedded in the body of a cow and a temperature signal from the sensor is received to detect an abnormality in health condition (see Patent Document 2).

International Publication No. 2013/145329 Japanese Unexamined Patent Publication No. 2007-296042

However, the conventional mechanism for detecting an abnormality in livestock by attaching a sensor to the livestock puts a heavy burden on the livestock and may cause stress on the livestock. Therefore, there is a problem that the types and numbers of sensors to be attached are limited, and the information that can be detected from livestock is limited to those that detect a specific abnormality.

The present invention has been made in view of the above, and is an information processing system and information processing that eliminates the burden on livestock and the like to be monitored and enables reading more behavioral information from the monitored target. It is intended to provide equipment and programs.

In order to solve the above-mentioned problems and achieve the object, the information processing system according to the present invention has a spatial sensor that outputs spatial information in a monitoring area and a plurality of objects to be monitored from the spatial information output from the spatial sensor. A first detecting means for detecting the object information of an object and a second detecting means for detecting the arrangement information of moving objects in contact with each other among a plurality of moving objects in the object information detected by the first detecting means. And, the object information and the arrangement information are converted into an approximate figure that approximates the contour of the moving object, and multiple movements are made based on the approximate figure in chronological order corresponding to the spatial information output from the spatial sensor at a predetermined frame rate. A generation means for generating behavior information of an object, a determination means for determining whether the behavior of a moving object is an abnormal behavior from the behavior information, and a notification of an abnormality when the behavior of a moving object is determined to be an abnormal behavior. It is characterized by having a notification means.

The present invention has the effect of eliminating the burden on the monitoring target and making it possible to read more behavioral information from the monitoring target.

FIG. 1 is a diagram illustrating an embodiment of an information processing system according to the present invention. FIG. 2 is a diagram showing an example of a system configuration of an information processing system including a surveillance camera provided in a cow bunch. FIG. 3 is a diagram showing an example of the hardware configuration of the surveillance camera unit. FIG. 4 is a diagram showing an example of the hardware configuration of the analysis server device. FIG. 5 is a diagram showing an example of a table configuration of an information table related to a monitoring service. FIG. 6 is a diagram showing an example of setting information in which determination items for abnormal behavior are set for each training stage. FIG. 7 is an explanatory diagram of the function of the analysis server device. FIG. 8 is an explanatory diagram of a method of detecting the contour of a cow. FIG. 9 is a diagram showing an example of the auxiliary unit. FIG. 10 is an explanatory diagram of auxiliary processing by the search unit and the specific unit. FIG. 11 is an explanatory diagram when the outline of the cow is approximated to a figure. FIG. 12 is a diagram showing an example of an area change of a cow contour image. FIG. 13 is a diagram showing an example of the flow of the tracking process. FIG. 14 is a diagram showing an example of a flow of alarm basic data generation processing. FIG. 15 is a diagram showing an example of the flow of the threshold value determination process. FIG. 16 is a diagram showing an example of a flow of abnormal behavior determination processing of cattle.

Hereinafter, examples of the information processing system, the information processing apparatus, and the program according to the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. For example, in the examples shown below, livestock cattle, which are dairy cows and beef cattle, will be described as monitoring targets, but if it is a "moving object", it will be applied to other livestock, animals and organisms other than livestock, and the like. May be applied. Further, in the embodiment shown below, a spatial sensor that outputs a depth image indicating the depth of each subject in the monitoring area as "spatial information" is shown as an example, but the "spatial information" is not limited to the depth image and is not limited to the depth image. Any other object may be used as long as it can specify the area or contour of the moving subject in the object. Further, in the following embodiment, the contour information is shown as an example of "object information", but the "object information" may be a region of a moving object itself. Further, in the examples shown below, the individual contour information is shown as an example of "arrangement information", but the "arrangement information" may be other information as long as it is information that can identify the area of the individual in the object information.

FIG. 1 is a diagram illustrating an embodiment of an information processing system according to the present invention. FIG. 1 shows a cow bunch 90 provided in a barn and a surveillance camera (an example of a “spatial sensor”) 10a for monitoring the behavior of a cow 91 in the barn 90. The cow bunch 90 represents one cow bunch separated by a fence 90a. Cows 91 born at the same time are housed in the cow bunch 90. Although FIG. 1 shows two cows 91 as an example, the number of cows 91 may be one or more. Although not shown, the drinking fountain and the feeding ground may be inside the fence 90a or outside the fence 90a.

As shown in FIG. 1, a surveillance camera 10a is installed above the cow bunch 90, and the breeding area 90b of the cow bunch 90 is photographed by the surveillance camera 10a. The breeding area 90b is the monitoring area of this embodiment. The height H at which the surveillance camera 10a is installed is adjusted so that the breeding area 90b is within the shooting range. In this embodiment, the surveillance camera 10a is installed at a height of about 4.2 m with respect to the breeding area 90b having a width W × depth L of 5.0 m × 4.1 m.

The breeding area 90b is sensed by the surveillance camera 10a, and a frame image is output from the surveillance camera 10a at a predetermined frame rate. The behavior of the cow 91 is analyzed by the server device of the monitoring service provider based on the continuous frame image (“continuous frame image” described later). Then, when the result showing the abnormal behavior of the cow 91 is obtained in this data analysis, the server device notifies the livestock farmer of the cow 91 of the abnormal signal.

FIG. 2 is a diagram showing an example of a system configuration of an information processing system including a surveillance camera 10a provided in the cow bunch 90. As shown in FIG. 2, the information processing system 1 has a configuration in which a plurality of surveillance camera units 10, a cloud 20, an analysis server device 30, a management PC 40, a user terminal 50, and the like are connected via a network 60. Has.

The surveillance camera unit 10 is installed in the barn 90 of the barn of each livestock farmer who uses the surveillance service. Although only one cow bunch 90 is shown in FIG. 2, the surveillance camera unit 10 is individually installed in each cow bunch 90 in the barn. In addition, one surveillance camera unit 10 may be installed for some cow bunches 90.

The surveillance camera unit 10 is an image transmission unit having a surveillance camera 10a and a control PC 10b. The surveillance camera 10a is installed above each cow bunch 90 with the photographing direction facing downward, and photographs the breeding area 90b of each cow bunch 90. The control PC 10b of the surveillance camera 10a sends a continuous frame image output by the surveillance camera 10a to an access point by WiFi or the like, and causes the access point to transmit the continuous frame image to the cloud 20 through the network 60.

The cloud 20 constructs a storage network and provides a data storage service. The cloud 20 stores and stores the user's stored data (continuous frame image in this embodiment) in the storage 21 on the storage network.

The analysis server device 30 acquires a continuous frame image (copy) from the cloud 20 and performs an analysis process on the behavior of the cow 91 on the continuous frame image. When the analysis server device 30 detects the abnormal behavior of the cow 91 by the analysis process, the analysis server device 30 notifies the user terminal 50 of the livestock farmer of the cow 91 of a signal (abnormal signal 70) notifying the user terminal 50 of the abnormal behavior of the cow 91. The abnormality signal 70 is notified by an alarm, e-mail, or the like.

The management PC 40 is a personal computer provided by each livestock farmer in the management building 80 and the like. The management PC 40 sets the surveillance camera unit 10 and the like according to the corresponding setting program.

The user terminal 50 is a mobile phone, a smartphone, a tablet terminal, or the like owned by each livestock farmer. The user terminal 50 receives an abnormal signal 70 due to an alarm or the like from the analysis server device 30, and the cow 91 has an abnormal behavior due to a notification sound, light emission of an LED (Light Emitting Diode), a message screen display, or the like. Is notified to the owner of the user terminal 50. Further, the user terminal 50 may acquire an e-mail notifying the content of the abnormal behavior of the cow 91 as an abnormal signal 70.

Further, the user terminal 50 accesses a URL (Uniform Resource Locator) designated by a company that provides a monitoring service, and reproduces and displays a continuous frame image output from the surveillance camera unit 10 of the user's cow bunch 90. It is also possible to do. The continuous frame image to be reproduced is a color continuous frame image output from the color camera 11c (see FIG. 3) of the surveillance camera unit 10.

The network 60 is a wireless LAN (Local Area Network) network, a public network, the Internet, or the like. The access point to which the control PC 10b is connected to WiFi is configured in the wireless LAN network of the network 60.

FIG. 3 is a diagram showing an example of the hardware configuration of the surveillance camera unit 10. The surveillance camera 10a of the surveillance camera unit 10 includes an infrared projector 11a, an infrared camera 11b, a color camera 11c, and the like.

The infrared projector 11a irradiates the entire cow bunch 90 with infrared rays. The infrared camera 11b continuously outputs an image (referred to as a "depth image") showing the depth to each subject in the cow bunch 90 at a predetermined frame rate. Specifically, the infrared camera 11b calculates the depth in pixel units at a predetermined frame rate based on the infrared light reflected from each subject in the cow bunch 90, and the image in which each depth is associated with a predetermined pixel value. Is output continuously. For example, the distance to each subject is calculated as the depth based on the time from the irradiation of infrared light to the reception of the reflected light from each subject. In this embodiment, the continuous frame image indicating the depth is simply referred to as a "continuous frame image" to distinguish it from the color continuous frame image.

The color camera 11c continuously outputs a color image of the cow bunch 90 at a predetermined frame rate. Specifically, the color camera 11c converts each subject in the cow bunch 90 into a color image at a predetermined frame rate based on the light reflected by the illumination light or sunlight, and continuously outputs the color image. In this embodiment, it is clearly indicated that the color is "color".

The surveillance camera 10a is connected to the control PC 10b via a USB (Universal Serial Bus) cable 10c. The surveillance camera 10a outputs a continuous frame image and a color continuous frame image with the settings instructed by the control PC 10b.

The control PC 10b has a control unit 12 having a computer configuration such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). A USB controller 13, a WiFi communication controller 14, and the like are connected to the control unit 12.

In the control unit 12, the CPU executes the ROM program and performs the following processing, for example. The control unit 12 acquires a continuous frame image and a color continuous frame image from the surveillance camera 10a under the control of the USB controller 13, and obtains the acquired continuous frame image and the color continuous frame image in the cloud 20 under the control of the communication controller 14. Send to.

The USB controller 13 performs USB communication with a target (infrared camera 11b, color camera 11c, etc.) of the surveillance camera 10a via the USB cable 10c. The USB controller 13 mainly acquires a continuous frame image and a color continuous frame image from the surveillance camera 10a.

The communication controller 14 connects to the access point 60a via WiFi and transmits a continuous frame image and a color continuous frame image to the cloud 20 at any time.

FIG. 4 is a diagram showing an example of the hardware configuration of the analysis server device 30. In the analysis server device 30 shown in FIG. 4, a CPU 31, a ROM 32, a RAM 33, an HDD (Hard Disk Drive) 34, a communication board 35, and the like are connected via a bus 36.

The CPU 31 executes a program and performs arithmetic processing and control processing of each part. The ROM 32 is a non-volatile memory for storing a fixed program such as a BIOS. The RAM 33 is a volatile memory used by the CPU 31 as a work area.

The HDD 34 is an auxiliary storage device. The HDD 34 stores a basic program C1 such as an OS (Operating System), an analysis program C2 related to a monitoring service, an information table T, setting information D, and the like. The basic program C1 includes a Web (World Wide Web) server, a database server, and the like.

The communication board 35 is a communication board for performing communication such as Ethernet (registered trademark). The communication board 35 is connected to, for example, an in-house LAN of a service providing company via a communication cable.

Subsequently, the table configuration of the information table T related to the monitoring service will be described.
FIG. 5 is a diagram showing an example of the table configuration of the information table T related to the monitoring service. FIG. 5A is a user information registration table T1 for registering user information of a user who starts a monitoring service. FIG. 5B is a cow bunch information registration table T2 for registering information about the cow bunch 90 in which the surveillance camera 10a is installed. FIG. 5C is an individual information registration table T3 for registering individual information of the cow 91 in the cow bunch 90. FIG. 5D is a recording table T4 for sequentially recording monitoring data indicating the behavior of the cow 91. FIG. 5E is an alarm basic data setting table T5 for setting alarm basic data generated from monitoring data.

The user information registration table T1 shown in FIG. 5A has, as items, a user's "user ID (Identification)" t11, a user's "telephone number" t12, and a user's "email address". It has t13, a user's "URL for viewing an image" t14, and the like. Here, the "telephone number" t12 and the "email address" t13 are examples of information indicating a notification destination for notifying the user of the abnormality of the cow 91. The "URL for viewing an image" t14 is a URL for acquiring a color continuous frame image obtained by photographing the state of the user's cow bunch 90.

The cow bunch information registration table T2 shown in FIG. 5B has "user ID" t11, "cow bunch ID" t21 of the cow bunch 90 set as a monitoring target, and cow 91 for each cow bunch 90 as items. Has a "birth date" t22 and the like. Here, the surveillance camera unit ID (MAC address of the control PC 10b, etc.) of the surveillance camera unit 10 installed in the cow chamber 90 is registered in the "cow bunch ID" t21. In the "birth date" t22, the birth date of the cow 91 in the cow bunch 90 is registered. In this example, the case where the cow 91 born on the same day is housed in the same cow cell 90 is shown. When cows 91 having different birth dates are housed in the same cow bunch 90, the table configuration may be modified so that the birth dates can be managed for each individual cow 91 instead of the cow bunch 90 units.

The individual information registration table T3 shown in FIG. 5 (c) has "cow bunch ID" t21, "individual ID" t31 of the cow 91 in the cow bunch 90, and the like as items. Here, in the "individual ID" t31, the individual IDs of all the cows 91 housed in the cow bunch 90 are registered for each cow bunch 90. The individual IDs may be arbitrarily set as long as they can be distinguished from each other.

The recording table T4 shown in FIG. 5D has "individual ID" t31, "period" t41, "monitoring data" t42, "contact flag" t43, and the like as items. Here, in the "period" t41, the information of the set date and time when the data is set in the "monitoring data" t42 is set. In the "monitoring data" t42 and the "contact flag" t43, a monitoring data of the cow 91 and a contact flag (an example of "contact number of times information") indicating contact between the cows 91, which will be described later, are set, respectively.

The alarm basic data setting table T5 shown in FIG. 5 (e) has "individual ID" t31, "period" t51, "behavior information (alarm basic data)" t52, and the like as items. Here, in the "period" t51, a date including the setting date and time of the monitoring data and a period of morning, noon, and night are set. The "behavior information (alarm basic data)" t52 includes the "walking distance" t521 of the cow 91, the "standing time" t522, the "sitting time" t523, and the "number of contacts" t524 with other cows 91. , "Lying time" t525 and other setting items of alarm basic data for information representing the behavior of the cow 91. Here, the "alarm basic data" is basic data for determining the abnormal behavior of the cow 91 from the behavior of the cow 91. Each item of "behavior information (alarm basic data)" t52 is an example, and may be appropriately transformed into other items. Here, the sitting position refers to a posture in which the knees are lowered and the chest is raised. Lying refers to the posture of throwing out the limbs and lying down.

FIG. 6 is a diagram showing an example of setting information D in which determination items for abnormal behavior are set for each training stage. The training stage is an example of "stage information", and the setting information D is an example of "correspondence information". The setting information D shown in FIG. 6 is provided with the first stage d11, the second stage d12, the third stage d13, the fourth stage d14, the fifth stage d15, and the like as the training stage information d1. Here, as an example, the first stage d11 is "first milk", the second stage d12 is "nursing", the third stage d13 is "fat and nurturing", the fourth stage d14 is "estrus", and the fifth stage d15 is "estrus". "Pregnancy", ..., etc. are shown. Since each setting of the breeding stage is divided according to the number of days from the birth of the cow 91, it has information indicating the number of days and the period from the birth of the cow 91. For example, in "first milk", information indicating the period from birth to 7 days is set.

Further, the setting information D is associated with the training stage information d1 and the determination item information d2 regarding the abnormal behavior. As an example, the judgment items include "out of friends" d21, "estrus" d22, "signs of illness" d23, "unable to stand" d24, etc. 1 "is set up. Here, as an example, the first stage d11 and the second stage d12 are flagged as "out of friends" d21 and "sign of illness" d23, and the third stage d13 is "sign of illness" d23 and "unable to stand". A flag is set for d24, a flag is set for "estrus" d22 for the fourth stage d14, and a flag is set for "unable to stand" d24 for the fifth stage d15.

Items shown here as an example may be added or deleted as appropriate. Further, the items to be determined in each training stage (first stage d11, second stage d12, ...) may be changed as appropriate.

Further, the training stage of the setting information D may be changed as appropriate. For example, the breeding stage may be further subdivided. Further, the breeding stages may be divided according to the type of cow 91 (beef cattle, dairy cow), and individual breeding stages corresponding to each may be set.

7 to 11 are explanatory views of the functions of the analysis server device 30. The analysis server device 30 realizes the function by the CPU 31 appropriately reading and executing the basic program C1 and the analysis program C2 of the HDD 34 into the RAM 33. The function may be realized by allocating it to one CPU core or by distributing it to a plurality of CPU cores. Further, a plurality of analysis server devices 30 may be provided, and the functions may be distributed to each CPU for realization.

Hereinafter, various functions shown in FIG. 7 will be described with reference to FIGS. 8 to 11 as appropriate. It should be noted that between each function, notification of the completion of processing, etc. shall be given by interprocess communication or the like.

In FIG. 7, as the main functions related to the analysis process, the table operation unit 301, the reading unit 302, the detection unit 303, the auxiliary unit 304, the monitoring data generation unit 305, the association unit 306, and the setting unit are shown. 307, an alarm basic data generation unit 308, a threshold value determination unit 309, a determination unit 310, and a notification unit 311 are shown.

The table operation unit 301 registers, updates, deletes, extracts, and the like data for the information table T.

The reading unit 302 reads out the frame images Mn (n = 1, 2, ...) For one frame from the continuous frame image M0 of the ID for each surveillance camera unit ID in the order in which they were taken. For example, the continuous frame image M0 is sequentially transmitted from the cloud 20 (see FIG. 2) in the order in which it was taken, and is stored in the reception buffer of the communication board 35. The reading unit 302 reads out one frame of the frame image Mn for the continuous frame image M0 in the reception buffer.

The detection unit 303 detects the contour of the cow (contour information P1) from the read frame image Mn.

FIG. 8 is an explanatory diagram of a method of detecting the contour of a cow. FIG. 8A shows a color image for explaining the frame image Mn which is a depth image. FIG. 8B shows a frame image Mn which is a depth image. FIG. 8C shows the contour information P1 detected from the frame image Mn which is a depth image.

FIG. 8A shows a color image of two cows 91 of the same growing period at the moment when they are in the standing position shown in FIG. In the standing position, the depth becomes shallow because the distance between the cow 91 and the infrared camera 11b is short. In the image of FIG. 8A, the head of the cow 91 is cut off from the image because the head is put out in a feeding box outside the fence, but the head of the cow 91 is at a position lower than the body. Therefore, the depth is deeper than the fuselage. In addition, the ground has the longest distance from the infrared camera 11b and the deepest depth.

As shown in FIG. 8B, the frame image Mn of the depth image is an image showing a plurality of constant depth regions E1, E2, .... Regions E1, E2, ... Of a constant depth, respectively, indicate that each pixel has the same depth in the region. Areas E1, E2, ... Of a constant depth have different depths to the subject between adjacent areas, and have different depths from each other. The pixel value of each pixel configured in the frame image Mn corresponds to the depth representing the distance to the subject corresponding to the position of each pixel. Here, for the sake of explanation, each region E1, E2, ... Is represented by changing the diagonal line spacing according to the depth. The narrower the diagonal line spacing, the shallower the depth. Further, the region having a constant depth is divided into regions E1 to E8, but the number of these regions is an example, and the number of divisions differs depending on the decomposition ability of the depth.

When the cow 91 is in the sitting position or lying down, the depth of the cow 91 becomes deeper and the value of the depth becomes large. The ground is shown in white because it has the deepest depth. In addition, the fence 90a, the feeding area, the drinking area, etc. are represented by a fixed depth over a continuous frame.

The detection unit 303 covers a region having a constant depth that has changed, such as moving from a plurality of regions E1, E2, ... In the frame image Mn with respect to the immediately preceding frame image Mn-1. The area is identified and its contour is detected. When the adjacent constant depth areas change in conjunction with each other, the cow area includes the linked constant depth areas. In FIG. 8B, since the region E1 changes, the region E1 is specified as a cow region. Further, since the region E2 and the region E3 adjacent thereto change in conjunction with each other, the region E2 and the region E3 are specified as different cow regions.

In FIG. 8C, the two identified cow regions are shown by rectangular frames F1 and F2. The detection unit 303 detects the contours f1 and f2 of the cow as the contour information P1 from the region of the cow in the specified rectangular frames F1 and F2. The contours f1 and f2 of the cow are detected from the difference in depth information from other regions.
In addition, the area of cattle may be specified by another method. For example, in the fence 90a, there are feeding boxes, drinking fountains, etc. in addition to the cows to be monitored, but these are in fixed positions. Therefore, the area excluding the area indicated by the coordinates such as the food box and the drinking fountain in the frame image Mn is specified as the area of each cow.

The auxiliary unit 304 performs an auxiliary process for detecting the contour (individual contour information P2) of each cow when it is in contact with the contour (contour information P1) detected by the detection unit 303. The auxiliary process is a process that enables the contour of each cow 91 to be detected when the contour of the cow detected by the detection unit 303 is detected as one contour by the contact of a plurality of cows 91. By this processing, even if a plurality of cows 91 being tracked are detected as one contour by contact over each frame image Mn of the continuous frame image M0, it is possible to identify each cow 91 in the meantime.

FIG. 9 is a diagram showing an example of the auxiliary unit 304. As shown in FIG. 9, the auxiliary unit 304 has a search unit 304a and a specific unit 304b. The auxiliary processing by the search unit 304a and the specific unit 304b will be described with reference to FIG.

FIG. 10 is an explanatory diagram of auxiliary processing by the search unit 304a and the specific unit 304b. FIG. 10A shows the contour f3 of cows when two cows 91 are in contact with each other in the frame image Mn. FIG. 10B shows an example of auxiliary processing for the cow contour f3 of FIG. 10A.

The search unit 304a scans from one point (start point v0) on the contour f3 of the detected cow in one direction (for example, scanning direction V1) on the contour f3, and sequentially from the start point v0, an arbitrary number of points on the contour f3. Only the skipped points (search points v1, v2, ...) Are searched, and the line segment connecting the adjacent search points (skip contour lines g1, g2, ...) Is obtained. For example, the skip contour line g1 is obtained as a line segment connecting the start point v0 and the search point v1.

The specific portion 304b obtains a singular point in which the angle (intersection angle) formed by the adjacent skip contour lines (for example, the skip contour line g1 and the skip contour line g2) suddenly changes within a predetermined angle. For example, the specific unit 304b sequentially obtains the intersection angle of adjacent skip contour lines from the start point v0 toward the scanning direction V1, and within a predetermined angle from the comparison with the intersection angle obtained in the immediately preceding frame Mn-1. Determine if it is a point that changes suddenly. When the obtained point does not correspond to a point that suddenly changes within a predetermined angle, the intersection angle of the next adjacent skip contour lines in the scanning direction V1 is determined. In this example, the search point v5 is determined as a point that suddenly changes within a predetermined angle and is specified as one of the singular points. If the engine returns to the starting point v0 without any point corresponding to a sudden change within a predetermined angle, it is determined that the contour has no singular point, and the auxiliary process is terminated.

When one of the singular points is specified in the scanning direction, the processing for specifying the singular point in the scanning direction V1 is stopped, and the processing for specifying the singular point is started from the start point v0 in the direction opposite to the scanning direction V1. .. If another singular point is specified in the reverse direction, the singular point identification process in that direction is also stopped. In this example, the search point-v18 is determined as a point that changes abruptly within a predetermined angle and is specified as one of the other singular points.

The auxiliary part 304 is one by the contact of two cows 91 by the boundary line connecting a set of singular points (in this example, the search point v5 and the search point-v18) specified by the specific part 304b. The represented contour f3 is separated into contours f1 and f2 of the two cows 91, respectively, and distinguished. The individual contour information P2 is information in which the contour f3 included in the contour information P1 is updated to the contour f1 and the contour f2 separated by the auxiliary processing.

Here, as an example, an example is shown in which the specific unit 304b scans in the scanning direction V1 to specify a singular point, and then scans in the opposite direction to specify the singular point. You may find the singularity of the set. Further, depending on the contact method, not only one set but also a plurality of sets of singular points may be obtained.

Further, here, as an example, the case where two cows are in contact is taken as an example, but even in the case of contact with three or more cows, the boundary between each cow is obtained by specifying a singular point, and the contour is separated for each cow 91. do.

The monitoring data generation unit 305 generates monitoring data P3 for monitoring the behavior of each cow 91 from the contour of each cow 91 (individual contour information P2). In this embodiment, the monitoring data generation unit 305 generates an approximate figure (function data or the like) that approximates the contour of the cow to the figure as the monitoring data P3. Specifically, the monitoring data generation unit 305 performs a conversion that approximates each contour to the above figure when one region is divided into a plurality of regions at the boundary by the contour of each cow and the auxiliary unit 304.

FIG. 11 is an explanatory diagram when the outline of the cow is approximated to a figure. FIG. 11A shows the individual contour information P2 of the frame image Mn in which the contour of the cow is shown by the contour f3 (see FIG. 10A). After being separated into contours f1 and f2 by the auxiliary unit 304, the monitoring data generation unit 305 converts the respective contours f1 and f2 into an elliptical approximation by, for example, the least squares method. FIG. 11B shows an example of an approximate ellipse obtained by approximating the respective contours f1 and f2 to an ellipse by the least squares method. The approximate ellipses r1 and r2 are obtained as data of a unique elliptic function indicating the arrangement of the frame image Mn in the quadratic plane coordinate system.

When the contour of the cow 91 is separated by the auxiliary portion 304 as in this example, a contact flag A indicating contact is added to the data of each elliptic function. The contact flag A is provided with, for example, "1" or "0" in the first bit.

The association unit 306 associates the monitoring data P3 with the individual ID of the cow 91. Specifically, the mapping unit 306 uses the table operation unit 301 to set the monitoring camera unit ID of the user's cow bunch 90 and the individual ID of the cow 91 in the cow bunch 90 as an information table at the start of the monitoring service. The information acquired from T and associated with the surveillance camera unit ID and the individual ID is set as the tracking information P4. After that, when the number of contours indicating each cow matches the number of individual IDs, the association unit 306 sets the monitoring data P3 (including the contact flag A) corresponding to each cow to one of the individual IDs. Set to as the initial value. Then, the monitoring data P3 (including the contact flag A) of each cow 91 obtained in chronological order from the continuous frame image is set in the tracking information P4 at any time. Regarding the association of the monitoring data P3 after the initial value with respect to the individual ID, for example, priority is given to data having a small difference (difference in the center position, etc.) from the monitoring data P3 immediately before being associated with the individual ID. conduct. As a result, in the tracking information P4, the monitoring data P3 in the time series order of each individual ID is associated with each individual ID.

The setting unit 307 separates the contact flag A from the monitoring data P3, and the table operation unit 301 sets the monitoring data P3 and the contact flag A in the recording table T4. For example, the setting unit 307 uses the set number of the monitoring data P3 set in the tracking information P4 as a trigger, or at a predetermined time interval (for example, every few seconds), all except the latest monitoring data P3 and the contact flag A. The monitoring data P3 and the contact flag A are set in the recording table T4. In this example, the latest monitoring data P3 is left for use in determining which individual ID the next required monitoring data P3 is associated with.

The alarm basic data generation unit 308 generates action information as alarm basic data based on the monitoring data P3 of the recording table T4 (including the contact flag A when counting the number of contacts). For example, the alarm basic data generation unit 308 is set by the setting unit 307, or at predetermined time intervals (for example, every few seconds, minutes, every few hours, etc.) from the recording table T4 to each individual ID of each surveillance camera unit ID. The monitoring data P3 and the contact flag A of the above are extracted by the table operation unit 301, and the behavior information of the cow 91 is generated from the monitoring data P3 and the contact flag A for each individual ID of each surveillance camera unit ID. The behavior information includes, for example, the walking distance of the cow 91, the standing time, the sitting time, the lying time, the number of contacts with other cows 91, and the like. In addition, information indicating the behavior of the cow 91 may be set as appropriate.

In general, the cow 91 repeats a posture in which the knees are lowered and the chest is raised (sitting position) and a posture in which the limbs are thrown out and lying down (lying). During this repetitive motion, the cow 91 may be in an incapable of standing up, unable to return to the sitting position from lying down. The inability to stand may be caused by a fall accident. If the person cannot stand up and continues to lie down, it may cause an accident such as suffocation. Therefore, in the present embodiment, the lying time is set as the action information.

The behavior information of each cow is generated as follows, for example, in a predetermined period (in this example, morning, noon, and night). Regarding the standing time and sitting time of one cow 91, the standing or sitting state is determined from the monitoring data P3 of the cow 91 in chronological order within the period, and the standing or sitting state is obtained. Calculate the cumulative duration from. The standing position and the sitting position are determined by the area of the approximate figure which is the monitoring data P3. For example, a case larger than a predetermined area is a standing position, and a case smaller than a predetermined area is a sitting position.

Further, the number of contacts of the cow 91 is obtained from the contact flag A of the cow 91 in chronological order as one time from the contact to the separation.

Further, the walking distance of the cow 91 is obtained from the movement locus of the coordinates (center coordinates of the approximate ellipse, etc.) of the approximate figure of the cow 91.

As for the lying time of one cow 91, the lying state is determined from the monitoring data P3 of the cow 91 in the time series order within the period, and the cumulative duration after the state is obtained is calculated. The lying state is determined by the ratio of the vertical and horizontal lengths of the approximate figure which is the monitoring data P3. For example, in the case of an approximate ellipse, the lying state of the cow 91 is determined from the ratio of the respective lengths (major axis and minor axis) of the major axis and the minor axis. When the cow 91 is viewed from above, the ratio of the length to the minor axis changes greatly when the cow is in a lying state. From this change in the length ratio, the lying state of the cow 91 is judged.

Further, the lying state may be determined based on the contour area of the approximate figure which is the monitoring data P3. Here, FIG. 12 is a diagram showing an example of the area change of the contour image of the cow. In FIG. 12, the vertical axis represents the area (pixels) of the contour image of the cow, and the horizontal axis represents the time. As shown in FIG. 12, since the contour area in the lying state is larger than the contour area in the sitting state, the case where the area is larger than the predetermined area is defined as the lying position, and the case where the area is smaller than the predetermined area is defined as the sitting position.

As other information indicating the behavior of the cow 91, for example, regarding the number of times of drinking and the number of feedings, the coordinates of the head on the approximate figure of each cow 91 (one of the focal coordinates if it is an approximate ellipse) and the drinking fountain and feeding. Obtained from the distance to the coordinates of the field. When the distance is within the predetermined range and the distance is out of the predetermined range, it is calculated as one time.

In this way, the alarm basic data generation unit 308 sets the generated action information in the alarm basic data setting table T5.

The threshold value determination unit 309 determines a threshold value for regarding the behavior of the cow 91 as an abnormal behavior. In this example, one day is divided into a morning period, a daytime period, and a night period, and the latest threshold values (upper limit value and lower limit value) are determined for each period. Further, for each cow 91, a threshold value is determined for each item of behavior information. The threshold value is determined by performing a moving average based on the alarm basic data for a predetermined number of days (here, the latest few days) of the corresponding item. The determined threshold value is set in the threshold value information P5 for each individual ID of the cow 91 and for each period of morning, noon, and night.

The determination unit 310 determines the abnormal behavior of each cow 91 (dismemberment, estrus, signs of illness, inability to stand, etc.) from the behavior information of each cow 91 during the period included in the judgment such as the day of judgment.

Specifically, the determination unit 310 describes the stage corresponding to the setting of the "birth date" t22 of the cow 91 for each cow in the cow bunch 90 in the cow bunch information registration table T2 (see FIG. 5 (b)) from the setting information D. The determination item of the flag "1" is acquired, and the threshold value (upper limit value and lower limit value) of the period at the time of determination is acquired from the threshold value information P5. Then, the determination unit 310 searches the alarm basic data setting table T5 for an individual ID exceeding the threshold value for each individual ID of each cow bunch 90 and for each determination item acquired by the table operation unit 301. The individual ID exceeding the threshold value and the determination item thereof are extracted.

When the extraction result includes an individual ID exceeding the threshold value, the determination unit 310 determines that the cow 91 has abnormal behavior. The type of abnormal behavior is determined based on the determination items extracted together. For example, when the determination item is "number of contacts", the number of contacts with other cows 91 is higher than usual, so that it is determined to be an abnormal behavior indicating "estrus". When the determination item is "lying", it is determined that the cow 91 is in an "unable to stand" state because the lying time continues. The determination method is not limited to this, and may be appropriately changed. For example, when a plurality of determination items are extracted for one individual ID, the abnormal behavior may be determined by combining them.

When the determination unit 310 determines that the behavior is abnormal, the notification unit 311 acquires the address such as the email address of the corresponding livestock farmer set in the user information registration table T1, and performs an alarm, an e-mail, or the like. Notifies the destination of the abnormal signal.

Next, the processing in the analysis server device 30 will be described. The analysis server device 30 receives the user's registration information from the user's terminal, the administrator's terminal, or the like as a preparatory process for starting the monitoring service, and uses the registration information in the user information registration table T1 (FIG. 5A). (See), the cow bunch information registration table T2 (see FIG. 5 (b)), and the individual information registration table T3 (see FIG. 5 (c)). After the setting, the analysis server device 30 receives a signal indicating the start of the monitoring service from the user's terminal, the administrator's terminal, or the like, and analyzes the behavior of the cow 91 of each cow bunch 90 of the user. Start.

13 to 16 are explanatory views of an analysis process for analyzing the behavior of the cow 91 in the cow cell 90 of the continuous frame image M0 output from one surveillance camera unit 10 of the user. The analysis process includes tracking process, alarm basic data generation process, threshold value determination process, abnormal behavior determination process, and the like. Hereinafter, an example of each process will be described with reference to FIGS. 13 to 16.

FIG. 13 is a diagram showing an example of the flow of the tracking process. First, the reading unit 302 acquires the value (surveillance camera unit ID) of the "cow bunch ID" t21 of the user's cow bunch 90 that has accepted the start of the monitoring service from the cow bunch information registration table T2 by the table operation unit 301. (S1).

Subsequently, the reading unit 302 reads out one frame worth of frame images in the order in which the surveillance camera unit ID was taken from the transmission source continuous frame image M0 (S2).

Subsequently, the detection unit 303 determines whether the number of read frames is two or more (S3). When the number of read frames is one (S3: No determination), the process returns to step S2, and the reading unit 302 reads out the frame image in the order in which the images were taken.

When the number of frames is 2 or more (S3: Yes determination), the detection unit 303 subsequently detects the contour of the cow for the frame image (frame image M2) read out later (S4). Specifically, the detection unit 303 identifies a region in the frame image M2 that has changed from each region having a different depth as compared with the immediately preceding frame image (frame image M1) as a cow region, and the detection unit 303 identifies the region as the cow region. Detect contours.

Subsequently, the auxiliary unit 304 performs an auxiliary process for detecting the individual contour of the cow 91 for each detected contour (S5). Specifically, the auxiliary unit 304 obtains two singular points from the intersection angle of the skip contour lines provided on the contour, and when the two singular points are found, two heads are bounded by the line connecting the two points. The individual contours of the cow 91 are detected.

Subsequently, the monitoring data generation unit 305 generates monitoring data P3 indicating the behavior of each cow from the detected individual contours of each cow (S6). In this embodiment, the monitoring data P3 is generated as an approximate figure so that the behavior of each cow 91 can be monitored by the center position and aspect ratio of the approximate figure.

Subsequently, the mapping unit 306 acquires the value of the "individual ID" t31 associated with the value of the "cow bunch ID" t21 (surveillance camera unit ID) from the individual information registration table T3 by the table operation unit 301. Then, the tracking information P4 in which the acquired surveillance camera unit ID and the individual ID are associated with each other is set in the RAM 33 (S7).

Subsequently, the association unit 306 determines whether the number of individual IDs set in the tracking information P4 and the number of monitoring data match (S8). When the numbers do not match (S8: No determination), from step S2, the contours of the remaining cows 91 are detected for the frame images whose shooting order is next, and monitoring data is generated. Regarding the monitoring data of the cow 91 that has already been detected, the detection unit 303, the auxiliary unit 304, and the monitoring data generation unit 305 are targeted for each frame image in the meantime until the monitoring data of the remaining cow 91 is generated. Performs the same processing, and the mapping unit 306 updates and manages the latest monitoring data.

When the number of individual IDs and the number of monitoring data match (S8: Yes determination), the setting unit 307 monitors the monitoring data of each cow 91 for each individual ID for the tracking information P4 (in this case, the initial value). Is assigned and set (S9). In this case, the monitoring data of each cow 91 is arbitrarily assigned to each individual ID. In addition, an individual ID corresponding to each cow 91 may be assigned. When assigning an individual ID corresponding to each cow 91, for example, the individual ID of each cow is extracted from a color image by image analysis. As the individual ID, an individual identification number or the like provided on each cow 91 at a position that can be read by a surveillance camera and a size that can be analyzed by engraving or the like is used. Then, from the coordinates in the depth image indicated by the monitoring data of each cow 91 and the coordinates of each individual ID in the color image, the individual IDs having the same coordinates and the monitoring data are associated with each other. When the color image is not used, for example, the IC tag for each individual ID attached to each cow 91 is read by a reading device fixed at a predetermined position, and the distance and arrangement of each IC tag are determined based on the reception strength and the like. Etc. are specified, and the individual ID and the monitoring data are associated with each other based on the specified information. Alternatively, the individual ID may be arbitrarily assigned, and later, the user or the administrator may rewrite the correspondence of the individual ID. The method of assigning an individual ID corresponding to each cow 91 shown here is an example, and other methods may be used.

After the above initial value setting process, the various functions perform the following processes. First, one frame followed by the reading unit 302 is read (S10), and the detection unit 303 detects the contour of the cow for the frame image Mn (S11). The detection unit 303 detects the contour of the cow by comparing it with the frame image Mn-1 immediately before.

Subsequently, the auxiliary unit 304 performs an auxiliary process of detecting the individual contour of the cow 91 for the contour of each cow (S12). This process is the same as the process of step S5.

Subsequently, the monitoring data generation unit 305 generates monitoring data indicating the behavior of each cow 91 from the detected contour of each cow 91 (S13). This process is the same as the process of step S6.

Subsequently, the association unit 306 sets the monitoring data of each cow 91 in each individual ID of the tracking information P4 in chronological order (S14).

Subsequently, the association unit 306 determines whether or not the number of monitoring data associated with the tracking information P4 has reached a predetermined number (S15).

If the predetermined number has not been reached (S15: No determination), processing is performed for the following one frame from step S10.

When the predetermined number is reached (S15: Yes determination), the setting unit 307 sets the monitoring data arranged in the time-series order of the tracking information P4 in the recording table T4 by the table operation unit 301 (S16). After that, from step S10, processing is performed for the following one frame.

FIG. 14 is a diagram showing an example of a flow of alarm basic data generation processing. This process is started at a predetermined timing. For example, the setting unit 307 is started when the number of times the monitoring data is set in the recording table T4 by the table operation unit 301 reaches a predetermined number of times. It should be noted that this process is performed for each individual ID of each surveillance camera unit ID.

First, the alarm basic data generation unit 308 acquires the monitoring data of the individual ID from the recording table T4 by the table operation unit 301 (S21), and generates the behavior information of the cow 91 from the monitoring data (S22).

Subsequently, the alarm basic data generation unit 308 sets the behavior information of the cow 91 in the alarm basic data setting table T5 by the table operation unit 301 (S23).

Since the method of generating the action information has already been explained, the description here will be omitted.

FIG. 15 is a diagram showing an example of the flow of the threshold value determination process. The following processing shall be performed for each individual ID of each surveillance camera unit ID. First, the threshold value determination unit 309 determines whether or not the threshold value update time has been reached (S31). The update time is, for example, the switching time between the night and the morning of the next day (for example, 4:00 AM) at the end of the day. The time for switching between morning and noon, the time for switching between day and night, and the like may be used. Further, the time of each switching may be specified.

When the update time is reached (S31: Yes determination), the threshold value determination unit 309 acquires the action information for each item of the latest several days of the individual ID from the alarm basic data setting table T5 by the table operation unit 301 (S32). ..

Subsequently, the threshold determination unit 309 divides the period of morning, noon, and night, and aggregates the behavior information for several days of the individual ID by the moving average for each item, and from the aggregation result for each item, the relevant The threshold value of the individual ID is determined for each item of action information and for each period (morning period, daytime period, night period) (S33).

Then, the threshold value determination unit 309 updates the threshold value set of each individual ID of the threshold value information P5 to the determined value (S34). It is assumed that a predetermined default value is set as the initial value of the threshold value.

If the update time has not been reached (S31: No determination), the process waits until the update time is reached.

FIG. 16 is a diagram showing an example of the flow of the abnormal behavior determination process of the cow 91. The determination unit 310 performs the next abnormal behavior determination process at predetermined time intervals (for example, every 5 minutes).

First, the determination unit 310 acquires the determination item of the cow bunch 90 for determining the abnormal behavior from the setting information D (S41).

Subsequently, the determination unit 310 acquires the threshold value set for the individual ID of the cow 91 of the cow bunch 90 from the threshold value information P5 for each acquired determination item (S42).

Subsequently, the determination unit 310 uses the table operation unit 301 to display the action information items in the alarm basic data setting table T5 for the period (morning, noon, or night) including the time of the determination in the alarm basic data setting table T5. The individual ID of the cow 91 whose other aggregated value exceeds the threshold value of each item acquired from the threshold information P5 is searched, and the individual ID of the cow 91 exceeding the threshold value and the determination item thereof are extracted (S43). Here, the aggregated value is, for example, an aggregated value in the morning period (4:00 to 9:00 AM) on the day of determination when the determination time is 9:00 AM in the morning period. Specifically, the table operation unit 301 compares the aggregated value of the behavior information with the corresponding upper limit value and the lower limit value for each judgment item for each cow 91, and the aggregated value of the behavior information is the upper limit value and the lower limit value. Determine if it contains abnormal behavior that is out of range.

Subsequently, the determination unit 310 determines the abnormal behavior of the cow 91 from the extraction result (S44).

When the individual ID of the cow 91 and its determination item are extracted (S44: Yes determination), the notification unit 311 acquires an address such as an e-mail address set in the user information registration table T1 and uses that address. Notify an abnormal signal (S45). At this time, even if the individual ID of the cow 91 judged to be abnormal behavior, the loss of friends indicated by the judgment item, estrus, signs of illness, the content of abnormal behavior such as inability to stand up, etc. are notified to the destination. good. In addition, veteran coping methods may be stored in a database, coping methods corresponding to the contents of abnormal behavior may be extracted from the database, and the contents may be notified.

If the individual ID of the cow 91 and the item name of the action information are not extracted (S44: No determination), this process ends.

The search for the individual ID of the cow 91 showing the abnormal behavior is performed as follows, for example. For example, the individual ID in which the sitting time is continuing in the behavior information is searched, and the latest figure of the subject such as the approximate ellipse of the individual ID is acquired. Then, if the aspect ratio of the figure is out of the predetermined range or the contour area is larger than the predetermined area, it is determined that the cow 91 is in the lying state. As an example, when the figure is an approximate ellipse, the lying down of the cow 91 is determined from the ratio of the respective lengths (major axis and minor axis) of the major axis and the minor axis. When the cow 91 is viewed from above, the ratio of the length to the minor axis changes greatly when the cow is in a lying state. The lying down of the cow 91 is determined from this change in the length ratio. When the lying state continues for a certain period of time or more, the cow 91 is determined to be in an incapable state of being unable to return to the sitting position from lying down, and is determined to have abnormal behavior.

Further, the determination unit 310 searches for an individual ID having a large number of contacts in the behavior information in the latest predetermined period, and the travel distance of the individual ID in the latest predetermined period is longer than that of other cows 91, or feed. When the number of times of eating is small or the standing time is longer than that of other cows 91, it is judged that the cow is in estrus and it is judged that there is an abnormality.

Further, the determination unit 310 searches for an individual ID having an extremely small number of contacts in the behavior information of the latest predetermined period, determines that the cow 91 is out of the group, and determines that there is an abnormality.

Further, the determination unit 310 searches for an individual ID having an extremely short travel distance in the behavior information of the latest predetermined period, and determines a sign of illness from the number of times of drinking or feeding the individual ID in the latest predetermined period. Then, it is determined that there is an abnormality.

In this embodiment, monitoring data is generated for the cattle to be monitored, and the monitoring data is used to determine abnormal behavior. However, environmental information (for example, information such as temperature and weather) is further added to the monitoring data. May be included. For example, a temperature sensor is connected to the surveillance camera 10a, and the temperature information in the cow cell detected by the temperature sensor is output to the cloud through the controller PC. Further, the weather information is acquired from, for example, a weather database that provides the weather information in chronological order. Abnormal behavior is determined by including these environmental information in the behavior information indicating the behavior of the cow.

As described above, in this embodiment, it is not necessary to directly attach the sensor to the cow to be monitored, so that the burden on the cow can be reduced or eliminated. In addition, monitoring data including various information on cattle can be accumulated at any time, and slight changes in cattle behavior can be detected in real time or with a small time lag from the accumulated monitoring data. Therefore, if an abnormal behavior is observed in a cattle, it is possible to immediately notify the livestock farmer of the cattle breeder and have the cattle take action at an early stage. If the cow is in a non-standing state, the upper body of the cow is raised, and if the cow shows signs of illness, early treatment can reduce the mortality rate of the cow.

In addition, even for inexperienced new farmers, if there is an abnormal behavior of the cow, the abnormality of the cow is notified as well as the expert, so that it is less likely to be overlooked without noticing the change in the behavior of the cow, and the death of the cow. The waste rate can be reduced. In addition, since it is possible to notify the target method of the expert by e-mail by setting, it is possible to take the same measures as the expert.

In this embodiment, the detection unit 303 mainly corresponds to the "first detection means". The auxiliary unit 304 mainly corresponds to the "second detection means". Mainly, the monitoring data generation unit 305, the association unit 306, the setting unit 307, the alarm basic data generation unit 308, and the like correspond to the "generation means". Mainly, the threshold value determination unit 309, the determination unit 310, and the like correspond to the "determination means". The notification unit 311 mainly corresponds to the "notification means". The correspondence between each means and each function is not limited to this. Depending on the configuration of the information processing system, the combination may be changed as appropriate, some functions may be replaced with other functions, some functions may be deleted, or other functions may be added.

In this embodiment, an example in which the CPU loads the HDD program into the RAM to realize the function is shown, but even if a part or all of the function is configured by hardware such as ASIC (Application Specific Integrated Circuit). good.

(Modification example)
Further, a part or all of the function may be modified so that it can be performed by AI (artificial intelligence) which is a "machine learning means". For example, the AI performs processing such as the alarm basic data generation unit 308, the threshold value determination unit 309, and the determination unit 310.

Regarding the alarm basic data generation unit 308, AI performs a process of generating action information from the monitoring data acquired by the alarm basic data generation unit 308. For example, monitoring data is given to AI as an input value, and machine learning is repeated with AI. From the AI, the output value of the behavior information calculated by the method optimized to minimize the error in the determination of the abnormal behavior is obtained.

For the threshold value determination unit 309, the threshold value determination process is performed by AI. For example, behavior information is given to AI as an input value, and machine learning is repeated with AI. From the AI, the output value of the threshold value calculated by the method optimized to minimize the error in the determination of the abnormal behavior is obtained.

The determination unit 310 uses AI in the determination process of abnormal behavior. By giving AI the action information within the period at the time of judgment as an input value and repeating machine learning, the result of the judgment item is optimized by the calculation method that minimizes the error such as the result of the judgment item of abnormal behavior. Get an output value that indicates.

The AI may perform a series of processes in which any two or more of the alarm basic data generation unit 308, the threshold value determination unit 309, and the determination unit 310 are combined.

1 Information processing system 10 Surveillance camera unit 10a Surveillance camera 10b Control PC
20 Cloud 21 Storage 30 Analysis server device 40 Management PC
50 User terminal 60 Network 70 Abnormal signal 80 Administration building 90 Ushibo 90b Training area

Claims (10)

A spatial sensor that outputs spatial information in the monitoring area,
A first detection means for detecting object information of a plurality of moving objects to be monitored from the spatial information output from the spatial sensor, and
A second detecting means for detecting the arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detecting means.
Based on the approximate figure in chronological order corresponding to the spatial information output from the spatial sensor at a predetermined frame rate by converting the object information and the arrangement information into an approximate figure that approximates the contour of the moving object. , A generation means for generating behavioral information of the plurality of moving objects,
A determination means for determining whether the behavior of the moving object is an abnormal behavior from the behavior information,
When the behavior of the moving object is determined to be an abnormal behavior, a notification means for notifying the abnormality and
An information processing system characterized by having. The object information is contour information indicating the contours of the plurality of moving objects.
The second detection means is an adjacent line segment among the line segments connecting two adjacent points on the contour of the plurality of moving objects in the spatial information output from the spatial sensor at a predetermined frame rate . A set or a set of singular points, which are points whose angle is changed within a predetermined angle within a plurality of frames, is detected, and the contours of the moving objects in contact with each other are defined as boundaries connecting the singular points of each set. Detects placement information that distinguishes between boundaries,
The information processing system according to claim 1. It has correspondence information that associates the stage information of the plurality of moving objects with the determination items of the abnormal behavior.
Based on the corresponding information, the determination means determines whether or not the behavior of the moving object is an abnormal behavior from the behavior information for the determination item of the abnormal behavior corresponding to the stage information of the plurality of moving objects.
The information processing system according to claim 1 or 2 , wherein the information processing system is characterized by the above. The determination means compares the threshold value indicating the abnormal behavior determined based on the behavior information for the most recent predetermined number of days of the plurality of moving objects with the behavior information on the determination day of the determination means, and determines the abnormal behavior. Determine if there is,
The information processing system according to any one of claims 1 to 3 , wherein the information processing system is characterized by the above. The determination means determines that the plurality of moving objects are in an incapable state based on the ratio of the length of the major axis to the length of the minor axis of the approximate figure.
The information processing system according to claim 1 . The determination means determines that the plurality of moving objects cannot stand up based on the contour area of the approximate figure.
The information processing system according to claim 1 . The generation means generates contact count information as the action information based on the detection of the arrangement information by the second detection means.
The information processing system according to any one of claims 1 to 6 , wherein the information processing system is characterized by the above. A machine learning means for performing machine learning is provided as a part of the generation means or the determination means.
The information processing system according to any one of claims 1 to 7 . The first detection means for detecting the object information of a plurality of moving objects to be monitored from the spatial information output from the spatial sensor, and
A second detecting means for detecting the arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detecting means.
Based on the approximate figure in chronological order corresponding to the spatial information output from the spatial sensor at a predetermined frame rate by converting the object information and the arrangement information into an approximate figure that approximates the contour of the moving object. , A generation means for generating behavioral information of the plurality of moving objects,
A determination means for determining whether the behavior of the moving object is an abnormal behavior from the behavior information,
When the behavior of the moving object is determined to be an abnormal behavior, a notification means for notifying the abnormality and
Information processing device with. Computer,
The first detection means for detecting the object information of a plurality of moving objects to be monitored from the spatial information output from the spatial sensor, and
A second detecting means for detecting the arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detecting means.
Based on the approximate figure in chronological order corresponding to the spatial information output from the spatial sensor at a predetermined frame rate by converting the object information and the arrangement information into an approximate figure that approximates the contour of the moving object. , A generation means for generating behavioral information of the plurality of moving objects,
A determination means for determining whether the behavior of the moving object is an abnormal behavior from the behavior information,
When the behavior of the moving object is determined to be an abnormal behavior, a notification means for notifying the abnormality and
A program to make it work.

Images