JP7648071B2 - Monitoring device, monitoring program, storage medium, and monitoring method - Google Patents

Description

The present invention relates to a monitoring device, a monitoring program, a storage medium, and a monitoring method, and in particular to a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device.

Livestock production has been rapidly expanding in scale in recent years. Traditionally, most producers were family-run businesses, but now agricultural cooperatives and corporations produce large quantities of chicken meat, eggs, pigs, beef, and other livestock for domestic and international markets.

As livestock production became larger, production techniques and livestock breeding also improved, making it possible to steadily produce large quantities of high-quality livestock in standardized livestock barns. In particular, automated machines suitable for feeding and watering livestock were put to practical use, and in large-scale livestock barns, almost no human labor was required for tasks other than cleaning.

However, there are also drawbacks to large-scale operations. In large-scale livestock farms, in order to ensure the stable production of high-quality livestock, it is necessary to quickly detect and remove livestock that are in a poor condition, such as sick or dead livestock. This requires experienced staff to concentrate and inspect the vast livestock farms, observing the condition of the livestock. However, there are only a few experienced staff, and inexperienced staff or staff who lack concentration will miss things or miss inspections, which can lead to diseases being transmitted to other livestock or new diseases emerging due to the decay of carcasses.

Surveillance cameras have been used for a long time, but with the spread of Internet lines, they are increasingly being connected to a network and monitored by a remote server. Cameras also have more pixels, making it possible to analyze recorded images after the fact and identify accidents and crimes. The most common use of surveillance cameras today is to install unmanned surveillance cameras in buildings and monitor the surveillance camera footage from a remote location to help with crime prevention. Security personnel actually view the many images from the surveillance cameras on multiple display devices, recognize suspicious targets (suspicious people or objects), and, if necessary, dispatch security guards or warn the suspicious person.

Conventional technologies for automating surveillance cameras include an image processing device, an image processing system, and a program (see Patent Document 1). This technology involves "setting a gaze area within a panoramic image of a surveillance area, analyzing the image of the set gaze area with an image analysis means to detect a moving object, and operating the zoom camera of the camera unit based on the detection to photograph the moving object. At that time, spatial relationship information with the panoramic image and temporal relationship information showing the transition of the gazed-part image on the time axis in the panoramic image are created for the gazed-part image, and the gazed-part image is assigned with each of the relationship information and stored in the image storage unit. During a search, the panoramic image and the gazed-part image are searched for using each of the relationship information, and the two images are displayed superimposed on the display unit."

JP 2010-233185 A

The above-mentioned conventional technology involves first setting a surveillance area, and then zooming in and photographing any moving objects within that surveillance area as suspicious targets. This is a technology that automatically tracks and zooms in on suspicious individuals, but it does not allow for "automatic setting of surveillance areas." Furthermore, when the surveillance area is large or when multiple moving objects are present within the surveillance area, unless a high-speed computing device is used, the processing cannot keep up, and there are disadvantages such as making it impossible to capture suspicious targets.

Thus, although the technology for automating surveillance cameras has advanced, no technology has been developed to identify or pinpoint abnormal individuals in livestock or agricultural products.

The object of the present invention is to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method.

Another object of the present invention is to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device.

In order to solve the above-mentioned problems, a monitoring device (such as a livestock barn monitoring device) according to a first aspect of the present invention comprises:
An image acquisition unit that acquires an image including location information of the location where the image was taken;
A memory unit for storing the acquired image and standard specific part information, which is standard physical information of a specific part of an individual (in the case of chickens, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.);
an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
Including,
It is characterized by:

In addition, the monitoring device (livestock barn monitoring device) according to the second invention is
an image acquisition unit that acquires an image including location information of the location where the image was taken;
A memory unit for storing the acquired images and abnormal specific part information, which is physical information of a specific part of an abnormal individual (in the case of chickens, color information, dimensional information, shape information, etc. of the comb or wattle of an abnormal, sick, weakened, injured, or dead chicken (specific parts that tend to indicate a health condition are preferable));
an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the individual with the specific part and an image including the specific part;
Including,
It is characterized by:

In addition, the monitoring device (livestock barn monitoring device) according to the third invention is
an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken;
A storage unit for storing the acquired image;
A generating unit that generates standard specific part information, which is standard physical information of at least one specific part of an individual based on each individual in the stored image (in the case of a chicken, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.);
an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit which compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
Including,
It is characterized by:

Further, a monitoring device (livestock barn monitoring device) according to a fourth aspect of the present invention comprises:
The generation unit,
Generate standard specific part information, which is standard physical information of at least one specific part of an individual, based on only images of each individual from a predetermined period (preferably the most recent 1-3 weeks) among the stored images. (In the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.)
It is characterized by:
Since each part of a livestock changes shape, size, color, etc. as it grows, this configuration can reflect the standard condition of the current livestock, making it possible to more accurately identify abnormal individuals.
Further, a monitoring device (livestock barn monitoring device) according to a fifth aspect of the present invention comprises:
In the third or fourth invention,
The generating unit, the identifying unit, or the warning unit,
It operates based on AI techniques,
It is characterized by:

Further, a monitoring device (livestock barn monitoring device) according to a sixth aspect of the present invention comprises:
An image acquisition unit that acquires an image including location information of the location where the image was taken;
A memory unit for storing the acquired images and an AI model trained using AI techniques (based on specific parts of the individual labeled in the previously acquired images);
A recognition unit that uses the AI model to recognize specific parts of the stored image (such as the comb and wattles in the case of chickens);
a warning unit that uses the AI model to determine whether the identified specific portion is abnormal, and if the specific portion is abnormal, outputs warning information including location information of the individual having the specific portion and an image including the specific portion;
The present invention is characterized in that it includes:

In addition, a monitoring device (livestock barn monitoring device) according to a seventh aspect of the present invention comprises:
an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken;
A storage unit for storing the acquired image;
A generation unit that labels each individual in the stored images as abnormal (indicating unhealthy conditions such as illness, death, or weakness) or labels a specific part, and generates an AI model trained using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit that uses the AI model to determine whether the identified specific portion is abnormal, and if the specific portion is abnormal, outputs (transmits) warning information including location information of the individual having the specific portion and an image including the specific portion;
The present invention is characterized in that it includes:

In addition, a monitoring device (livestock barn monitoring device) according to an eighth aspect of the present invention comprises:
an image acquisition unit that acquires an image of an individual including location information of the location where the image was taken;
A storage unit for storing the acquired video;
A generation unit that labels the actions of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification unit that identifies the behavior of an individual in the stored video using an AI model;
a warning unit that uses an AI model to determine whether or not the behavior of the identified individual is abnormal, and if the behavior is abnormal, outputs warning information including location information of the individual and an image including the individual;
The present invention is characterized in that it includes:
Preferably, the AI model is generated by weighting stored images taken within a specified period (preferably the most recent 1-3 weeks). Since the shape, size, color, etc. of each part of livestock change as they grow, this configuration makes it possible to reflect the standard condition of the current livestock, making it possible to more appropriately identify abnormal individuals. For example, in the case of pigs, it is possible to determine abnormalities by setting the pattern of spots that appear on the skin when they have a specific infection, the color, the degree of bloodshot eyes, etc.

In addition, a monitoring device (livestock barn monitoring device) according to a ninth aspect of the present invention comprises:
a monitoring information acquisition unit that acquires location information of a monitored location and monitoring information including measurement values (such as sound, odor, temperature, and humidity) measured at the monitored location;
A storage unit for storing the acquired monitoring information;
A generator that labels the measurements of the stored monitoring information and generates a trained AI model using AI techniques (e.g., techniques such as reinforcement learning or deep learning);
an identification unit that identifies measurement values of the stored monitoring information using the AI model;
a warning unit that uses the AI model to determine whether the identified measurement value is abnormal, and if the identified measurement value is abnormal, outputs warning information including location information and the measurement information;
The present invention is characterized in that it includes:

As mentioned above, the solution of the present invention has been described as an apparatus, but the present invention can also be realized as a method, program, or storage medium on which a program is recorded that substantially corresponds to these, and it should be understood that these are also included within the scope of the present invention. Note that each step of the method and program below uses a processor such as a CPU or DSP for data processing as necessary, and input data and processed/generated data are stored in a storage device such as a magnetic tape, HDD, or memory.

For example, a monitoring program according to a tenth aspect of the present invention,
A monitoring program that causes one or more processors to function as the monitoring device according to any one of the first to ninth aspects of the present invention.

For example, a storage medium according to an eleventh aspect of the present invention is a storage medium,
A computer-readable storage medium storing the monitoring program according to a tenth aspect of the present invention.

For example, a monitoring method using a monitoring device according to a twelfth aspect of the present invention, which realizes the present invention as a method, is as follows:
An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken;
A storage step in which the arithmetic device stores the acquired image and standard specific part information, which is standard physical information of a specific part of an individual (in the case of chickens, color information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.);
An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual with the specific part and an image including the specific part;
Including,
It is characterized by:

A monitoring method according to a thirteenth aspect of the present invention comprises:
An image acquisition step in which a computing device acquires an image including location information of a place where the image was taken;
a storage step in which the computing device stores the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual (in the case of chickens, color information, shape information, etc. of the comb, wattles (specific parts that tend to indicate a health condition) of an abnormal, sick, weakened, injured, or dead chicken);
An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the individual with the specific part and an image including the specific part;
Including,
It is characterized by:

A monitoring method according to a fourteenth aspect of the present invention comprises:
An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken;
A storage step in which the computing device stores the acquired image;
A generation step in which the computing device generates standard specific part information, which is standard physical information of at least one specific part of an individual based on each individual in the stored image (in the case of a chicken, color information, shape information, etc. of the chicken's comb, wattles, etc.);
An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning step in which the computing device compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual of the specific part and an image including the specific part;
Including,
It is characterized by:

A monitoring method according to a fifteenth aspect of the present invention comprises:
In a monitoring method according to a thirteenth aspect of the present invention,
The generating step includes:
Generate standard specific part information, which is standard physical information of at least one specific part of an individual, based on only images of each individual from a predetermined period (preferably the most recent 1-3 weeks) among the stored images. (In the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.)
It is characterized by:
Since each part of a livestock changes shape, size, color, etc. as it grows, this configuration can reflect the standard condition of the current livestock, making it possible to more accurately identify abnormal individuals.

A monitoring method according to a sixteenth aspect of the present invention comprises:
In the fourteenth or fifteenth aspect of the present invention,
The generating step, the identifying step, or the step
It operates based on AI techniques,
It is characterized by:

A monitoring method according to a seventeenth aspect of the present invention comprises:
An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken;
A storage step in which the computing device stores the acquired image and an AI model trained using an AI technique based on the labeled specific part of the individual;
An identification step in which the computing device uses the AI model to identify a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning step in which the computing device uses the AI model to determine whether the identified specific portion is abnormal, and if the specific portion is abnormal, outputs warning information including location information of the individual in the specific portion and an image including the specific portion;
The present invention is characterized in that it includes:

A monitoring method according to an eighteenth aspect of the present invention comprises:
An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken;
A storage step in which the computing device stores the acquired image;
A generation step in which the computing device labels specific parts of each individual in the stored images and generates an AI model trained using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification step in which the computing device identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image using an AI model;
a warning step in which the computing device uses an AI model to determine whether the identified specific portion is abnormal, and if abnormal, outputs (transmits) warning information including location information of the individual in the specific portion and an image including the specific portion;
The present invention is characterized in that it includes:

A monitoring method according to a nineteenth aspect of the present invention comprises:
An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken;
a storage step in which the computing device stores the acquired image;
A generation step in which the computing device labels the actions of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification step in which the computing device identifies the motion of an individual in the stored video using the AI model;
a warning step in which the computing device uses the AI model to determine whether the behavior of the identified individual is abnormal, and if the behavior is abnormal, outputs warning information including location information of the individual and an image including the individual;
The present invention is characterized in that it includes:

A monitoring program according to a twentieth aspect of the present invention, which is a program implementing the present invention,
A monitoring program for causing a computer to execute the monitoring method according to any one of the eleventh to nineteenth aspects of the present invention.

In addition, a storage medium according to a twenty-first aspect of the present invention is a storage medium,
A computer-readable storage medium storing the monitoring program according to the twentieth aspect of the present invention.

As mentioned above, up to this point, we have been explaining the aspects of a monitoring device. Below, we will explain the aspects of a general mobile monitoring device, such as a drone or a self-propelled device using wheels or tracks, and a server that uses distributed processing.

A monitoring device according to a twenty-second aspect of the present invention comprises:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device,
An imaging unit that captures an image;
A drive unit that moves the device itself;
An acquisition unit (such as a GPS unit) that acquires the location of the mobile monitoring device;
a communication unit that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means;
having
The server,
a communication unit that receives the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position;
A storage unit for storing the monitoring information, including the position of the mobile monitoring device and an image at the position, and standard specific body part information, which is standard physical information of a specific body part of an individual (in the case of chickens, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.);
an identification unit that identifies a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
having
It is characterized by:
In other words, the identified specific part is compared with the standard specific part information stored in the memory unit, and if it is determined that the identified specific part is abnormal (abnormal event (phenomenon)), warning information is output that includes location information of the individual with the specific part and an image including the specific part.

A monitoring device according to a twenty-third aspect of the present invention comprises:
The drive unit of the mobile monitoring device is
It is self-propelled,
It is characterized by:

A monitoring device according to a twenty-fourth aspect of the present invention comprises:
The generating unit, the identifying unit, or the warning unit,
It operates based on AI techniques,
It is characterized by:

In addition, a monitoring device (livestock barn monitoring device) according to a 25th aspect of the present invention comprises:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device,
An imaging unit that captures an image;
A drive unit that moves the device itself;
An acquisition unit (such as a GPS unit) that acquires the location of the mobile monitoring device;
a communication unit that transmits monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means;
having
The server,
a communication unit that receives the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position;
A memory unit for storing the received images and an AI model trained using AI techniques (based on the specific parts of the individual labeled in the previously acquired images);
A recognition unit that uses the AI model to recognize specific parts of the stored image (such as the comb and wattles in the case of chickens);
a warning unit that uses the AI model to determine whether the identified specific portion is abnormal, and if the specific portion is abnormal, outputs warning information including location information of the individual having the specific portion and an image including the specific portion;
The present invention is characterized in that it includes:

The server,
a memory unit for storing a number of images and monitored events associated with at least some of the images;
The monitoring unit,
When monitoring whether a set monitoring event has occurred, an AI technique is used to refer to the storage unit and monitor whether the set monitoring event has occurred.
It is characterized by:

A monitoring device according to a twenty-sixth aspect of the present invention comprises:
The communication unit of the mobile monitoring device
Further receiving monitoring event salience information (feeding information (feeding location, time, etc.), intruder intimidation information (alarm sound generation location, time, etc.)),
The drive unit of the mobile monitoring device is
Controlling the movement of the device itself based on the monitored event salience information.
It is characterized by:

A monitoring device according to a twenty-seventh aspect of the present invention comprises:
The communication unit of the server
Further receiving monitoring event salience information (feeding information (feeding location, time, etc.), intruder intimidation information (alarm sound generation location, time, etc.)),
The server,
Based on the monitoring event salience information, the driving unit of the mobile monitoring device is controlled via the communication unit to move the mobile monitoring device.
It is characterized by:

A monitoring device according to a twenty-eighth aspect of the present invention comprises:
The monitored event salience information is
The information may be one or more selected from the group consisting of feeding information (e.g., feeding location and time), intruder scare information (e.g., warning light, warning vibration, warning sound location and time), odor information (e.g., odors that the monitored chickens dislike or are excited by), light information (e.g., lighting and indoor illuminance, flashing lights, rotating lights), opening/closing information (e.g., opening/closing of windows and ventilation fans), and air conditioning setting information.
It is characterized by:

A monitoring device according to a twenty-ninth aspect of the present invention comprises:
The mobile monitoring device,
The device further includes a monitoring event conspicuous situation generating means for generating the situation of the monitoring event conspicuous information (e.g., a rotating light that generates light, a speaker that generates sound, a rod-shaped part that makes contact with the chicken coop as it moves and generates vibrations and sound, a signal output part that starts feeding, a control signal output part that controls the output of lighting and sound, etc.).
It is characterized by:

A monitoring program according to a 30th aspect of the present invention comprises:
A monitoring program that causes one or more processors to function as the monitoring device according to any one of the twenty-first to twenty-eighth aspects of the present invention.

A storage medium according to a thirty-first aspect of the present invention comprises:
A computer-readable storage medium storing the monitoring program according to the thirtieth aspect of the present invention.

FIG. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 2 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. FIG. 3 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention. FIG. 4 is a schematic diagram illustrating another monitoring technique of the mobile monitoring device of the present invention. FIG. 5 is a schematic diagram for explaining the function of the acquisition unit in the present device. FIG. 6 is a schematic diagram for explaining the flight route setting function in this device. FIG. 7 is a schematic diagram for explaining a monitoring technique by the monitoring unit. FIG. 8 is a schematic diagram for explaining map-like alarm information. FIG. 9 is a schematic diagram for explaining map-like alarm information. FIG. 10 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. FIG. 11 is a schematic diagram for explaining a monitoring technique of the mobile monitoring device of the present apparatus provided with a monitoring event prominent situation generating means. FIG. 12 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. FIG. 13 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present invention when applied to a chicken coop. FIG. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 15 is a flowchart illustrating an example of a process executed by the monitoring device illustrated in FIG. FIG. 16 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (generation of standard specific site information). FIG. 17 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilization of standard specific part information). FIG. 18 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (AI model generation). FIG. 19 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing an AI model). FIG. 20 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (application of AI model generation).

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

<Embodiment 1>
Fig. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, the monitoring device MOS1 includes mobile monitoring devices MM1 and MM2 and a server SV1. The server SV1 has a control unit (CPU, arithmetic processing unit, processor) CON, an input unit IN, an output unit OUT, a communication unit COM, a memory unit MEM, and a display unit DIS. The mobile monitoring device MM2 has the same configuration and functions as the mobile monitoring device MM1 unless otherwise specified, but some components are omitted for convenience of drawing and explanation.

The mobile monitoring device MM1 has cameras CM1-3 as an imaging unit, a sensor SNSR (barometer, gyro sensor, etc.), a control unit CON (CPU, arithmetic processing unit, processor), a communication unit COM, a monitoring unit MON (this function may be provided on the server side), an output unit OUT, a memory unit STR, and a GPS unit GR, but may also have an input unit (not shown) and a display unit (not shown). It is preferable to have two cameras to photograph the monitoring target such as a chicken coop and one to photograph a position indicator (such as a point-shaped marker or a line-shaped indicator) installed on the ceiling or floor, but one or two cameras may serve both functions. It is also preferable to use a high-performance model that can take pictures even at night with little environmental light. Furthermore, the camera may be provided with a separate infrared detector or the above-mentioned camera may be equipped with an infrared function to detect the temperature of the monitoring target and use it as a coloring material to determine the condition of the target, for example, whether it is dead or sick.

The mobile monitoring device MM1 has, as a sensor SNSR, a position measurement unit (not shown, which may be a GPS unit) that measures the position of the device itself, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, an altitude sensor that measures altitude, and a rangefinder that measures or estimates the distance traveled by the device itself.

The mobile monitoring device MM1 includes:
An imaging unit CM1-3 that captures an image;
A driving unit DR that moves the device itself;
an acquisition unit RET (such as a GPS unit) for acquiring the location of the mobile monitoring device;
a communication unit COM for transmitting monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means;
has.

The server SV1 is
The server,
A communication unit COM (and an acquisition unit ACQ) that receives (acquires) the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position;
a memory unit MEM for storing the monitoring information MI including the position of the mobile monitoring device and an image at that position, and standard specific part information SPI which is standard physical information of a specific part of an individual (in the case of a chicken, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.);
An identification unit IDP for identifying a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit WAN which compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
has.
For example, color information of a healthy chicken's comb, wattles, etc. is preferably expressed in RGB values. Below are the colors with the strongest red tints:
RGB
(255.0.0)
It is preferable to set the RGB values within a range that is healthy and standard and acceptable, and it is preferable to set the RGB values from statistical information of the livestock barn. In addition, as for the size information, since individuals that cannot grow to a standard size (dimensions) are likely to die due to illness, weakness, or other factors, it is important in terms of an appropriate breeding environment to be able to grasp such abnormalities.

The output alarm is preferably an email, an audio warning, an image, a warning to a security guard, a warning to an administrator, or a report to a security company or the police. The monitoring unit preferably monitors whether a set monitoring event has occurred by using AI techniques and referring to the image information IM in the storage unit (or information derived from the image information (such as a characteristic pattern of the image)).

The mobile monitoring device MM1,
The guide portion GD is movably supported, placed or suspended on the guide rail and guided along the guide rail;
The drive unit DR is
The device itself is moved along the guide rail while being supported, placed or suspended on the guide rail.

CAD information on the building, shelves, chicken coop, etc. to be monitored may be acquired from a building information server or memory unit MEM connected via the network NET. A movement route is set from the CAD information, position index placement information that corresponds to or is made to correspond to the spatial coordinate system of this CAD information, and layout information on the objects to be monitored that are placed in the building (chicken coop cages, desks, chairs, equipment, etc.). The display unit DIS can display information stored in the device and information generated.

It is preferable that each functional unit included in the control unit of server SV1 is realized by a program module loaded into the memory space of the control unit. Furthermore, if the mobile device is realized as a drone or a self-propelled device, each functional unit can be realized by a program module loaded into the memory space of the control unit, or each functional unit can be realized by a built-in GPS unit or camera unit. Typically, the server downloads software from a storage unit or a website that causes the processor to function as each unit of this device, and installs and starts the software on the PC, causing the PC to operate as this device. Note that each functional unit provided in the control unit is merely a collection of steps that have a certain degree of functionality, and it is possible to combine multiple functional units into one functional unit, incorporate part of it into another functional unit, or split it into other independent functional units.

Please note that in this way, the generated/extracted information, intermediate data, and acquired data can be sent to the outside, displayed on the display unit, and stored in the storage unit in the other actual embodiments described below. It is preferable that the present device is constructed on a computer such as a general-purpose computer, a specific-purpose computer, a server, a PC, or a mobile terminal such as a smartphone, or that a program module that realizes (executes) the functions and processing procedures (methods) of the present device on a computer is stored in a CPU or storage unit of the computer, or read from an external server or storage, and this is also the case in each of the subsequent embodiments. In addition, each functional unit may be distributed to a separate computer or device connected by a network. In addition, multiple functional units may be combined into one, or some of the processing steps may be assigned to other functional units. Furthermore, in this embodiment, the server and the mobile monitoring device are specified as separate entities, but the mobile monitoring device may include some or all of the functions of the server.

The server SV1 has a generation unit GEN that generates standard specific part information (in the case of chickens, color information, dimensional information, shape information, etc. of the chicken's comb, wattles, etc.) which is standard physical information of at least one specific part of an individual based on each individual in the stored image. This makes it possible to obtain reference information that matches the documents of the livestock house and livestock, making it possible to more appropriately identify, specify, and extract abnormal individuals and issue an alarm.

Figure 16 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (generation of standard specific body part information). As shown in the figure, standard specific body part information SPI1 (which is numerical information on the color of the comb, preferably specified in RGB) and standard specific body part information SPI2 (which is numerical information on the size of the wattle) are generated from image information SI3 showing chickens BD1-BD7 and stored in a memory unit. The following figure shows how the standard specific body part information is utilized.

Figure 17 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing standard specific body part information). As shown in the figure, the standard specific body part information SPI1 is compared with the color of the comb of chicken BD12. In this case, the color of the comb of chicken BD12 falls within the range of the standard specific body part information SPI1, so it is determined that they match and chicken BD12 is determined to be normal. Then, a comparison is made with another individual.

On the other hand, the standard specific body part information SPI1 is compared with the color of the comb of chicken BD17. In this case, the color of the comb of chicken BD17 is outside the range of the standard specific body part information SPI1, so it is determined that there is no match, i.e., an abnormality, and warning information including the position and image of individual BD17 is output.

Figure 2 is a flow chart showing an example of processing executed by the monitoring device shown in Figure 1. The monitoring device performs processing in cooperation with two devices, a server SV1 and a mobile monitoring device MM1. If information and functions available on other servers are installed in the mobile device, it is possible to eliminate the need for cooperation with the server.

In step S11, the camera of the mobile monitoring device captures an image. Next, in step S12, the acquisition unit acquires the position of the mobile monitoring device. Then, in step S13, monitoring information including the position of the mobile monitoring device and an image at that position is transmitted to the server via wireless or wired communication means. In step S14, the server moves the mobile monitoring device along the guide rail while the drive unit of the mobile monitoring device confirms the position using a position indicator and/or a gyro sensor and acceleration sensor (or GPS, rangefinder). Note that the movement may be a simple method such as supplying motor power for a predetermined distance (such as the distance of one chicken cage) without using a sensor, or operating the motor for a predetermined time.

In step S15, the camera of the mobile monitoring device captures an image. Next, the position of the mobile monitoring device is obtained, and in step S16, monitoring information including the position of the mobile monitoring device and an image at that position is sent to a server via wireless or wired communication means. In step S17, it is determined whether the end of the rail has been reached, or whether the battery installed in the mobile monitoring device is running low on power. If the determination conditions are met, the process proceeds to step S19-2 and the device returns to the base. If the determination conditions are not met, the process proceeds to step S18.

In step S18, it is determined whether an abnormal event (monitoring event) has occurred in the image. If an abnormal event has not occurred, the process returns to step S15 and movement and imaging are continued. If an abnormal event has occurred, an alarm may be issued, but in order to reduce false recognition, it is preferable to proceed to step S18-1, stop or slow down the mobile monitoring device, and continue observing the image of the abnormal event. Then, if it is determined in step S18-2 that the occurrence of the abnormal event is continuing, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S19. If it is determined in step S18-2 that the occurrence of the abnormal event is not continuing, it is determined that the occurrence is a false recognition, and no alarm is issued, and the process returns to step S15 and movement (monitoring) is continued. After issuing an alarm in step S19, the process proceeds to step S19-1, and, as in step S17, it is determined whether the end of the rail (route) is reached or whether the remaining power of the onboard battery is insufficient. If the determination condition is met, the mobile monitoring device returns to its base (charging spot). If the judgment conditions are not met, return to step S15 and continue moving (monitoring).

The determination of an abnormal event is made, for example, by comparing the identified specific part with the standard specific part information stored in the memory unit, and if the specific part and the standard specific part information are apart by a predetermined numerical value or more, or if the specific part and the standard specific part information differ, warning information is output that includes location information of the individual with the specific part and an image including the specific part. In other words, the identified specific part is compared with the standard specific part information stored in the memory unit, and if it is determined that the identified specific part is abnormal (abnormal event (phenomenon)), warning information is output that includes location information of the individual with the specific part and an image including the specific part.

Figure 3 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of this device. As shown in the figure, the position indicators PI1 to 4 installed on the ceiling of the building BLD are photographed by the camera CM3, and the position of the device is recognized based on the indicators. The position indicators are arranged at the coordinates of the position indicator arrangement information, and the mobile monitoring device MM1 controls the movement (drive) of the device while recognizing the position of the device by the position indicators and sequentially recognizing the position indicators PI1 to 4 along the movement route RT1. When moving, the device moves along a guide rail GR (which may be placed or installed on the ceiling or wall) placed on the floor. The position indicator may be integrated into the guide rail GR. Alternatively, the position may be measured and estimated using a range finder or the like without using the position indicator. Also, the position indicator is preferably one or more of fluorescent materials, letters, figures, patterns, colors, etc., and is preferably photographed by the camera CM3 facing the ceiling to image recognize the position indicator. Each location index is unique, and by recognizing them individually and referring to the location index placement information LILI, it is possible to determine where the mobile monitoring device is located. Alternatively, the location index may be a device that emits a specific radio or optical signal, a WIFI base station, or a wireless tag (in this case, the signal is read by a wireless communication unit, optical receiver, etc., instead of a camera). It is also possible to provide an indoor GPS signal unit instead of a location index, and have the GPS unit of the mobile monitoring device read this.

Then, while moving, the mobile monitoring device uses cameras CM1 and CM2 to capture images of the left and right monitored shelves SBJ1 and SBJ2 (installed in the building BLD) and monitors whether a monitoring event is occurring.

Figure 4 is a schematic diagram illustrating another monitoring technique of the mobile monitoring device of the present device. As shown in the figure, the mobile monitoring device MM2 is self-propelled, driven by tires or caterpillars, and moves along a guide line GL placed on the floor or painted on it. Other functions are the same as those in Figure 16. The guide line GL is preferably a magnetic tape, a coating impregnated with magnetism, or a film, tape, or coating with a distinctive color scheme (e.g., white, yellow, etc.). The guide line corresponds to the flight route described above, and functions as a travel route.

Figure 5 is a schematic diagram explaining the function of the acquisition unit in this device. As shown in the figure, this device acquires CAD information CI of a building. This CAD information is three-dimensional CAD, and the components, walls, ceilings, floors, etc. inside the building have three-dimensional spatial coordinates. In addition, this device acquires layout information SLI of the installations to be monitored (chicken cages, desks, chairs, equipment, etc.) placed in the building, which corresponds to the spatial coordinate system of the CAD information CI of the building. The layout information SLI is incorporated (merged) into the CAD information CI. Then, this device acquires position index arrangement information PI1 to 4 corresponding to the spatial coordinate system of the CAD information CI of the building, and as shown in the lower part of the figure, the position index arrangement information PI1 to 4 is incorporated (merged) into the CAD information CI. As a result, CAD information CIALL is obtained, which describes the entire internal structure of the building, including the position indexes and installations placed in the building. This device sets flight routes and movement routes in this CAD information CIALL, which is explained in the following diagram.

Here, the mobile device is described as a drone, but of course, the mobile device may be a self-propelled vehicle. FIG. 6 is a schematic diagram for explaining the function of setting a flight route in this device. As shown in the figure, in the CAD information CIALL, which also describes the position of the drone base DB, there are monitoring targets SBJ1 and SBJ2, which are installations arranged corresponding to the layout information SLI, and there is an aisle (a space suitable for flight) between them. This device automatically sets a flight route FR1 based on a route setting rule (stored in the memory unit) that the flight route is set in a space that can monitor the monitoring targets and has a predetermined width (it can also be set manually). Similarly, there are monitoring targets SBJ3 and SBJ4, and there is an aisle (a space suitable for flight) between them. This device has a function of recognizing a space large enough for a person to pass through as an aisle. The flight route FR2 is also set automatically based on route setting rules (stored in the memory unit) that require the target to be monitored, to be in a space of a certain size, and to be set at a certain height (usually 160 cm, which is about the eye level of an average person). (It can also be set manually.) As shown in the figure, when flight routes FR1 and FR2 are set, the order of the flight routes is set so that FR1 comes first and FR2 comes second. Alternatively, all flight routes may be linked together to set a single flight route.

In this example, the target of monitoring is assumed to be a chicken coop with one level, but in the case of two or three levels, the flight route will be the same coordinates in plan view, but it is possible to set a different flight route with successively changing altitudes to monitor the second, third, etc. levels. Alternatively, as in Figure 1, a camera for capturing images of each level can be installed for each level.

Figure 7 is a schematic diagram explaining the monitoring technique. An example of monitoring a chicken coop will be explained. As shown in the figure, chickens BD1 to BD7 are present on the screen interface SL1 captured by the camera. This device has logic to recognize specific parts of chickens, such as the comb and beak, by shapes, colors, patterns, etc., and captures all chickens photographed on the screen. Then, the device observes the movements of each individual chicken and extracts those that are motionless. In addition, the device compares the specific parts with standard specific part information, which is the standard physical information of the specific parts of an individual chicken, which is the master stored in the memory unit (in the case of chickens, color information, dimensional information, shape information, etc. of the comb, wattles, etc. of a standard healthy chicken), and if the specific parts and the standard specific part information are separated by a predetermined value or more, or if the specific parts and the standard specific part information are different, that is, whether there is an abnormality or not is determined. In the screen interface SL2, the comb HDT4 of the chicken BD4 is judged to be abnormal because it is whitish and is a color tone quite different from the bright red color of the comb of a normal healthy chicken, which is the standard specific body part information. Such chickens and chickens that are not moving are determined to be monitoring events, that is, abnormal events (death, illness, or weakness). The chicken BD4 in the screen interface SL3 is a chicken that has been determined to be an abnormal event, and an alarm is issued at this point (or when it returns to the base). In addition, in the case of chickens, since their bodies are covered with feathers, it is difficult to measure their body temperature from the outside using normal methods. However, the comb part has no feathers and blood vessels run just below it, so by measuring the temperature of this part with an infrared camera or the infrared function of a normal camera, it is possible to easily measure the body temperature and reliably determine whether the chicken is dead or not. This device captures the comb as a region of interest using a known image recognition technique, and furthermore, by measuring the temperature of the comb, it is possible to more accurately determine abnormal events.

In this way, in this embodiment, it is possible to identify dead, sick, or weak chickens by checking specific parts of the chicken, such as the comb and the maroon color of the wattles.

Figure 8 is a schematic diagram explaining map-like alarm information. Images and videos of the monitoring event and the coordinates of the location are transmitted (output) from the drone or mobile device. This makes it difficult for the staff who receive the alarm to understand the situation, so the device issues map-like alarm information WRN1 that displays the abnormal event occurrence points OABE1-3 of the buildings BLD1 and 2 where the monitoring event (abnormal event) occurred, as well as detailed information such as "Abnormal event occurred (third row), cage number A101BR031." This alarm is created by flattening CAD information to display the abnormal event occurrence points. Based on the alarm information, the staff checks the cage and performs the prescribed work (in this example, removing dead or weakened individuals (chickens)).

Figure 9 is a schematic diagram explaining map-like alarm information. Map-like alarm information WRN1 is a plan view, but in a chicken coop or the like, there are several cage shelves in the vertical direction, and it is unclear at what height in the plan view the cage shelves are. Map-like alarm information WRN2 in Figure 13 corresponds to this. This is used for the purpose of allowing an operator to look at the map in Figure 8, reach the point where an abnormal event has occurred, and then look at the map in Figure 9 to understand at what vertical height of the shelf the abnormality has occurred.

Figure 10 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of this device when applied to a chicken coop. As shown in the figure, the guide section GD of the mobile monitoring device MM3 is supported on a guide rail GR installed on the ceiling SL of the building BLD, and the mobile monitoring device MM3 is suspended. The mobile monitoring device MM3 controls the movement (drive) of its own device along the movement route RT1 while recognizing its own position using some kind of position indicator or a range finder. When moving, it moves along the guide rail GR installed on the ceiling SL. The position indicator may be integrated into the guide rail GR. Also, the position may be measured and estimated using a range finder or the like without using a position indicator.

The mobile monitoring device MM3 has cameras CM2-3 for the chicken coops CG31 and CG41 on the upper level, and cameras CM4-5 for the chicken coops CG32 and CG42 on the lower level. In this example, dead or sick or otherwise weak chickens are abnormal events, while chickens BD11 and BD12 are normal and are not recognized as abnormal events. Chickens BD13 and BD14 are crouching or fallen, so the server determines this to be an abnormal event and issues an alarm. When an alarm is issued, the chicken coop number CG32 or CG42 indicating the location, or some other location information, and the image are output.

By using the monitoring event salience information, it is possible to improve the accuracy of monitoring and reduce false alarms to a considerable extent. The monitoring event salience information is one or more selected from the group consisting of feeding information (e.g. feeding location, time, etc.), intruder scare information (e.g. warning light, warning vibration, warning sound location, time, etc.), odor information (e.g. odors that the monitored chickens dislike or are excited by), light information (e.g. lighting and indoor illuminance, flashing lights, rotating lights), opening/closing information (e.g. opening/closing of windows and ventilation fans), and air conditioning setting information.

Therefore, it is preferable to further provide the mobile monitoring device with a monitoring event salience situation generating means for generating the monitoring event salience information situation (for example, a rotating light that generates light, a speaker that emits sound, a rod-shaped part that comes into contact with the chicken coop as it moves and produces vibrations and sound, a signal output unit that starts feeding, a control signal output unit that controls the output of lighting and sound, etc.).

Figure 11 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of the present device equipped with a monitoring event conspicuous situation generating means. As shown in the figure, the mobile monitoring device MM4 has, as a monitoring event conspicuous situation generating means, rotating lights PL1, PL2 (rotating lights) that emit light (preferably light, color, or wavelength that activates the chickens without stressing them) only in the area to be monitored to stimulate the chickens, and chicken coop striking rods BR1, BR2 that emit vibrations and sounds only in the area to be monitored to stimulate the chickens. It is preferable that the vibrations and sounds at this time are as close as possible to the sound of feeding and the vibrations of feeding. This is because healthy chickens will always react to the sound and vibrations of feeding and become active. In addition, a speaker that reproduces and plays the sound of feeding is suitable as a monitoring event conspicuous situation generating means. Alternatively, a means for emitting the smell of food or the smell of males (pheromones) is also effective as a monitoring event conspicuous situation generating means.

Figure 12 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of this device when applied to a chicken coop. As shown in the figure, the line reading sensor LSN of the mobile monitoring device MM5 reads the guideline GL that is installed, laid or painted on the floor FL of the building, and the device automatically moves along this guideline GL. The mobile monitoring device MM5 recognizes its own position using the guideline (preferably in combination with a rangefinder or gyro sensor, etc.) and controls the movement (drive) of its own device along the guideline GL. The guideline may not only be used for guidance, but may also incorporate a position indicator. Also, the position may be measured and estimated using a rangefinder, etc., without using a position indicator.

Figure 13 is a schematic diagram explaining the monitoring technique of the mobile monitoring device of the present device when applied to a chicken coop. As shown in the figure, the line reading sensor LSN installed on the top of the mobile monitoring device MM6 reads the guideline GL installed, laid or painted on the ceiling SL of the building, and automatically moves along this guideline GL. The mobile monitoring device MM5 controls the movement (drive) of its own device along the guideline GL while recognizing its own device's position by the guideline (preferably using a rangefinder, accelerometer, and/or gyro sensor in combination). The guideline may be one that not only guides but also integrates a position indicator. Also, the position may be measured and estimated by a rangefinder or the like without using a position indicator. The advantage of this embodiment is that it can prevent the guideline installed on the floor from being read incorrectly when it is soiled with chicken droppings or feathers in a chicken coop, etc. This is because there is almost no possibility of the ceiling being contaminated by chicken droppings or feathers.

The above-mentioned monitoring technique uses image information as a monitoring method, but a monitoring technique using a distance sensor will be explained in the following embodiment 3.

<Embodiment 2>
Fig. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, the monitoring device MOS1 includes mobile monitoring devices MM1 and MM2 and a server SV2. The server SV2 has a control unit (CPU, arithmetic processing unit, processor) CON, an input unit IN, an output unit OUT, a communication unit COM, a memory unit MEM, and a display unit DIS. Unless otherwise specified, the mobile monitoring device MM2 has the same configuration and function as the mobile monitoring device MM1, but for convenience of drawing and explanation, some components are omitted.

In addition, the server SV2
an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken;
A memory unit MEM for storing the acquired image;
A generation unit GEN that labels specific parts of each individual in the stored images and generates an AI model AIM (stored in the storage unit as an AI algorithm and various derived data) trained using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification unit IDP that uses the AI model to identify specific parts in the stored image (such as the comb and wattles in the case of chickens);
A warning unit WAN that uses the AI model to determine whether the identified specific part is abnormal or not, and if it is abnormal, outputs (transmits) warning information including location information of the individual with the specific part and an image including the specific part;
Includes.

The present invention can also be implemented in a form in which the motion of an individual is identified and an abnormality is determined.
The server SV2 is
A generation unit GEN that labels the actions of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification unit IDP that uses an AI model to identify the behavior of an individual in the stored video;
A warning unit WAN that uses an AI model to determine whether the behavior of the identified individual is abnormal, and if abnormal, outputs warning information including location information of the individual and an image including the individual;
Includes.

The mobile monitoring device MM1 has cameras CM1-3 as an imaging unit, a sensor SNSR (barometer, gyro sensor, etc.), a control unit CON (CPU, arithmetic processing unit, processor), a communication unit COM, a monitoring unit MON (this function may be provided on the server side), an output unit OUT, a memory unit STR, and a GPS unit GR, but may also have an input unit (not shown) and a display unit (not shown). It is preferable to have two cameras to photograph the monitoring target such as a chicken coop and one to photograph a position indicator (such as a point-shaped marker or a line-shaped indicator) installed on the ceiling or floor, but one or two cameras may serve both functions. It is also preferable to use a high-performance model that can take pictures even at night with little environmental light. Furthermore, the camera may be provided with a separate infrared detector or the above-mentioned camera may be equipped with an infrared function to detect the temperature of the monitoring target and use it as a coloring material to determine the condition of the target, for example, whether it is dead or sick.

The mobile monitoring device MM1 has, as a sensor SNSR, a position measurement unit (not shown, which may be a GPS unit) that measures the position of the device itself, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, an altitude sensor that measures altitude, and a rangefinder that measures or estimates the distance traveled by the device itself.

The mobile monitoring device MM1 includes:
An imaging unit CM1-3 that captures an image;
A driving unit DR that moves the device itself;
an acquisition unit RET (such as a GPS unit) for acquiring the location of the mobile monitoring device;
a communication unit COM for transmitting monitoring information, including a position of the mobile monitoring device and an image at the position, to the server via a wireless or wired communication means;
has.

The server SV1 is
The server,
A communication unit COM (and an acquisition unit ACQ) that receives (acquires) the monitoring information from the mobile monitoring device, the monitoring information including a position of the mobile monitoring device and an image at the position;
a memory unit MEM for storing the monitoring information MI including the position of the mobile monitoring device and an image at that position, and standard specific part information SPI which is standard physical information of a specific part of an individual (in the case of a chicken, color information, dimensional information, shape information, etc. of a standard healthy chicken's comb, wattles, etc.);
An identification unit IDP for identifying a specific part (such as a comb or wattles in the case of a chicken) in the stored image;
a warning unit WAN which compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
has.

The output alarm is preferably an email, an audio warning, an image, a warning to a security guard, a warning to an administrator, or a report to a security company or the police. The monitoring unit preferably monitors whether a set monitoring event has occurred by using AI techniques and referring to the image information IM in the storage unit (or information derived from the image information (such as a characteristic pattern of the image)).

The mobile monitoring device MM1,
The guide portion GD is movably supported, placed or suspended on the guide rail and guided along the guide rail;
The drive unit DR is
The device itself is moved along the guide rail while being supported, placed or suspended on the guide rail.

CAD information on the building, shelves, chicken coop, etc. to be monitored may be acquired from a building information server or memory unit MEM connected via the network NET. A movement route is set from the CAD information, position index placement information that corresponds to or is made to correspond to the spatial coordinate system of this CAD information, and layout information on the objects to be monitored that are placed in the building (chicken coop cages, desks, chairs, equipment, etc.). The display unit DIS can display information stored in the device and information generated.

It is preferable that each functional unit included in the control unit of server SV1 is realized by a program module loaded into the memory space of the control unit. Furthermore, if the mobile device is realized as a drone or a self-propelled device, each functional unit can be realized by a program module loaded into the memory space of the control unit, or each functional unit can be realized by a built-in GPS unit or camera unit. Typically, the server downloads software from a storage unit or a website that causes the processor to function as each unit of this device, and installs and starts the software on the PC, causing the PC to operate as this device. Note that each functional unit provided in the control unit is merely a collection of steps that have a certain degree of functionality, and it is possible to combine multiple functional units into one functional unit, incorporate part of it into another functional unit, or split it into other independent functional units.

Figure 15 is a flow chart showing an example of processing executed by the monitoring device shown in Figure 1. The monitoring device performs processing in cooperation with two devices, a server SV1 and a mobile monitoring device MM1. If information and functions available on other servers are installed in the mobile device, it is possible to eliminate the need for cooperation with the server.

The generation unit first labels specific parts of each individual in the stored images, generates an AI model trained using an AI technique (e.g., techniques such as reinforcement learning or deep learning), and stores the model in the storage unit. Then, in step S21, the camera of the mobile monitoring device captures an image. Next, in step S22, the acquisition unit acquires the position of the mobile monitoring device. Then, in step S23, the monitoring information including the position of the mobile monitoring device and the image at that position is transmitted to the server via wireless or wired communication means. In step S24, the server moves the mobile monitoring device along the guide rail while the driving unit of the mobile monitoring device confirms the position using a position indicator and/or a gyro sensor and an acceleration sensor (or GPS, rangefinder). Note that the movement may be a simple method such as supplying motor power to move a predetermined distance (e.g., the distance of one chicken cage) without using a sensor, or operating the motor for a predetermined time.

In step S25, the camera of the mobile monitoring device captures an image. Next, the position of the mobile monitoring device is obtained, and in step S26, monitoring information including the position of the mobile monitoring device and an image at that position is sent to a server via wireless or wired communication means. In step S27, it is determined whether or not the end of the rail has been reached, or whether or not the remaining power of the battery installed in the mobile monitoring device is insufficient. If the determination conditions are met, the process proceeds to step S29-2 and the device returns to the base. If the determination conditions are not met, the process proceeds to step S28.

In step S28, it is determined whether an abnormal event (monitoring event) has occurred in the image. That is, in the stored image, the AI model is used to identify specific parts in the image (in the case of a chicken, the comb, wattles, etc.), and the warning unit uses the AI model to determine whether the identified specific part is abnormal, and if it is abnormal, it outputs (transmits) warning information including location information of the individual with the specific part and an image including the specific part.

If no abnormal event has occurred, the process returns to step S25 and the movement and imaging are continued. If an abnormal event has occurred, an alarm may be issued, but in order to reduce false recognition, it is preferable to proceed to step S28-1, stop or slow down the mobile monitoring device, and continue observing the image of the abnormal event. Then, if it is determined in step S28-2 that the occurrence of the abnormal event is continuing, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S29. If it is determined in step S28-2 that the occurrence of the abnormal event is not continuing, it is determined that the abnormal event is a false recognition, and no alarm is issued, but the process returns to step S25 and the movement (monitoring) is continued. After issuing the alarm in step S29, the process proceeds to step S29-1, and, as in step S27, it is determined whether the end of the rail (route) has been reached or whether the remaining power of the onboard battery is insufficient. If the judgment condition is met, the process returns to the base (charging spot) of the mobile monitoring device. If the judgment condition is not met, the process returns to step S25 and the movement (monitoring) is continued.

In this example, the judgment is made based on an image of a specific part of the body, but it may also be made based on the movement of the individual. In that case, the configuration is as follows.

In addition, the monitoring device (livestock barn monitoring device)
an image acquisition unit that acquires an image of an individual including location information of the location where the image was taken;
A storage unit for storing the acquired video;
A generation unit that labels the actions of each individual in the stored video and generates a trained AI model using an AI technique (e.g., a technique such as reinforcement learning or deep learning);
An identification unit that identifies the behavior of an individual in the stored video using an AI model;
a warning unit that uses an AI model to determine whether or not the behavior of the identified individual is abnormal, and if the behavior is abnormal, outputs warning information including location information of the individual and an image including the individual;
The present invention is characterized in that it includes:
Preferably, among the stored images, images taken within a predetermined period (preferably, about the past 1-3 weeks) are weighted to generate an AI model. Since the shape, size, color, etc. of each part of livestock change as they grow, this configuration makes it possible to reflect the standard condition of the livestock in its current state, making it possible to more appropriately identify abnormal individuals. In addition, by acquiring growth information of individuals in the barn (e.g., 1 week after birth, 2 months after birth, etc.) and customizing the AI model based on this, it becomes possible to more accurately identify, specify, and monitor sick, weakened, and dead livestock.

Although the present invention has been described in terms of capturing an image of an individual, it is also possible to identify and pinpoint abnormalities in an individual using not only images and videos, but also other measurement information that reflects the condition of the individual or the livestock facility.

For example, the monitoring device (livestock barn monitoring device)
a monitoring information acquisition unit that acquires location information of a monitored location and monitoring information including measurement values (such as sound, odor, temperature, and humidity) measured at the monitored location;
A storage unit for storing the acquired monitoring information;
A generator that labels the measurements of the stored monitoring information and generates a trained AI model using AI techniques (e.g., techniques such as reinforcement learning or deep learning);
an identification unit that identifies measurement values of the stored monitoring information using the AI model;
a warning unit that uses the AI model to determine whether the identified measurement value is abnormal, and if the identified measurement value is abnormal, outputs warning information including location information and the measurement information;
The present invention is characterized in that it includes:

Figure 18 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (AI model generation). As shown in the figure, an AI model AIM1 (which is numerical information on the color of the comb, preferably specified in RGB) and an AI model AIM2 (which is numerical information on the size of the wattle) are generated from image information SI3 showing chickens BD1-BD7, and stored in a memory unit. There are many methods for generating an AI model, for example, a specific part, the comb area, may be extracted, color information therein may be extracted, and a label indicating the health status of each individual may be added, but it is also possible to simply add a label indicating the health status of each individual, have the AI extract the specific parts and their features, and have the AI search and learn one specific part related to the health status (including illness and death), or a combination of two or more specific parts, to generate an AI model. The following figure shows how to use an AI model. In addition to specific body parts, the AI model can also use excrement, environmental temperature, humidity, odors, sounds, etc.

Figure 19 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (utilizing an AI model). As shown in the figure, the AI model AIM1 is compared with the color of the comb of chicken BD16. In this case, the color of the comb of chicken BD16 falls within the range of AI model AIM1, so it is determined that there is a match, and chicken BD16 is determined to be normal. Then, a different individual is compared.

On the other hand, the AI model AIM1 is compared with the color of the comb of chicken BD19. In this case, the color of the comb of chicken BD19 is outside the range of AI model AIM1, so it is determined that there is no match, i.e., an abnormality, and warning information including the position and image of individual BD19 is output.

Figure 20 is a schematic diagram of a monitoring technique of a monitoring device according to one embodiment of the present invention (application of AI model generation). This involves adding a label to each individual indicating whether the individual is healthy or not (for example, it is preferable to only add labels of unhealthy, dead, and sick. Those without a label can be automatically determined to be healthy), learning this, and automatically generating an AI model. It is also possible to automatically extract specific information of specific parts that indicate characteristics of abnormality, unhealthy, dead, and sickness using a known AI algorithm. For example, AI model AIM5 is the pattern of specific part 1, AI model AIM6 is the shape of specific part 2, and AI model AIM7 is the temperature of specific part 3, etc. For example, the temperature of a chicken's comb and wattle indicates the health state, and AIM model AIM7 is automatically generated even if the operator does not specify it. The advantage of this AI is that by simply attaching a label to a certain event (here, the unhealthy state of livestock), the AI can extract features that indicate symptoms of specific parts that would normally be overlooked (temperature, shape, color, pattern, etc.), and generate an AI model that uses them. This allows the AI model to identify features that indicate symptoms of specific parts (temperature, shape, color, pattern, etc.) even in the case of an unknown disease, and uses these to automatically identify and identify livestock in an unhealthy state. In addition, AI models may be generated for individual specific parts or functions, such as AI models AIM5 and AIM6, but a single AI model that includes all specific parts and functions may also be generated. With an AI model that includes everything, it is possible to estimate and identify health and abnormal conditions in more detail based on the correlation between the features of each specific part.

Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present invention. For example, the processes and functions included in each part and each step can be rearranged so as not to cause logical inconsistencies, and multiple means/parts and steps can be combined into one or divided. Alternatively, it should be noted that some of the components, functions, processes, steps, etc. of the device, method, program, etc. according to the present invention can be arranged on a remote server, etc. In addition, although the present invention has been described as a real-time monitoring device, it can also be used as an editing device or editing system that uses a captured image as input and extracts the time period or moment when a suspicious subject was captured.

ACQ...Acquisition unit AIM...Model BD11, BD12...Chicken BD13, BD14...Chicken BD4...Chicken BLD...Building BLD1...Building BR1, BR2...Chicken coop striking stick CAD...Dimension CG31...Chicken coop CG32...Chicken coop CG32...Chicken coop number CG42...Chicken coop number CI...Information CIALL...Information CM1...Camera CM1...Imaging unit CM2...Camera CM3...Camera CM4...Camera COM...Communication unit CON...Control unit DB...Drone base DIS...Display unit DR...Drive unit FL...Floor FR1...Flight route FR1, FR2...Flight route FR2...Flight route GD...Guide unit GEN...Generation unit GL...Guide line GR...Guide rail GR...Unit HDT4...Combined part IDP...Identification unit IM...Image information IN...Input unit LILI...Position index Layout information LSN...Line reading sensor MEM...Memory unit MI...Monitoring information MM1...Mobile monitoring device MM2...Mobile monitoring device MM3...Mobile monitoring device MM4...Mobile monitoring device MM5...Mobile monitoring device MM6...Mobile monitoring device MON...Monitoring unit MOS1...Monitoring device NET...Network OUT...Output unit PL1...Revolving light RET...Acquisition unit RT1...Movement route SBJ1, SBJ2...Monitoring targets SBJ1, SBJ2...Shelves SBJ3, SBJ4...Monitoring target SL...Ceiling SL1...Screen interface SL2...Screen interface SL3...Screen interface SLI...Layout information SNSR...Sensor SPI...Standard specific part information STR...Memory unit SV1...Server SV2...Server WAN...Warning unit WRN1...Alarm information WRN2...Alarm information

Claims (14)

A monitoring device, comprising:
An image acquisition unit that acquires an image including location information of the location where the image was taken;
a storage unit for storing the acquired image and standard specific part information, which is standard physical information of a specific part of an individual;
an identification unit for identifying a specific portion within the stored image;
a warning unit which compares the identified specific part with standard specific part information stored in the storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
The standard specific portion information is standard dimensional information or shape information of the specific portion,
the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit,
A monitoring device, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. A monitoring device, comprising:
an image acquisition unit that acquires an image including location information of the location where the image was taken;
a storage unit for storing the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual;
an identification unit for identifying a specific portion within the stored image;
a warning unit which compares the identified specific part with abnormal specific part information stored in the storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the location of the individual in the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
the abnormal specific part information is standard dimensional information or shape information of the specific part,
the abnormality specific part information is generated based on images of the individual for a predetermined period stored in the storage unit,
A monitoring device, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. A monitoring device, comprising:
an image acquisition unit that acquires an image of an individual, the image including location information of the location where the image was taken;
A storage unit for storing the acquired image;
A generating unit that generates standard specific part information, which is standard physical information of at least one specific part of an individual, based on each individual in the stored images;
an identification unit for identifying a specific portion within the stored image;
a warning unit which compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information differ, outputs warning information including location information of the individual with the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
The standard specific portion information is standard dimensional information or shape information of the specific portion,
the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit,
A monitoring device, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. 4. The monitoring device according to claim 3,
The generation unit,
generating standard specific part information, which is standard physical information of at least one specific part of each individual, based on only images of each individual within a predetermined period of time among the stored images;
A monitoring device comprising: 5. The monitoring device according to claim 3,
The generating unit, the identifying unit, or the warning unit,
It operates based on AI techniques,
A monitoring device comprising: A monitoring program that causes one or more processors to function as a monitoring device according to any one of claims 1 to 5. A computer-readable storage medium storing the monitoring program according to claim 6. 1. A monitoring method comprising:
An image acquisition step in which a computing device acquires an image including location information of the location where the image was taken;
A storage step in which the computing device stores the acquired image and standard specific part information, which is standard physical information of a specific part of an individual;
an identification step in which the computing device identifies a specific portion in the stored image;
a warning step in which the computing device compares the identified specific part with standard specific part information stored in a storage unit, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual with the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
The standard specific portion information is standard dimensional information or shape information of the specific portion,
the standard specific part information is generated based on images of the individual for a predetermined period stored in the storage unit,
A monitoring method, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. 1. A monitoring method comprising:
An image acquisition step in which a computing device acquires an image including location information of a place where the image was taken;
a storage step in which the computing device stores the acquired image and abnormal specific part information, which is physical information of a specific part of an abnormal individual;
an identification step in which the computing device identifies a specific portion in the stored image;
a warning step in which the computing device compares the identified specific part with abnormal specific part information stored in a storage unit, and if the specific part and the abnormal specific part information are within a predetermined numerical value, or if the specific part and the abnormal specific part information match, outputs warning information including position information of the individual in the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
the abnormal specific part information is standard dimensional information or shape information of the specific part,
the abnormality specific part information is generated based on images of the individual for a predetermined period stored in the storage unit,
A monitoring method, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. 1. A monitoring method comprising:
An image acquisition step in which a computing device acquires an image of an individual, the image including location information of the location where the image was taken;
A storage step in which the computing device stores the acquired image;
A generating step in which the computing device generates standard specific part information, which is standard physical information of at least one specific part of the individual, based on each individual of the stored images;
an identification step in which the computing device identifies a specific portion in the stored image;
a warning step in which the computing device compares the identified specific part with the generated standard specific part information, and if the specific part and the standard specific part information are different by a predetermined value or more, or if the specific part and the standard specific part information are different, outputs warning information including position information of the individual of the specific part and an image including the specific part;
Including,
The specific site is a chicken's comb or wattle,
The standard specific portion information is standard dimensional information or shape information of the specific portion,
the standard specific part information is generated based on images of the individual for a predetermined period stored in a storage unit;
A monitoring method, wherein the predetermined period is a period during which changes in the specific part associated with the growth of each of the individuals can be grasped from the image. The monitoring method according to claim 10,
The generating step includes:
generating standard specific part information, which is standard physical information of at least one specific part of each individual, based on only images of each individual within a predetermined period of time among the stored images;
A monitoring method comprising: The monitoring method according to claim 10 or 11,
The generating step, the identifying step, or the warning step,
It operates based on AI techniques,
A monitoring method comprising: A monitoring program that causes one or more processors to function as a monitoring device according to any one of claims 8 to 12. A computer-readable storage medium storing the monitoring program according to claim 13.

Images