JP7300958B2 - IMAGING DEVICE, CONTROL METHOD, AND COMPUTER PROGRAM - Google Patents

Description

The present invention relates to an imaging device, a learning device, a control method, a learning method, and a computer program.

BACKGROUND ART There is known a trail camera that is used for ecological research of animals and countermeasures against wild animals (see, for example, Non-Patent Document 1). A trail camera has an infrared sensor that reacts to moving objects. The trail camera takes pictures when an animal approaches 5m to 10m from the camera. A trail camera stores photographed data obtained by photographing in a memory such as an SD memory card or an internal storage. By installing a trail camera fixedly outdoors, it is possible to constantly monitor animals. When the trail camera is fixedly installed outdoors or the like, it automatically switches to an infrared photographing mode using an infrared LED at night. Many trail cameras use dry batteries, and the batteries are replaced every several weeks to several months. In recent years, there are products that sequentially transmit shooting data to the cloud using a mobile network or the like, enabling real-time remote monitoring.

Roland Kays, Bart Kranstauber, Patrick A.; Jansen, Chris Carbone, Marcus Rowcliffe, Tony Fountain, and Sameer Tilak. "Camera traps as sensor networks for monitoring animal communities", The 34th IEEE Conference on Local Computer Networks, 2009.

With the above-described technology, the images and videos actually sent from the trail camera to the cloud often include image data of animals other than those to be monitored, and image data that does not include animals. Image data of animals other than those to be monitored and image data in which animals are not captured are often generated due to erroneous detection by the infrared sensor. Since trail cameras may send image data of animals other than those to be monitored to the cloud, or image data that does not contain animals, it is necessary for the observer to visually identify the images and videos sent from the trail cameras to the cloud. do.
In addition, unnecessary data transmission occurs when image data other than those to be monitored is transmitted from the trail camera to the cloud. In addition, since image data other than those to be monitored is stored in the SD memory card or internal storage mounted on the trail camera, the capacity of the SD memory card or internal storage becomes tight.
The present invention has been made in consideration of such circumstances, and its object is to reduce transmission of unnecessary image data and storage of unnecessary image data among image data obtained by imaging. An object of the present invention is to provide an imaging device, a learning device, a control method, a learning method, and a computer program.

(1) One aspect of the present invention is a relationship between an imaging unit that generates a still image by capturing a moving object when a moving object is detected, an image of the animal, and information indicating the type of the animal captured in the image. a recognition unit that recognizes the type of animal appearing in a recognition image that is a recognition target image obtained by imaging by the imaging unit, using a learning model that has learned the recognition performed by the recognition unit; a transmission determination unit that determines whether or not to transmit the image data of the recognition image based on the recognition result of the animal appearing in the image; and storage of the image data of the recognition image based on the recognition result. The recognition unit uses a learning model that has learned the relationship between the image of the animal and the information indicating the number of animals in the image to perform the The number of animals appearing in the recognition image is recognized, and the memory determination unit stores information indicating the number of animals based on the recognition result of the number of animals appearing in the recognition image performed by the recognition unit. If the information indicating the number of animals to be observed matches the information indicating the number of animals to be observed, it is determined that the image data is to be stored, and the information indicating the number of animals is the information indicating the number of animals to be observed that is specified in advance. If they do not match, it is determined that the image data is not stored, and the transmission determination unit indicates the number of animals based on the recognition result of the number of animals appearing in the recognition image performed by the recognition unit. When the information matches the information indicating the number of observation target animals specified in advance, it is determined that the image data is to be transmitted, and the information indicating the number of animals indicates the number of animals specified in advance to be observed. The imaging device determines not to transmit the image data when the information does not match .
(2 ) In one aspect of the present invention, in the imaging device described in (1) above, the recognition unit learns a relationship between an image of an animal and information indicating the state of the animal captured in the image. Using a model, the state of the animal appearing in the recognition image is recognized, and the transmission determination unit determines the state of the animal appearing in the recognition image performed by the recognition unit, based on the recognition result of the state of the animal appearing in the recognition image. It is determined whether or not to transmit the image data of the recognition image, and the storage determination unit determines whether or not to store the image data of the recognition image based on the recognition result.
( 3 ) In one aspect of the present invention, in the imaging device according to (1) or ( 2) above, the image of the animal includes an image of the animal captured with visible light and an image of the animal captured with infrared light. and are included.
( 4 ) In one aspect of the present invention, in the imaging device according to any one of (1) to ( 3 ) above, the communication unit receives the learning model transmitted by a learning device that generates the learning model. and the recognition unit uses the learning model received by the communication unit to recognize the type of animal appearing in the recognition image.

( 5 ) One aspect of the present invention uses a learning model that learns the relationship between an image of an animal and information indicating the type of animal captured in the image, and captures a still image when a moving object is detected. a step of recognizing the type of animal appearing in a recognition image, which is an image to be recognized, obtained by an imaging unit that generates an image ; determining whether or not to transmit the image data of the recognition image based on the recognition result of the animal; and determining whether or not to store the image data of the recognition image based on the recognition result. and in the recognizing step, using a learning model that has learned the relationship between an image of an animal and information indicating the number of animals appearing in the image, the number of animals appearing in the recognition image is In the step of recognizing the number and determining whether or not to transmit the image data, the number of animals is determined based on the recognition result of the number of animals appearing in the recognition image performed in the recognition step. When the information indicating the number of animals to be observed matches the information indicating the number of observation target animals specified in advance, it is determined that the image data is to be stored, and the information indicating the number of animals corresponds to the number of animals to be observed which is specified in advance. In the step of determining whether or not to store the image data of the recognition image if the image data does not match the information shown in the recognition step, Based on the result of recognition of the number of animals observed, if the information indicating the number of animals matches the information indicating the number of animals to be observed that has been designated in advance, it is determined that the image data is to be transmitted, and the number of animals is determined to be transmitted. does not match the information indicating the number of observation target animals specified in advance, the control method executed by the imaging device determines not to transmit the image data.

( 6 ) One aspect of the present invention uses a learning model in which a computer learns the relationship between an image of an animal and information indicating the type of animal captured in the image, and photographs a moving object when it detects it. a step of recognizing the type of animal appearing in a recognition image, which is an image to be recognized, obtained by imaging by an imaging unit that generates a still image; a step of determining whether or not to transmit the image data of the recognition image based on the recognition result of the animal stored in the device; and determining whether or not to store the image data of the recognition image based on the recognition result. and causing the computer to perform the step of determining, and in the step of recognizing, using a learning model that has learned the relationship between an image of an animal and information indicating the number of animals in the image, the recognition In the step of recognizing the number of animals appearing in the image and determining whether or not to transmit the image data, the number of animals appearing in the recognition image obtained in the recognizing step is recognized. Based on this, when the information indicating the number of animals matches the information indicating the number of animals to be observed that has been specified in advance, it is determined that the image data is to be stored, and the information indicating the number of animals is determined to be the observation target that has been specified in advance. If it does not match the information indicating the number of target animals, the step of determining not to store the image data and determining whether or not to store the image data of the recognition image is performed in the recognizing step. If the information indicating the number of animals matches the information indicating the number of animals to be observed, which is specified in advance, based on the recognition result of the number of animals appearing in the recognition image, the image data is transmitted. If the information indicating the number of animals does not match the information indicating the number of observation target animals specified in advance, it is determined that the image data is not to be transmitted.

According to the present invention, an imaging device, a learning device, a control method, a learning method, and a computer program that can reduce transmission of unnecessary image data and storage of unnecessary image data among image data obtained by imaging. can provide.

1 is a diagram showing a configuration example of an imaging system according to a first embodiment of the present invention; FIG. It is a figure which shows an example of observation target image data. It is a figure which shows an example of a still image. It is a figure which shows an example of image data DB. 4 is a sequence chart showing an example of the operation of the imaging system according to the first embodiment; It is a figure which shows an example of a still image. It is a figure which shows an example of a still image. It is a figure which shows the structural example of the imaging system which concerns on the 2nd Embodiment of this invention. FIG. 10 is a schematic diagram showing an example of an imaging system according to a second embodiment; FIG. 9 is a sequence chart showing an example of the operation of the imaging system according to the second embodiment;

Next, an imaging device, a learning device, a control method, a learning method, and a computer program according to this embodiment will be described with reference to the drawings. The embodiments described below are merely examples, and embodiments to which the present invention is applied are not limited to the following embodiments.
In addition, in all the drawings for explaining the embodiments, the same reference numerals are used for the parts having the same functions, and repeated explanations are omitted.
In addition, "based on XX" in the present application means "based on at least XX", and includes cases based on other elements in addition to XX. Moreover, "based on XX" is not limited to the case of using XX directly, but also includes the case of being based on what has been calculated or processed with respect to XX. "XX" is an arbitrary element (for example, arbitrary information).

(First embodiment)
(imaging system)
FIG. 1 is a diagram showing a configuration example of an imaging system according to the first embodiment of the present invention. In FIG. 1 , an imaging system 100 includes an imaging device 1 and a server 2 . FIG. 1 shows an imaging device 1, a server 2, a user U who acquires image data, and a terminal device 3 used by the user U. As shown in FIG.
The imaging device 1, the server 2, and the terminal device 3 communicate via the network NW. The network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a provider device, a radio base station, and the like.

The imaging device 1 generates a still image by photographing when detecting a moving object. The imaging device 1 uses the learning model to recognize the type of animal appearing in the generated still image. The imaging device 1 determines whether or not to transmit the image data of the still image to the server 2 based on the information indicating the recognized animal type. The imaging device 1 determines to transmit the image data to the server 2 when the information indicating the recognized type of animal matches the information indicating the type of observation target animal specified in advance. On the other hand, the imaging device 1 determines not to transmit the image data to the server 2 when the information indicating the type of the recognized animal does not match the information indicating the type of the animal to be observed which is specified in advance. When the imaging device 1 determines to transmit the image data to the server 2, the imaging device 1 includes the imaging device ID, the image data, the image data ID of the image data, and the imaging date and time information when the image data was generated. Image data notification information is created, and the created image data notification information is transmitted to the server 2 .
The server 2 receives the image data notification information transmitted by the imaging device 1, and the imaging device ID of the imaging device 1 that transmitted the image data notification information included in the received image data notification information, the image data, and the image. The image data ID of the data and the imaging date/time information when the image data was generated are acquired, and the acquired imaging device ID, the image data, the image data ID, and the imaging date/time information are associated and stored.

The imaging device 1 determines whether or not to store the image data in the storage unit 16 based on the information indicating the recognized animal type. The imaging device 1 determines to store the image data when the information indicating the type of the recognized animal matches the information indicating the type of the observation target animal specified in advance. On the other hand, if the information indicating the type of the recognized animal does not match the information indicating the type of the animal to be observed that has been designated in advance, the imaging device 1 determines not to store the image data. When the imaging apparatus 1 determines to store the image data, the imaging apparatus 1 stores the image data in association with the image data ID of the image data and the shooting date and time information when the image data was generated.

When the user performs an operation requesting image data to the terminal device 3, the terminal device 3 includes the imaging device ID of the imaging device that has transmitted the image data and information requesting the image data. Create an image data request destined for 2. The terminal device 3 transmits the created image data request to the server 2 .
The server 2 receives the image data request transmitted by the terminal device 3 and acquires the imaging device ID included in the received image data request. The server 2 acquires the image data stored in association with the acquired imaging device ID, and creates an image data response including the acquired image data and destined for the terminal device 3 that transmitted the image data request. The server 2 transmits the created image data response to the terminal device 3 .
The terminal device 3 receives the image data response transmitted by the server 2 and acquires the image data included in the received image data response. The terminal device 3 outputs by processing the acquired image data.
Each of the imaging device 1, the server 2, and the terminal device 3 included in the imaging system 100 will be sequentially described below.

(Imaging device 1)
An example of the imaging device 1 is a trail camera. Trail cameras are used for ecological surveys of animals and countermeasures against wild animals. Hereinafter, the description will be continued assuming that the imaging device 1 is a trail camera.
The imaging device 1 includes, for example, an imaging unit 11, a recognition unit 12, a transmission determination unit 13, a communication unit 14, a storage determination unit 15, a storage unit 16, a transmission control unit 17, and a power supply unit PSU. Prepare.

When an infrared sensor (not shown) attached to the imaging device 1 detects a moving object, the imaging unit 11 shoots to generate a still image (recognition image) that is an image to be recognized.
The storage unit 16 is implemented by a HDD (Hard Disk Drive), flash memory, RAM (Random Access Memory), ROM (Read Only Memory), or the like. The storage unit 16 stores observation target image data 16a and a learning model 16b. The observation target image data 16a and the learning model 16b may be stored on the cloud.
The observation target image data 16a is information in a table format in which an image data ID, image data, and imaging date/time information are associated with each other.
FIG. 2 is a diagram showing an example of observation target image data. An example of the observation target image data 16a includes an image data ID "****", image data "****", and imaging date and time information "****/****/****/****:* *” are associated. In addition, an example of the observation target image data 16a is associated with an image data ID “+++”, image data “++++”, and imaging date/time information “++++/++/++/++:++”. Returning to FIG. 1, the description continues.

The recognition unit 12, the transmission determination unit 13, the storage determination unit 15, and the transmission control unit 17 are configured such that a hardware processor such as a CPU (Central Processing Unit) executes a program (software) stored in the storage unit 16. It is realized by Some or all of these functional units are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit) (including circuitry), or by cooperation between software and hardware. The program may be stored in advance in a storage device such as a HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium such as a DVD or CD-ROM. It may be installed by being worn.

The recognition unit 12 acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12 uses the learning model 16b stored in the storage unit 16 to recognize the type of animal appearing in the acquired still image. Recognition unit 12 creates recognition result information including information indicating the type of recognized animal, and outputs the created recognition result information to transmission determination unit 13 and storage determination unit 15 together with image data. Here, an example of the learning model 16b is generated by learning the relationship between an image of an animal and information indicating the type of animal appearing in the image. Here, the learning model 16b is created in advance. The recognition unit 12 uses object recognition technology such as, for example, Faster R-CNN (Regions with Convolutional Neural Networks), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) to recognize the type of animal. .
As the learning model 16b, a first learning model generated by learning the relationship between an image of an animal photographed with visible light and information indicating the type of animal in the image, and an animal photographed with infrared light. A second learning model may be prepared that is generated by learning a relationship between an image and information indicating the type of animal depicted in the image. By configuring in this way, regardless of day or night, the type of animal appearing in the still image generated by imaging by the imaging unit 11 can be recognized.

FIG. 3 is a diagram showing an example of a still image. FIG. 3 shows an example of a still image I1 generated by the imaging unit 11 capturing an image. If the imaging unit 11 generates the still image I1 shown in FIG. 3, the recognition unit 12 generates recognition result information including information indicating that the raccoon dog has been recognized. Returning to FIG. 1, the description continues.
The transmission determination unit 13 acquires the recognition result information and the image data output by the recognition unit 12, and transmits the image data to the server 2 based on the information indicating the type of recognized animal included in the acquired recognition result information. Determine whether or not to transmit. For example, when the information indicating the type of recognized animal included in the recognition result information matches the information indicating the type of animal to be observed that has been designated in advance, the transmission determination unit 13 sends the image data to the server 2. Determine to send. On the other hand, if the information indicating the type of recognized animal included in the recognition result information does not match the information indicating the type of animal to be observed that has been designated in advance, the transmission determination unit 13 sends the image data to the server 2. Determine not to send. The transmission determination unit 13 outputs the image data to the transmission control unit 17 when determining to transmit the image data to the server 2 .

The storage determination unit 15 acquires the recognition result information and the image data output by the recognition unit 12, and stores the image data in the storage unit 16 based on the information indicating the type of recognized animal included in the acquired recognition result information. Determines whether or not to store (record) to The storage determination unit 15 is an example of a recording determination unit. For example, if the information indicating the type of recognized animal included in the recognition result information matches the information indicating the type of animal to be observed that has been designated in advance, the storage determination unit 15 stores the image data in the storage unit 16. It is determined to store to. On the other hand, if the information indicating the type of recognized animal included in the recognition result information does not match the information indicating the type of animal to be observed that has been specified in advance, the storage determination unit 15 stores the image data in the storage unit 16. It is determined not to be stored in the memory. When the storage determination unit 15 determines to store the image data in the storage unit 16, the storage determination unit 15 stores the image data in the observation target image data 16a of the storage unit 16 in association with the image data ID and the imaging date/time information.
The transmission control unit 17 acquires the image data output by the transmission determination unit 13, and sets the imaging device ID of the imaging device 1 that transmits the image data, the acquired image data, the image data ID of the image data, and the image data. Image data notification information including information on the date and time when the data was generated and addressed to the server 2 is created. The transmission control unit 17 outputs the created image data notification information to the communication unit 14 .

The communication unit 14 is implemented by a communication module. The communication unit 14 communicates with an external communication device via the network NW. The communication unit 14 may communicate using a communication method such as a wired LAN (Local Area Network). Further, the communication unit 14 may communicate by a communication method such as wireless LAN, Bluetooth (registered trademark), or LTE (registered trademark). The communication unit 14 holds communication information necessary for communicating with the server 2 via the network NW. The communication unit 14 acquires the image data notification information output by the transmission control unit 17 and transmits the acquired image data notification information to the server 2 .
The power supply unit PSU is a power supply for driving the imaging device 1 . Examples of power sources include external power sources such as outlets, dry batteries, solar panels, and secondary batteries.

(server 2)
The server 2 is implemented by a device such as a personal computer, server, smart phone, tablet computer, or industrial computer. The server 2 includes, for example, a communication unit 24, a storage unit 26, and a processing unit 28.

The storage unit 26 is implemented by an HDD, flash memory, RAM, ROM, or the like. The storage unit 26 stores an image data DB 26a.
The image data DB 26a is information in a table format that associates the imaging device ID of the imaging device 1 that has transmitted the image data notification information, the image data ID, the image data, and the imaging date and time information.
FIG. 4 is a diagram showing an example of an image data DB. An example of the image data DB 26a includes an imaging device ID "****", an image data ID "****", image data "****", and imaging date and time information "****/* */**/**:**” are associated. Further, in an example of the image data DB 26a, an imaging device ID "++++", an image data ID "+++", an image data "++++++", and imaging date/time information "++++/++/++/++:++" are associated. It is Returning to FIG. 1, the description continues.

The processing unit 28 is implemented, for example, by a hardware processor such as a CPU executing a program (software) stored in the storage unit 26 . Also, some or all of these functional units may be realized by hardware (including circuitry) such as LSI, ASIC, FPGA, and GPU, or by cooperation between software and hardware. may be implemented. The program may be stored in advance in a storage device such as an HDD or flash memory, or may be stored in a removable storage medium such as a DVD or CD-ROM. may be installed.
The processing unit 28 acquires the image data notification information received by the communication unit 24, and acquires the imaging device ID, the image data ID, the image data, and the imaging date and time information included in the acquired image data notification information. The processing unit 28 associates the acquired imaging device ID, image data ID, image data, and imaging date/time information, and stores them in the image data DB 26 a stored in the storage unit 26 .
The processing unit 28 acquires the image data request received by the communication unit 24 and acquires the imaging device ID included in the acquired image data request. The processing unit 28 acquires from the image data DB 26 a stored in the storage unit 26 the image data ID, the image data, and the imaging date and time information stored in association with the acquired imaging device ID. The processing unit 28 creates an image data response including the acquired image data ID, image data, and imaging date/time information and addressed to the terminal device 3 that transmitted the image data request. The processing unit 28 outputs the created image data response to the communication unit 24 .

The communication unit 24 is implemented by a communication module. The communication unit 24 communicates with an external communication device via the network NW. The communication unit 24 may communicate using a communication method such as a wired LAN, for example. Further, the communication unit 24 may communicate using a communication method such as wireless LAN, Bluetooth (registered trademark), or LTE (registered trademark). The communication unit 24 holds communication information necessary for communicating with the imaging device 1 via the network NW. The communication unit 24 receives the image data notification information transmitted by the imaging device 1 and outputs the received image data notification information to the processing unit 28 . The communication unit 24 receives the image data request transmitted by the terminal device 3 and outputs the received image data request to the processing unit 28 . The communication unit 24 acquires the image data response output by the processing unit 28 and transmits the acquired image data response to the terminal device 3 .

(Terminal device 3)
The terminal device 3 is implemented as a smart phone, a mobile terminal, a personal computer, a tablet terminal device, a wristwatch type terminal device, or other information processing equipment.
When the user performs an operation requesting image data to the terminal device 3, the terminal device 3 includes the imaging device ID of the imaging device 1 that has transmitted the image data and information requesting the image data. Create an image data request destined for 2. The terminal device 3 transmits the created image data request to the server 2 . The terminal device 3 receives the image data response transmitted by the server 2 and acquires the image data included in the received image data response. The terminal device 3 processes and outputs the acquired image data. For example, the terminal device 3 may display the processing result of the image data on a display unit (not shown).

(Operation of Imaging System)
FIG. 5 is a sequence chart showing an example of the operation of the imaging system according to the first embodiment.
(Step S1)
In the imaging device 1, the imaging unit 11 generates a still image by shooting when an infrared sensor (not shown) attached to the imaging device 1 detects a moving object.
(Step S2)
In the imaging device 1, the recognition unit 12 acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12 uses the learning model 16b to recognize the type of animal appearing in the acquired still image. Recognition unit 12 creates recognition result information including information indicating the type of recognized animal, and outputs the created recognition result information to transmission determination unit 13 and storage determination unit 15 together with image data.
(Step S3)
In the imaging device 1, the transmission determination unit 13 acquires the recognition result information and the image data output by the recognition unit 12, and based on the information indicating the type of the recognized animal included in the acquired recognition result information, the transmission determination unit 13 converts the image data into image data. to the server 2 or not. If it is determined not to transmit the image data to the server 2, the process proceeds to step S6.
(Step S4)
In the imaging device 1 , the transmission determination unit 13 outputs the image data to the transmission control unit 17 when it determines that the image data is to be transmitted to the server 2 . The transmission control unit 17 acquires the image data output by the transmission determination unit 13, and stores the imaging device ID, the acquired image data, the image data ID of the image data, and the imaging date and time information when the image data was generated. , and creates image data notification information addressed to the server 2 .
(Step S5)
In the imaging device 1 , the transmission determination unit 13 outputs the created image data notification information to the communication unit 14 . The communication unit 14 acquires the image data notification information output by the transmission determination unit 13 and transmits the acquired image data notification information to the server 2 .

(Step S6)
In the imaging device 1, the memory determination unit 15 acquires the recognition result information and the image data output by the recognition unit 12, and stores the image data based on the information indicating the type of recognized animal contained in the acquired recognition result information. is stored in the storage unit 16 or not. If it is determined not to store the image data in the storage unit 16, the process proceeds to step S1.
(Step S7)
In the imaging device 1, when the storage determination unit 15 determines to store the image data in the storage unit 16, the image data is stored in the storage unit 16 as the observation target image data 16a in association with the image data ID and the imaging date/time information. memorize to After that, the process moves to step S1.
(Step S8)
In the server 2 , the communication section 24 receives the image data notification information transmitted by the imaging device 1 and outputs the received image data notification information to the processing section 28 . The processing unit 28 acquires the image data notification information received by the communication unit 24, and acquires the imaging device ID, the image data ID, the image data, and the imaging date and time information included in the acquired image data notification information. The processing unit 28 associates the acquired imaging device ID, image data ID, image data, and imaging date/time information, and stores them in the image data DB 26 a stored in the storage unit 26 .
(Step S9)
When the user performs an operation to request image data from the terminal device 3, the terminal device 3 sends an image data request addressed to the server 2, including the imaging device ID and information requesting image data. to create
(Step S10)
The terminal device 3 transmits the created image data request to the server 2 .

(Step S11)
In the server 2 , the communication section 24 receives the image data request transmitted by the terminal device 3 and outputs the received image data request to the processing section 28 . The processing unit 28 acquires the image data request received by the communication unit 24 and acquires the imaging device ID included in the acquired image data request. The processing unit 28 acquires from the image data DB 26 a stored in the storage unit 26 the image data ID, the image data, and the imaging date and time information stored in association with the acquired imaging device ID. The processing unit 28 creates an image data response including the acquired image data ID, image data, and imaging date/time information and addressed to the terminal device 3 that transmitted the image data request. Here, if the processing unit 28 cannot acquire the image data ID, the image data, and the imaging date and time information stored in association with the acquired imaging device ID from the image data DB 26a stored in the storage unit 26 create an image data response containing information indicating that there is no image data.
(Step S12)
In the server 2 , the processing section 28 outputs the created image data response to the communication section 24 . The communication unit 24 acquires the image data response output by the processing unit 28 and transmits the acquired image data response to the terminal device 3 .
(Step S13)
The terminal device 3 receives the image data response transmitted by the server 2 and acquires the image data included in the received image data response. The terminal device 3 processes and outputs the acquired image data. For example, the terminal device 3 may display the processing result of the image data on a display unit (not shown).

In the first embodiment described above, the imaging unit 11 of the imaging device 1 captures a still image when the infrared sensor (not shown) detects a moving object, but the present invention is not limited to this example. For example, the imaging unit 11 may perform scheduled imaging in which images are taken periodically.
In the above-described first embodiment, a case where a still image is generated by imaging by the imaging device 1 has been described, but the present invention is not limited to this example. For example, a moving image may be generated by imaging by the imaging device 1 .
In the first embodiment described above, the imaging device 1 determines whether or not to transmit the image data of the still image to the server 2 based on the recognition result of the animal appearing in the still image generated by the imaging unit 11. Although the case of determining whether or not to store the image data in the storage unit 16 has been described, the present invention is not limited to this example. For example, in the imaging device 1, based on the recognition result of the animal appearing in the still image generated by the imaging unit 11, it is determined whether or not to transmit the image data of the still image to the server 2; , and determining whether or not to store the information in the storage unit 16 .
Although the imaging system 100 includes one imaging device 1 in the first embodiment described above, the imaging system 100 is not limited to this example. For example, a plurality of imaging devices 1 may be used. With this configuration, the server 2 can store the image data notification information in the image data DB 26a based on the image data notification information transmitted by each of the plurality of imaging devices 1 . By transmitting the image data request, the terminal device 3 can acquire the image data notification information notified by one of the imaging devices 1 .

In the first embodiment described above, an image data request includes an imaging device ID, and the server 2 creates an image data response including image data notification information corresponding to the imaging device ID. It is not limited to this example. For example, the image data request may include imaging date and time information in addition to the imaging device ID. In this case, the server 2 creates an image data response including image data notification information corresponding to the combination of the imaging device ID and imaging date/time information. Also, instead of the imaging date/time information, a range of imaging dates/times may be included. In this case, the server 2 creates an image data response including image data notification information corresponding to the combination of the imaging device ID and the imaging date and time range.
In the above-described first embodiment, the case where there is one server 2 has been described, but the present invention is not limited to this example. For example, there may be multiple servers 2 . In this case, the server 2 that transmits the image data notification information may be changed based on the type of animal recognized by the imaging device 1 .
In the first embodiment described above, the imaging device 1 may delete information corresponding to the image data notification information notified to the server 2 from the information included in the observation target image data 16a. By configuring in this way, the storage capacity of the imaging device 1 can be ensured.
In the first embodiment described above, the imaging device 1 may include the recognition result of the type of animal appearing in the still image in the image data notification information. Specifically, the imaging device 1 creates a recognition result of an animal type such as a wild boar in a language format such as JSON (Javascript Object Notation) or XML (eXtensible Markup Language), and extracts the created animal type recognition result. may be included in the image data notification information.
In the first embodiment described above, when the imaging device 1 determines to store the image data in the storage unit 16, the image data and the animal type recognition result are stored in the observation target image data 16a in the storage unit 16. You may make it memorize|store. Specifically, the imaging device 1 creates a recognition result of an animal type such as a wild boar in a language format such as JSON or XML, and stores the created recognition result of the animal type as image data and an image data ID. , may be stored in the observation target image data 16a of the storage unit 16 in association with the imaging date and time information.
In the first embodiment described above, the transmission determination unit 13 of the imaging device 1 determines whether or not to transmit the image data to the server 2, and the storage determination unit 15 determines whether or not to store the image data. You may make it differ from the conditions which determine whether.

According to the imaging system 100 according to the first embodiment, the image data of the still image is transmitted based on the recognition result of the type of the animal in the still image generated by the imaging device 1 such as a trail camera. and storage of the image data, or both. Therefore, even if image data of a still image is generated by capturing an image of an object other than the observation target, either or both of transmission to the server 2 and storage in the imaging device 1 are prevented. can. Either one or both of unnecessary data transmission and data storage can be reduced, so either one or both of the reduction in communication charges and the reduction in frequency of SD memory card replacement can be realized, resulting in increased convenience. can improve.
Shooting is also performed when the infrared sensor reacts to an object other than the observation target. In this case, if the image data is saved each time as in the past, there is a risk that the capacity will be exceeded before the SD memory card is replaced and the image data is extracted from the main memory. . In the imaging apparatus 1 according to the first embodiment, the memory determination unit 15 controls storage of desired image data, thereby reducing the capacity of a storage medium such as an SD memory card.
For example, when the imaging system 100 is used as a countermeasure against crop damage caused by wild boars, wild boars are set as observation targets. Based on the information determined by the recognition unit 12, the transmission determination unit 13 determines whether to transmit the data in which the boar appears. By configuring in this way, communication charges can be reduced.

(Modification 1 of the first embodiment)
(imaging system)
FIG. 1 can be applied to the configuration example of the imaging system according to Modification 1 of the first embodiment. An imaging system 100 a according to Modification 1 of the first embodiment differs from the imaging system 100 according to the first embodiment in that an imaging device 1 a is provided instead of the imaging device 1 .
The imaging device 1a generates a still image by photographing when a moving object is detected. The imaging device 1a uses the learning model 16ba to recognize the number of animals appearing in the generated still image. The imaging device 1a determines whether or not to transmit the image data of the still image to the server 2 based on the information indicating the number of recognized animals. When the information indicating the number of recognized animals matches the information indicating the number of observation target animals specified in advance, the imaging device 1 a determines to transmit the image data to the server 2 . On the other hand, if the information indicating the number of recognized animals does not match the information indicating the number of observation target animals specified in advance, the imaging device 1 a determines not to transmit the image data to the server 2 . When the imaging device 1a determines to transmit the image data to the server 2, the imaging device ID of the imaging device 1a that transmits the image data, the image data, the image data ID of the image data, and the image data Image data notification information including the generated imaging date and time information is created, and the created image data notification information is transmitted to the server 2 .
The server 2 receives the image data notification information transmitted by the imaging device 1a, acquires the imaging device ID, the image data, the image data ID, and the imaging date and time information included in the received image data notification information, and acquires The captured imaging device ID, the image data, the image data ID, and the imaging date/time information are associated with each other and stored.

The imaging device 1a determines whether or not to store the image data in the storage unit 16 based on the information indicating the number of recognized animals. When the information indicating the number of recognized animals matches the information indicating the number of observation target animals specified in advance, the imaging device 1a determines to store the image data. On the other hand, if the information indicating the number of recognized animals does not match the information indicating the number of observation target animals specified in advance, the imaging device 1a determines not to store the image data.
When the imaging device 1a determines to store the image data, it stores the image data in association with the image data ID of the image data and the shooting date and time information when the image data was generated.
Among the imaging device 1a, the server 2, and the terminal device 3 included in the imaging system 100a, the imaging device 1a different from the first embodiment will be described below.

(Imaging device 1a)
An example of the imaging device 1a is a trail camera. Trail cameras are used for ecological surveys of animals and countermeasures against wild animals. Hereinafter, the description will be continued assuming that the imaging device 1a is a trail camera.
The imaging device 1a includes, for example, an imaging unit 11, a recognition unit 12a, a transmission determination unit 13a, a communication unit 14, a storage determination unit 15a, a storage unit 16, a transmission control unit 17, and a power supply unit PSU. Prepare. The storage unit 16 stores the learning model ba.
The recognition unit 12a acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12a uses the learning model 16ba to recognize the number of animals appearing in the obtained still image. The recognition unit 12 creates recognition result information including information indicating the number of recognized animals, and outputs the created recognition result information together with image data to the transmission determination unit 13a and the storage determination unit 15a. Here, an example of the learning model 16ba is generated by learning the relationship between an image of an animal and information indicating the number of animals in the image. Here, the learning model 16ba is created in advance. The recognition unit 12a recognizes the type of animal using object recognition technology such as Faster R-CNN, YOLO, and SSD.
As the learning model 16ba, a first learning model generated by learning the relationship between an image of an animal photographed with visible light and information indicating the type of animal in the image, and an animal photographed with infrared light. A second learning model may be prepared that is generated by learning a relationship between an image and information indicating the type of animal depicted in the image. By configuring in this way, the number of animals appearing in the still image generated by imaging by the imaging unit 11 can be recognized regardless of day or night.

FIG. 6 is a diagram showing an example of a still image. FIG. 3 shows an example of the still image I2 generated by the imaging unit 11 shooting. If the imaging unit 11 generates the still image I2 shown in FIG. 6, the recognition unit 12a generates recognition result information including information indicating that three animals have been recognized. Returning to FIG. 1, the description continues.
The transmission determination unit 13a acquires the recognition result information and the image data output by the recognition unit 12a, and transmits the image data to the server 2 based on the information indicating the number of recognized animals included in the acquired recognition result information. Determine whether or not to transmit. For example, if the information indicating the number of recognized animals included in the recognition result information matches the information indicating the number of animals to be observed that has been designated in advance, the transmission determination unit 13a sends the image data to the server 2. Determine to send. On the other hand, if the information indicating the number of recognized animals included in the recognition result information does not match the information indicating the number of animals to be observed that has been designated in advance, the transmission determination unit 13a sends the image data to the server 2. Determine not to send. The transmission determination unit 13 a outputs the image data to the transmission control unit 17 when it determines that the image data is to be transmitted to the server 2 .

The storage determination unit 15a acquires the recognition result information and the image data output by the recognition unit 12a, and stores the image data in the storage unit 16 based on the information indicating the number of recognized animals included in the acquired recognition result information. It is determined whether or not to store to. The storage determination unit 15a is an example of a recording determination unit. For example, when the information indicating the number of recognized animals included in the recognition result information matches the information indicating the number of animals to be observed that has been designated in advance, the storage determination unit 15a stores the image data in the storage unit 16. It is determined to store to. On the other hand, if the information indicating the number of recognized animals included in the recognition result information does not match the information indicating the number of animals to be observed that has been specified in advance, the storage determination unit 15a stores the image data in the storage unit 16. It is determined not to be stored in the memory. When the storage determination unit 15a determines to store the image data in the storage unit 16, the storage determination unit 15a stores the image data in the observation target image data 16a of the storage unit 16 in association with the image data ID and the imaging date/time information.

(Operation of Imaging System)
FIG. 5 can be applied to an example of the operation of the imaging system 100a according to Modification 1 of the first embodiment. However, the processing from step S2 to step S4 is different from that of the imaging system 100 according to the first embodiment.
In step S2, in the imaging device 1a, the recognition unit 12a acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12a uses the learning model 16ba to recognize the number of animals appearing in the obtained still image. The recognition unit 12a creates recognition result information including information indicating the number of recognized animals, and outputs the created recognition result information together with image data to the transmission determination unit 13a and the storage determination unit 15a.
In step S3, in the imaging device 1a, the transmission determination unit 13a acquires the recognition result information and the image data output by the recognition unit 12a, and based on the information indicating the number of recognized animals included in the acquired recognition result information, to determine whether or not to transmit the image data to the server 2 .
In step S4, in the imaging device 1a, the memory determination unit 15a acquires the recognition result information and the image data output by the recognition unit 12a, and based on the information indicating the number of recognized animals contained in the acquired recognition result information, to determine whether or not to store the image data in the storage unit 16 . When it is determined not to store in the storage unit 16, the process proceeds to step S1.

In the modified example 1 of the first embodiment described above, the imaging device 1a determines whether the information indicating the number of recognized animals included in the recognition result information matches the information indicating the number of animals to be observed as specified in advance. Although the case of determining whether or not has been described, the present invention is not limited to this example. For example, the imaging device 1a determines whether or not the information indicating the number of recognized animals included in the recognition result information is equal to or greater than the information indicating the number of animals to be observed that is specified in advance. may
In the modified example 1 of the first embodiment described above, the imaging device 1a may include the recognition result of the number of animals appearing in the still image in the image data notification information. Specifically, the imaging device 1a may create the recognition result of the number of animals in a language format such as JSON or XML, and include the created recognition result of the number of animals in the image data notification information.
In the first modified example of the first embodiment described above, when the imaging device 1a determines to store the image data in the storage unit 16, the image data, the image data ID, the imaging date and time information, and the number of animals are stored. The recognition result may be stored in the observation target image data 16 a of the storage unit 16 . Specifically, the imaging device 1a creates the recognition result of the number of animals in a language format such as JSON or XML, and converts the created recognition result of the number of animals into image data, an image data ID, and an imaging date and time. It may be stored in the observation target image data 16a of the storage unit 16 in association with the information.
The above-described first embodiment and modification 1 of the first embodiment may be combined. Specifically, the imaging device 1a may use the learning model 16b and the learning model 16ba to recognize the type and number of animals appearing in the obtained still image. In this case, the imaging device 1a determines whether or not the information indicating the type and number of recognized animals included in the recognition result information matches the information indicating the type and number of animals to be observed that has been designated in advance. You may
In this case, a plurality of imaging devices 1a are installed at multiple points, and the imaging devices 1a installed at each of the multiple points create recognition result notification information including the imaging device ID of the imaging device 1a and recognition result information. Then, the created recognition result notification information may be transmitted to the cloud. The cloud side stores imaging device position information that associates the identification information of each of the plurality of imaging devices 1a with the position information where the imaging device 1a is installed. On the cloud side, the recognition result notification information transmitted by each of the plurality of imaging devices 1a is received, and the identification information of the imaging device 1a included in the received recognition result notification information, the recognition result information, and the stored imaging device Based on the positional information, an animal appearance map is created at the position where the imaging device 1a is installed on the map, in which the species and positions of the animals recognized at that position are displayed. On the cloud side, the animal infestation map is updated at predetermined intervals such as one hour, several hours, or one day. Here, image data may be included in the recognition result notification information. On the cloud side, at the position where the imaging device 1a is installed on the map, in addition to the type and position of the animal recognized at that position, a static image obtained by processing the image data is displayed. You can create a map. By operating the terminal device 3, the user can refer to the animal appearance map created on the cloud side. If a bear is set as the type of animal, a bear infestation map is created on the cloud side. By operating the terminal device 3, the user can refer to the bear infestation map created on the cloud side. By notifying the user of the bear-infestation map, detailed information can be provided when informing the user of the danger of bear-infestation. In addition, the created animal infestation map can be used for ecological research of animals.

According to the imaging system 100a according to Modification 1 of the first embodiment, still images are generated based on the recognition result of the number of animals appearing in the still images generated by imaging by the imaging device 1a such as a trail camera. image data transmission and storage of the image data, or both. Therefore, even if image data of a still image is generated by imaging an object other than the number of observation targets, either one or both of transmission to the server 2 and storage in the imaging device 1a are performed. can be prevented. Either one or both of unnecessary data transmission and data storage can be reduced, so either one or both of the reduction in communication charges and the reduction in frequency of SD memory card replacement can be realized, resulting in increased convenience. can improve.

(Modification 2 of the first embodiment)
(imaging system)
FIG. 1 can be applied to the configuration example of the imaging system according to Modification 2 of the first embodiment. An imaging system 100b according to Modification 2 of the first embodiment differs from the imaging system 100 according to the first embodiment in that an imaging device 1b is provided instead of the imaging device 1. FIG.
The imaging device 1b generates a still image by photographing when a moving object is detected. The imaging device 1b uses the learning model 16bb to recognize the state of the animal appearing in the generated still image. The imaging device 1b determines whether or not to transmit image data of a still image to the server 2 based on the information indicating the recognized state of the animal. The imaging device 1 b determines to transmit the image data to the server 2 when the information indicating the state of the recognized animal matches the information indicating the state of the animal to be observed which is specified in advance. On the other hand, the imaging device 1 b determines not to transmit the image data to the server 2 when the information indicating the state of the recognized animal does not match the information indicating the state of the animal to be observed which is specified in advance. When the imaging device 1b determines to transmit the image data to the server 2, the imaging device 1b generates the imaging device ID of the imaging device that transmits the image data, the image data, the image data ID of the image data, and the image data. image data notification information including the captured image date and time information is created, and the created image data notification information is transmitted to the server 2 .
The server 2 receives the image data notification information transmitted by the imaging device 1b, acquires the imaging device ID, the image data, the image data ID, and the imaging date and time information included in the received image data notification information, and acquires The captured imaging device ID, the image data, the image data ID, and the imaging date/time information are associated with each other and stored.
Among the imaging device 1b, the server 2, and the terminal device 3 included in the imaging system 100b, the imaging device 1b different from the first embodiment will be described below.

(Imaging device 1b)
An example of the imaging device 1b is a trail camera. Trail cameras are used for ecological surveys of animals and countermeasures against wild animals. Hereinafter, the description will be continued assuming that the imaging device 1b is a trail camera.
The imaging device 1b includes, for example, an imaging unit 11, a recognition unit 12b, a transmission determination unit 13b, a communication unit 14, a storage determination unit 15b, a storage unit 16, a transmission control unit 17, and a power supply unit PSU. Prepare. The storage unit 16 stores a learning model 16bb.
The recognition unit 12b acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12b uses the learning model 16bb to recognize the state of the animal appearing in the acquired still image. The recognition unit 12 creates recognition result information including information indicating the recognized state of the animal, and outputs the created recognition result information together with the image data to the transmission determination unit 13b and the storage determination unit 15b. Here, an example of the learning model 16bb is generated by learning the relationship between an image of an animal and information indicating the state of the animal shown in the image. Here, the learning model 16bb is created in advance. The recognition unit 12b recognizes the type of animal using object recognition technology such as Faster R-CNN, YOLO, and SSD.
As the learning model 16bb, a first learning model generated by learning the relationship between an image of an animal photographed with visible light and information indicating the type of animal in the image, and an animal photographed with infrared light. A second learning model may be prepared that is generated by learning a relationship between an image and information indicating the type of animal depicted in the image. By configuring in this way, regardless of day or night, the state of the animal captured in the still image generated by the imaging unit 11 can be recognized.

FIG. 7 is a diagram showing an example of a still image. FIG. 7 shows an example of a still image I3 generated by the imaging unit 11 capturing an image. If the imaging unit 11 generates the still image I3 shown in FIG. 7, the recognizing unit 12b generates recognition result information including information indicating that the trapped animal has been recognized. Returning to FIG. 1, the description continues.
The transmission determination unit 13b acquires the recognition result information and the image data output by the recognition unit 12b, and transmits the image data to the server 2 based on the information indicating the state of the recognized animal contained in the acquired recognition result information. Determine whether or not to transmit. For example, when the information indicating the state of the recognized animal included in the recognition result information matches the information indicating the state of the animal to be observed that has been designated in advance, the transmission determination unit 13b sends the image data to the server 2. Determine to send. On the other hand, if the information indicating the state of the recognized animal included in the recognition result information does not match the information indicating the state of the animal to be observed that has been designated in advance, the transmission determination unit 13b sends the image data to the server 2. Determine not to send. The transmission determination unit 13 b outputs the image data to the transmission control unit 17 when it determines that the image data is to be transmitted to the server 2 .

The storage determination unit 15b acquires the recognition result information and the image data output by the recognition unit 12b, and stores the image data in the storage unit 16 based on the information indicating the state of the recognized animal included in the acquired recognition result information. It is determined whether or not to store to. The storage determination unit 15b is an example of a recording determination unit. For example, if the information indicating the state of the recognized animal included in the recognition result information matches the information indicating the state of the animal to be observed that has been specified in advance, the storage determining unit 15b stores the image data in the storage unit 16. It is determined to store to. On the other hand, if the information indicating the state of the recognized animal included in the recognition result information does not match the information indicating the state of the animal to be observed that has been designated in advance, the storage determining unit 15b stores the image data in the storage unit 16. It is determined not to be stored in the memory. When the storage determination unit 15b determines to store the image data in the storage unit 16, the storage determination unit 15b stores the image data in the observation target image data 16a of the storage unit 16 in association with the image data ID and the imaging date/time information.

(Operation of Imaging System)
FIG. 5 can be applied to an example of the operation of the imaging system 100b according to Modification 2 of the first embodiment. However, the processing from step S2 to step S4 is different from that of the imaging system 100 according to the first embodiment.
In step S2, in the imaging device 1b, the recognition unit 12b acquires the image data of the still image generated by the imaging unit 11, and acquires the still image by processing the acquired image data of the still image. The recognition unit 12b uses the learning model 16bb to recognize the state of the animal appearing in the acquired still image. The recognition unit 12b creates recognition result information including information indicating the recognized state of the animal, and outputs the created recognition result information together with the image data to the transmission determination unit 13b and the storage determination unit 15b.
In step S3, in the imaging device 1b, the transmission determination unit 13b acquires the recognition result information and the image data output by the recognition unit 12b, and based on the information indicating the state of the recognized animal included in the acquired recognition result information, to determine whether or not to transmit the image data to the server 2 .
In step S4, in the imaging device 1b, the memory determination unit 15b acquires the recognition result information and the image data output by the recognition unit 12b, and based on the information indicating the state of the recognized animal included in the acquired recognition result information, to determine whether or not to store the image data in the storage unit 16 . When it is determined not to store in the storage unit 16, the process proceeds to step S1.

In the second modified example of the first embodiment described above, the imaging device 1b may include the recognition result of the state of the animal appearing in the still image in the image data notification information. Specifically, the imaging device 1b may create the recognition result of the state of the animal in a language format such as JSON or XML, and include the created recognition result of the state of the animal in the image data notification information.
In the second modified example of the first embodiment described above, when the imaging device 1b determines to store the image data in the storage unit 16, the image data, the image data ID, and the imaging date and time information are all stored as the state of the animal. The recognition result may be stored in the observation target image data 16 a of the storage unit 16 . Specifically, the imaging device 1b creates the animal state recognition result in a language format such as JSON or XML, and stores the created animal state recognition result as image data, an image data ID, and an imaging date and time. It may be stored in the observation target image data 16a of the storage unit 16 in association with the information.
The above-described first embodiment and modification 2 of the first embodiment may be combined. Specifically, the imaging device 1b may use the learning model 16b and the learning model 16ba to recognize the type and state of the animal appearing in the acquired still image. In this case, the imaging device 1b determines whether or not the information indicating the type and state of the recognized animal included in the recognition result information matches the information indicating the type and state of the animal to be observed that has been designated in advance. You may For example, the imaging device 1b may use the learning model 16b and the learning model 16ba to recognize a wild boar as the type of animal in the acquired still image and capture a trap as the state.
The above-described first embodiment, modified example 1 of the first embodiment, and modified example 2 of the first embodiment may be combined. Specifically, the imaging device 1b uses a learning model 16b, a learning model 16ba, and a learning model 16bb to recognize the types, numbers, and states of animals appearing in the obtained still image. good too. In this case, the imaging device 1b determines whether the information indicating the type, number, and state of the recognized animals contained in the recognition result information matches the information indicating the type, number, and state of the animals to be observed that has been designated in advance. It may be determined whether For example, using the learning model 16b, the learning model 16ba, and the learning model 16bb, the imaging device 1b recognizes a wild boar as the type of animal in the acquired still image, 3 as the number, and catching a trap as the state. You may do so.

When a wild boar is caught in a trap, the wild boar may act violently, causing the infrared sensor to intermittently react to the wild boar, and the imaging device 1b may take an image based on the reaction of the infrared sensor. In this case, if the information indicating the state of the recognized animal included in the recognition result information continues to match the information indicating the state of the animal to be observed that has been designated in advance, the transmission determination unit 13b transfers the image data to Even if it is determined to transmit to the server 2, the frequency of outputting the image data to the transmission control unit 17 may be reduced. This reduces the frequency with which similar image data is sent to the server 2 . Further, for example, if the information indicating the state of the recognized animal included in the recognition result information continues to match the information indicating the state of the animal to be observed that has been designated in advance, the storage determining unit 15b stores the image data as Even if it is determined to store the image data in the storage unit 16, the frequency of storing the image data in the observation target image data 16a of the storage unit 16 may be reduced. This can prevent similar image data from being stored.
The imaging device 1b and the trap may be operated in conjunction with each other. Specifically, the imaging device 1b makes sure that the information indicating the type, number, and state of the recognized animals included in the recognition result information matches the information indicating the type, number, and state of the animals to be observed as specified in advance. A control signal for activating the trap may be transmitted when the trap is activated. Specifically, when the imaging device 1b recognizes that the type of animal in the acquired still image is a wild boar, the number is 3, and the state is trap capture, the imaging device 1b sets a trap such as lowering a trap fence into a box trap. A control signal to operate may be transmitted. The trap receives the control signal transmitted by the imaging device 1b, and based on the received control signal, lowers the fence of the trap onto the box trap. By configuring in this way, trap activation determination can be performed without going through the cloud, so the time required for trap activation determination can be shortened. Also, it may be controlled whether or not to lower the fence of the trap onto the box trap depending on the time period. By configuring in this way, when the hunter goes to kill the trap, it is possible to lower the fence of the trap.

According to the imaging system 100b according to Modification 2 of the first embodiment, the still image is generated based on the recognition result of the state of the animal captured in the still image generated by imaging by the imaging device 1b such as a trail camera. image data transmission and storage of the image data, or both. Therefore, even if image data of a still image is generated by imaging an object other than the state of the observation target, either one or both of transmission to the server 2 and storage in the imaging device 1b are performed. can be prevented. Either one or both of unnecessary data transmission and data storage can be reduced, so either one or both of the reduction in communication charges and the reduction in frequency of SD memory card replacement can be realized, resulting in increased convenience. can improve.

(Second embodiment)
(imaging system)
FIG. 8 is a diagram showing a configuration example of an imaging system according to the second embodiment of the present invention. In FIG. 8 , the imaging system 100 c includes an imaging device 1 , a server 2 and a learning device 4 . FIG. 8 shows an imaging device 1, a server 2, a learning device 4, a user U who acquires image data, and a terminal device 3 used by the user U. FIG.
The imaging device 1, the server 2, the terminal device 3, and the learning device 4 communicate via the network NW. The network NW includes, for example, the Internet, WAN, LAN, provider equipment, wireless base stations, and the like.

In the imaging system 100c according to the second embodiment, the imaging device 1 uses the learning model generated and output by the learning device 4 to recognize the type of animal appearing in the acquired still image.
FIG. 9 is a schematic diagram showing an example of an imaging system according to the second embodiment.
The imaging device 1 transmits image data of a still image generated by imaging by the imaging unit 11 to the cloud for a while after being installed at the place where observation is performed. A still image contains a moving object such as an animal.
In the cloud, the image data transmitted by the imaging device 1 is acquired, and the acquired image data is accumulated. The learning device 4 generates the learning model 46b by learning the relationship between the still image of the animal generated by processing the accumulated image data and the information indicating the type of the animal in the still image. do. The learning device 4 stores the generated learning model 46b. Specifically, a program such as a WEB application is executed on the cloud, and the user can check the still image transmitted by the imaging device 1 . A user can label (annotate) image data of a still image on the web. For example, the user can add information indicating the type of animal to the image data of the still image. Here, labeling (annotation) of image data of still images may be automatically executed on the WEB by image recognition technology.

The learning device 4 generates learning data by creating a plurality of combinations of still images showing animals and information indicating the types of animals given to the image data of the still images. When sufficient learning data is collected, the learning device 4 uses an object recognition technology such as SSD, YOLO, Faster R-CNN, etc. to identify a still image of an animal and the type of animal in the still image. A learning model 46b is generated by learning the relationship with the indicated information. The learning device 4 sequentially updates the generated learning model 46b.
When the learning model 46b with a predetermined accuracy is generated, the learning device 4 creates learning model notification information addressed to the imaging device 1, including the generated learning model 46b, transmission conditions, and storage conditions. and transmits the created learning model notification information to the imaging device 1 . Here, the transmission condition is a condition for transmitting image data to the server 2 when the learning model 46b is applied. The save condition is a condition for saving image data when the learning model 46b is applied. Specifically, an example of the transmission condition is "if the recognized animal is a boar, send image data", and an example of the storage condition is "if the recognized animal is a boar, save the image data". is.
The imaging device 1 receives the learning model notification information transmitted by the learning device 4, sets the learning model 46b included in the received learning model notification information, the transmission condition, and the storage condition, and sets the set learning model 46b. , the transmission condition and the storage condition.
Using the learning model 16b, if the type of animal in the acquired still image cannot be recognized, and if the score indicating the reliability of the recognized animal type is lower than the score threshold, the imaging device 1 recognizes the still image. Send the image information of the image to the cloud. The learning device 4 updates the learning model 46 b based on the image data of the still image transmitted by the imaging device 1 .
Among imaging device 1, server 2, terminal device 3, and learning device 4 included in imaging system 100c, learning device 4 different from the first embodiment will be described below.

(Learning device 4)
The learning device 4 is implemented by a device such as a personal computer, server, smart phone, tablet computer, or industrial computer. The learning device 4 includes a communication unit 44, a storage unit 46, and a learning unit 48, for example.
The storage unit 46 is implemented by an HDD, flash memory, RAM, ROM, or the like. The storage unit 46 stores a learning model 46b.
The learning model 46b is generated by learning the relationship between a still image of an animal and information indicating the type of animal in the still image.
The learning unit 48 is implemented, for example, by a hardware processor such as a CPU executing a program (software) stored in the storage unit 46 . Also, some or all of these functional units may be realized by hardware (including circuitry) such as LSI, ASIC, FPGA, and GPU, or by cooperation between software and hardware. may be implemented. The program may be stored in advance in a storage device such as an HDD or flash memory, or may be stored in a removable storage medium such as a DVD or CD-ROM. may be installed.
The learning unit 48 acquires the image data received by the communication unit 44 and accumulates the acquired image data in the storage unit 46 . The learning unit 48 generates the learning model 46b by learning the relationship between the still image of the animal generated by processing the accumulated image data and the information indicating the type of the animal captured in the still image. . The learning unit 48 stores the generated learning model 46 b in the storage unit 46 . Specifically, the learning device 4 generates learning data by generating a plurality of combinations of a still image showing an animal and information indicating the type of animal assigned to the image data of the still image. do. Learning device 4 uses object recognition technology such as SSD, YOLO, and Faster R-CNN when sufficient learning data is collected, based on the generated learning data, still images of animals and their still images A learning model 46b is generated by learning the relationship with the information indicating the type of animal in the image. The learning unit 48 sequentially updates the generated learning model 46b.
The learning unit 48 creates learning model notification information including the generated learning model 46b, transmission conditions, and storage conditions, and addressed to the imaging device 1, when a learning model that can obtain a predetermined accuracy is generated. , and outputs the created learning model notification information to the communication unit 44 .

The communication unit 44 is realized by a communication module. The communication unit 44 communicates with an external communication device via the network NW. The communication unit 44 is an example of an output unit. The communication unit 44 may communicate using a communication method such as a wired LAN, for example. Also, the communication unit 44 may communicate by a communication method such as wireless LAN, Bluetooth (registered trademark), or LTE (registered trademark). The communication unit 44 holds communication information necessary for communicating with the imaging device 1 via the network NW. The communication unit 44 receives image data transmitted by the imaging device 1 and outputs the received image data to the learning unit 48 . The communication unit 44 acquires the learning model notification information output by the learning unit 48 and transmits the acquired learning model notification information to the imaging device 1 .

(Operation of Imaging System)
FIG. 10 is a sequence chart showing an example of the operation of the imaging system according to the second embodiment. FIG. 10 shows processing in which the learning device 4 generates the learning model 46 b and transmits the generated learning model 46 b to the imaging device 1 .
(Step S21)
In the imaging device 1, the imaging unit 11 generates a still image by shooting when an infrared sensor (not shown) attached to the imaging device 1 detects a moving object.
(Step S22)
In the imaging device 1 , the imaging unit 11 outputs image data of the generated still image to the transmission control unit 17 . The transmission control unit 17 acquires the image data output by the imaging unit 11 and creates image data notification information including the acquired image data and addressed to the learning device 4 .
(Step S23)
In the imaging device 1 , the transmission control section 17 outputs the created image data notification information to the communication section 14 . The communication unit 14 acquires the image data notification information output by the transmission control unit 17 and transmits the acquired image data notification information to the learning device 4 .
(Step S24)
In the learning device 4 , the communication unit 44 receives the image data notification information transmitted by the imaging device 1 and outputs the received image data notification information to the learning unit 48 . The learning unit 48 acquires the image data notification information output by the communication unit 44 , acquires image data included in the acquired image data notification information, and accumulates the acquired image data in the storage unit 46 .

Steps S25 to S28 can apply steps S21 to S24.
(Step S29)
In the learning device 4, the learning unit 48 learns the relationship between the still image of the animal generated by processing each of the plurality of accumulated image data and the information indicating the type of animal captured in the still image. By doing so, a learning model 46b is generated.
(Step S30)
In the learning device 4 , the learning unit 48 creates learning model notification information including the generated learning model 46 b , transmission conditions, and storage conditions, and addressed to the imaging device 1 .
(Step S31)
In the learning device 4 , the learning unit 48 outputs the created learning model notification information to the communication unit 44 . The communication unit 44 acquires the learning model notification information output by the learning unit 48 and transmits the acquired learning model notification information to the imaging device 1 .
(Step S32)
In the imaging device 1, the communication unit 14 receives the learning model notification information transmitted by the learning device 4, and outputs the received learning model notification information to the recognition unit 12, the transmission determination unit 13, and the storage determination unit 15. do. The recognition unit 12 acquires the learning model notification information output by the communication unit 14 and stores the learning model 46 b included in the acquired learning model notification information in the learning model 16 b of the storage unit 16 . The transmission determination unit 13 acquires the learning model notification information output by the communication unit 14 and sets transmission conditions included in the acquired learning model notification information. The storage determination unit 15 acquires learning model notification information output by the communication unit 14 and sets storage conditions included in the acquired learning model notification information. For subsequent operations, FIG. 5 can be applied.

In the second embodiment described above, the case where the learning device 4 is applied to the first embodiment has been described, but the present invention is not limited to this example. For example, the learning device 4 may be applied to Modification 1 of the first embodiment. In this case, the learning device 4 generates a learning model 16ba that is used in the imaging device 1a according to the first modification of the first embodiment. Also, the learning device 4 may be applied to the modification 2 of the first embodiment. In this case, the learning device 4 generates a learning model 16bb that is used in the imaging device 1b according to the second modification of the first embodiment.
In the above-described second embodiment, the case where the learning device 4 creates learning model notification information including the learning model 46b, transmission conditions, and storage conditions and addressed to the imaging device 1 has been described. Examples are not limited. For example, the learning device 4 may create learning model notification information including the learning model 46 b and addressed to the imaging device 1 . In this case, transmission conditions and storage conditions are set in the imaging device 1 . With this configuration, the amount of data transmitted from the learning device 4 to the imaging device 1 can be reduced.
In the second embodiment described above, the case where the learning model 46b is generated based on the image data included in the image data notification information transmitted by the imaging device 1 has been described, but the invention is not limited to this example. For example, in addition to the image data included in the image data notification information transmitted by the imaging device 1, the learning model 46b is generated based on the image data included in the image data notification information transmitted by the imaging devices other than the imaging device 1. good too. Also, the learning model 46b may be shared with other users.
According to the imaging system 100c according to the second embodiment, the imaging system 100c includes the learning device 4 that generates the learning model 46b. By configuring in this way, the learning model 46b can be generated based on the installation position of the imaging device 1 . Therefore, it is possible to increase the types of animals to be observed. Since the learning model is generated based on the position where the imaging device 1 is set, the environment in which the imaging device 1 is installed, and the like, the accuracy of recognizing the type of animal captured in the still image can be improved.

Although the embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design changes and the like are also included within the scope of the present invention.
For example, a computer program for realizing the functions of the devices described above may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read and executed by a computer system. Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices.
In addition, "computer-readable recording medium" includes writable nonvolatile memories such as flexible discs, magneto-optical discs, ROMs and flash memories, portable media such as DVDs (Digital Versatile Discs), and computer system built-in media. A storage device such as a hard disk that
Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic Random Access Memory)), which holds the program for a certain period of time, is also included.
Further, the above program may be transmitted from a computer system storing this program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in a transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the program may be for realizing part of the functions described above.
Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

100, 100a, 100b, 100c... imaging system, 1, 1a, 1b... imaging device, 2... server, 3... terminal device, 4... learning device, 11... imaging unit, 12, 12a, 12b... recognition unit, 13, 13a, 13b... Transmission determination unit 14... Communication unit 15, 15a, 15b... Storage determination unit 16a... Observation target image data 16b, 16ba, 16bb... Learning model 16... Storage unit 17... Transmission control unit, DESCRIPTION OF SYMBOLS 24... Communication part, 26... Storage part, 26a... Image data DB, 28... Processing part, 44... Communication part, 46... Storage part, 46b... Learning model, 48... Learning part

Claims (6)

an imaging unit that generates a still image by shooting when a moving object is detected ;
Using a learning model that learns the relationship between an image of an animal and information indicating the type of animal captured in the image, the recognition image, which is an image to be recognized, is captured by the imaging unit. a recognition unit that recognizes the type of animal that is
a transmission determination unit that determines whether or not to transmit the image data of the recognition image based on the recognition result of the animal appearing in the recognition image performed by the recognition unit;
a storage determination unit that determines whether or not to store the image data of the recognized image based on the recognition result ,
The recognition unit recognizes the number of animals appearing in the recognition image using a learning model that has learned the relationship between an image of an animal and information indicating the number of animals appearing in the image,
Based on the result of recognition of the number of animals appearing in the recognition image performed by the recognition unit, the memory determination unit stores the information indicating the number of animals as information indicating the number of observation target animals specified in advance. When the information indicating the number of animals matches with the information indicating the number of animals to be observed, it is determined not to store the image data. death,
Based on the recognition result of the number of animals appearing in the recognition image performed by the recognition unit, the transmission determination unit determines that the information indicating the number of animals is information indicating the number of animals to be observed, which is specified in advance. When the information indicating the number of animals matches with the information indicating the number of animals to be observed, it is determined not to transmit the image data. imaging device. The recognition unit recognizes the state of the animal shown in the recognition image using a learning model that has learned the relationship between the image of the animal and the information indicating the state of the animal shown in the image,
The transmission determination unit determines whether or not to transmit the image data of the recognition image based on the recognition result of the state of the animal appearing in the recognition image performed by the recognition unit,
2. The imaging apparatus according to claim 1 , wherein said storage determination unit determines whether or not to store image data of said recognition image based on said recognition result. 3. The imaging device according to claim 1 , wherein the animal image includes an animal image captured with visible light and an animal image captured with infrared light. a communication unit that receives the learning model transmitted by a learning device that generates the learning model;
The imaging apparatus according to any one of claims 1 to 3 , wherein the recognition section uses the learning model received by the communication section to recognize the type of animal appearing in the recognition image. Using a learning model that learns the relationship between an image of an animal and information indicating the type of animal captured in the image, the imaging unit that generates a still image by capturing when a moving object is detected captures the image. a step of recognizing the type of animal appearing in the recognition image, which is the image to be recognized obtained by
a step of determining whether or not to transmit the image data of the recognition image based on the recognition result of the animal appearing in the recognition image obtained in the recognition step;
determining whether or not to store the image data of the recognized image based on the recognition result ;
In the recognizing step, the number of animals appearing in the recognition image is recognized using a learning model that has learned the relationship between the animal image and information indicating the number of animals appearing in the image,
In the step of determining whether or not to transmit the image data, information indicating the number of animals is stored in advance based on the recognition result of the number of animals appearing in the recognition image performed in the recognition step. If the information indicating the number of animals to be observed matches the specified information indicating the number of animals to be observed, it is determined that the image data is to be stored, and the information indicating the number of animals does not match the information indicating the number of animals to be observed that has been specified in advance. In this case, it is determined that the image data is not stored,
In the step of determining whether or not to store the image data of the recognition image, information indicating the number of animals based on the recognition result of the number of animals appearing in the recognition image performed in the recognition step. However, if it matches the information indicating the number of animals to be observed which is specified in advance, it is determined that the image data is to be transmitted, and the information indicating the number of animals is the information indicating the number of animals to be observed which is specified in advance. A control method executed by the imaging device , in which it is determined not to transmit the image data if they do not match . to the computer,
Using a learning model that learns the relationship between an image of an animal and information indicating the type of animal captured in the image, the imaging unit that generates a still image by capturing when a moving object is detected captures the image. a step of recognizing the type of animal appearing in the recognition image, which is the image to be recognized obtained by
determining whether or not to transmit the image data of the recognition image based on the recognition result of the animal appearing in the recognition image obtained in the recognition step;
determining whether or not to store the image data of the recognized image based on the recognition result ;
to the computer;
In the recognizing step, the number of animals in the recognition image is recognized using a learning model that has learned the relationship between the animal image and information indicating the number of animals in the image,
In the step of determining whether or not to transmit the image data, information indicating the number of animals is stored in advance based on the recognition result of the number of animals appearing in the recognition image performed in the recognition step. If the information indicating the number of animals to be observed matches the information indicating the number of animals to be observed, the image data is determined to be stored, and the information indicating the number of animals does not match the information indicating the number of animals to be observed that has been specified in advance. In this case, it is determined that the image data is not stored,
In the step of determining whether or not to store the image data of the recognition image, information indicating the number of animals based on the recognition result of the number of animals appearing in the recognition image performed in the recognition step. is identical to the information indicating the number of observation target animals specified in advance, the image data is determined to be transmitted, and the information indicating the number of animals is the information indicating the number of animals specified in advance to be observed. A computer program that determines that the image data is not to be transmitted if it does not match .

Images