JP7658540B2 - Imaging system, recording device, recording method and program - Google Patents

Description

The present invention relates to an imaging system, a recording device, a recording method, and a program.

A system has been proposed that uses an imaging means to monitor a specific target, and when an abnormality occurs in the target, the acquisition of information indicating the occurrence of the abnormality is used as a trigger to store image data captured before and after the occurrence of the abnormality in a storage means (see, for example, Patent Document 1). A system with similar functions to this system has also been provided, which employs a network camera as the imaging means, and in which a recording device for recording image data captured by the network camera before and after the occurrence of the abnormality is connected to the network camera via a network. In such a system, for example, when the recording device acquires information indicating the occurrence of an abnormality, it extracts and saves image data before and after the occurrence of the abnormality from the image data sent from the network camera to the recording device.

JP 2019-193283 A

In a system in which a network camera and a recording device are connected via a network as described above, in order to prevent image data loss due to communication delays and to achieve stable video transmission, it is common for the recording device to buffer the image data acquired from the network camera in a temporary storage means and then accumulate it in storage. In this case, a discrepancy occurs between the time when the image data after passing through the buffer is saved in storage and the time when the image data was actually captured. For this reason, there is a risk that the desired image data will not be obtained even if image data around the time when the abnormality occurred is acquired based on the time when each image data was saved in storage.

The present invention has been made in consideration of the above, and aims to provide an imaging system, a recording device, a recording method, and a program that can appropriately record video data before and after an event occurs.

In order to achieve the above object, an imaging system according to the present invention comprises:
An imaging device;
a recording device connected to the imaging device via a network,
The recording device is
a video buffer for temporarily storing video data;
a video acquisition unit that acquires the video image data transmitted from the imaging device and stores the video image data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period determination unit that, when a preset event occurs in an object to be imaged by the imaging device, determines , with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculation unit has elapsed, as an extraction period for extracting a portion of the moving image data stored in the moving image storage unit;
The video extracting unit extracts a portion of the video data corresponding to the extraction period from the video data stored in the video storage unit.

From another viewpoint, the recording device according to the present invention is
A recording device connected to an imaging device via a network,
a video buffer for temporarily storing video data;
a video acquisition unit that acquires the video image data transmitted from the imaging device and stores the video image data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period determination unit that , when a preset event occurs in an object to be imaged by the imaging device, determines , with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculation unit has elapsed , as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
The video data extracting unit extracts a portion of the video data corresponding to the extraction period from the video data stored in the video storage unit.

A recording method according to another aspect of the present invention comprises:
A method for recording moving image data received from an imaging device via a network, comprising:
acquiring video data transmitted from the imaging device and storing the video data in a video buffer that temporarily stores the video data;
fetching image data constituting the moving image data stored in the moving image buffer from the moving image buffer , and storing the image data in a moving image storage unit that stores the moving image data obtained from the imaging device in association with fetching time information indicating a time point when the image data was fetched;
a delay time calculation step of calculating a delay time caused by passing through the video buffer;
a step of, when a preset event occurs in an object to be imaged by the imaging device, specifying, with respect to the extraction time information, a period around a preset fixed time centered on a point in time when the delay time calculated by the delay time calculation step has elapsed from a time point when the event occurred, as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
Extracting a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit.

From another aspect, the program according to the present invention is
Computer,
a video buffer for temporarily storing video data;
a video acquisition unit that acquires video image data transmitted from an imaging device and stores the video data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period specifying unit that, when a preset event occurs in an object to be imaged by the imaging device , specifies, with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculating unit has elapsed, as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
a moving image extracting unit that extracts a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit;
Function as.

In the imaging system according to the present invention, when a preset event occurs in a subject imaged by the imaging device, the extraction period determination unit determines a period including a point in time when the delay time caused by passing through the video buffer has elapsed as an extraction period from which to extract a portion of the video data stored in the video storage unit. This makes it possible to determine the extraction period based on the aforementioned delay time so that at least the point in time when the event occurs is included in the aforementioned extraction period, and therefore makes it possible to appropriately record video data before and after the point in time when the event occurs.

1 is a diagram showing a configuration of an imaging system according to a first embodiment of the present invention; FIG. 1 is a diagram illustrating a hardware configuration of an imaging system according to a first embodiment. FIG. 2 is a diagram showing a functional configuration of the imaging system according to the first embodiment. 4 is an explanatory diagram of the operation of the imaging system according to the first embodiment; FIG. 5 is a flowchart showing an example of the flow of a recording process executed by the imaging device according to the first embodiment. 10A and 10B are diagrams illustrating the operation of an imaging system according to a comparative example. FIG. 11 is a diagram showing a functional configuration of an imaging system according to a second embodiment of the present invention. FIG. 2 is a diagram showing a part of the protocol stack of the Real-time Transport Protocol (RTP). 13 is a flowchart showing an example of the flow of a recording process executed by an imaging device according to a second embodiment. FIG. 11 is a diagram showing a functional configuration of an imaging system according to a third embodiment of the present invention. 13 is a flowchart showing an example of the flow of a recording process executed by an imaging device according to a third embodiment. FIG. 13 is a diagram showing a functional configuration of an imaging system according to a modified example.

(Embodiment 1)
An imaging system according to an embodiment of the present invention will be described in detail below with reference to Figs. 1 to 4. The imaging system according to this embodiment includes an imaging device and a recording device connected to the imaging device via a network. The recording device includes a video acquisition unit that acquires video data transmitted from the imaging device and stores it in a video buffer, a video extraction unit that extracts image data constituting the video data stored in the video buffer from the video buffer at a second time point after the first time point when the video acquisition unit acquires the image data, and stores the image data in the video storage unit, an extraction period determination unit that, when a preset event occurs in an imaging target by the imaging device, determines a period including a time point when the event occurs and a delay time due to passing through the video buffer as an extraction period for extracting a part of the video data stored in the video storage unit, and a video extraction unit that extracts a part of the video data corresponding to the extraction period from the video data stored in the video storage unit. This embodiment will be described in detail below.

As shown in FIG. 1, the imaging system according to the present embodiment includes a network camera 2 and a recording device 1 connected to the network camera 2 via a network NW. The network NW is, for example, a wired LAN (Local Area Network). The network camera 2 is used, for example, as a surveillance camera for monitoring production equipment. As shown in FIG. 2, the network camera 2 is an imaging device including a CPU (Central Processing Unit) 201, a main memory 202, an auxiliary memory 203, an imaging unit 204, a wired communication interface 206, and a bus 209 connecting each unit. The main memory 202 is composed of a volatile memory such as a RAM (Random Access Memory). The auxiliary memory 203 is composed of a non-volatile memory such as a ROM (Read Only Memory) and stores a program for controlling the network camera 2. The wired communication interface 206 is connected to the network NW and transmits and receives data via the network NW. The CPU 201 loads a program stored in the auxiliary memory unit 203 into the main memory unit 202 and executes it, thereby functioning as a video transmission unit 211 that transmits video image data representing video images captured by the imaging unit 204 to the recording device 1, as shown in FIG. 3.

The main memory unit 202 shown in FIG. 2 also has a captured image buffer 221 that temporarily stores video data representing video images captured by the imaging unit 204, as shown in FIG. 3. The video transmission unit 211 repeats the process of transmitting one frame of image data constituting the video data by the imaging unit 204 to the recording device 1, for example, each time the imaging unit 204 completes writing one frame of image data constituting the video data to the captured image buffer 221. The video transmission unit 211 also transmits the video data stored in the captured image buffer 221 to the recording device 1, for example, by RTP (Real-time Transport Protocol).

Returning to FIG. 2, the recording device 1 includes a CPU 101, a main memory unit 102, an auxiliary memory unit 103, a wired communication interface 106, an abnormality detection interface (hereinafter referred to as the "abnormality detection I/F") 107, a clock unit 108, and a bus 109 connecting the various units. The main memory unit 102 is composed of a volatile memory such as a RAM, and is used as a working area for the CPU 101. The auxiliary memory unit 103 is composed of a non-volatile memory such as a ROM, or an SSD (Solid State Drive), an HDD (Hard Disk Drive), or the like, and stores a program for controlling the recording device 1. The wired communication interface 106 is connected to the network NW, and transmits and receives data via the network NW. The abnormality detection I/F 107 is an interface that acquires trigger information. The abnormality detection I/F 107 may be, for example, a serial port such as RS-232. The trigger information is based on, for example, an abnormality detection signal output from the production equipment when an abnormality occurs in the production equipment that is the subject of imaging by the network camera 2. The timing unit 108 has, for example, a hardware timer.

The CPU 101 loads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as a video acquisition unit 111, a video extraction unit 112, a trigger acquisition unit 113, an average calculation unit 114, a delay time calculation unit 115, an extraction period determination unit 116, and a video extraction unit 117, as shown in Fig. 3. The main storage unit 102 shown in Fig. 2 also has a video buffer 121 that temporarily stores video data acquired from the network camera 2 in a FIFO (First In First Out) format, as shown in Fig. 3. The auxiliary storage unit 103 shown in Fig. 2 also has a video storage unit 131 that stores video data acquired from the network camera 2, and an extracted video storage unit 132 that stores video data extracted from a portion of the video data stored in the video storage unit 131, as shown in Fig. 3. The video storage unit 131 stores multiple pieces of image data constituting the video data retrieved from the video buffer 121 by the video retrieval unit 112, each of which is associated with time information indicating the time at which the video retrieval unit 112 retrieved the image data (hereinafter also referred to as retrieval time information).

The extracted video storage unit 132 stores video data obtained by extracting, from the video data stored in the video storage unit 131, a portion corresponding to a period before and after an event occurrence point, which is the point at which a preset event occurs in the subject captured by the network camera 2. The video data stored in the extracted video storage unit 132 is used, for example, to analyze the cause of a preset event when the event occurs in the subject captured by the network camera 2.

The video acquisition unit 111 acquires video data transmitted from the network camera 2 and stores it in the video buffer 121. The video acquisition unit 111 stores multiple image data constituting the acquired video data in chronological order in the video buffer 121. Here, the delay time from the time when the network camera 2 captures the image of the imaging target to the time when the video acquisition unit 111 acquires the video data is approximately equal to the time required for the network camera 2 to transmit one frame of image data to the recording device 1, and is sufficiently small to be negligible compared to the delay time from the time when the video acquisition unit 111 acquires the video data to the time when the video extraction unit 112 extracts the video data from the video buffer 121, which will be described later. Therefore, the time when the video acquisition unit 111 acquires the image data captured at the time of event occurrence can be considered to be approximately equal to the time of the aforementioned event occurrence.

The video retrieval unit 112 retrieves any image data constituting the video data stored in the video buffer 121 from the video buffer 121 at a time (second time point) that is the delay time due to passing through the video buffer 121 from the time (first time point) that the video acquisition unit 111 retrieved the image data, and stores it in the video storage unit 131. As shown in FIG. 4, for example, the video retrieval unit 112 retrieves image data PI (T0) acquired from the network camera 2 at the first time point T0 from the video buffer at the second time point T1 and stores it in the video storage unit 131. As a result, the video storage unit 131 stores the image data PI (T0) in association with the second time point T1.

Returning to FIG. 3, when the trigger acquisition unit 113 receives an abnormality detection signal via the abnormality detection I/F 107, that is, when it acquires trigger information indicating that the aforementioned event has occurred in the image capture target of the network camera 2, it notifies the extraction period determination unit 116 and the average calculation unit 114 of event occurrence notification information notifying that an event has occurred.

The average calculation unit 114 calculates the average data amount of the video data stored in the video buffer 121. The average calculation unit 114 constantly monitors the data amount of the video data stored in the video buffer 121, and when it acquires the event occurrence notification information from the trigger acquisition unit 113, it calculates the average data amount of the data amount stored in the video buffer 121 at each time point within a preset average calculation target period immediately before acquiring the event occurrence notification information. Here, the length of the average calculation target period is set to, for example, several seconds to several minutes. In this way, the average calculation unit 114 can calculate the smoothed data amount based on the average calculation target period. The calculation of the average data amount is useful when the buffered data amount fluctuates due to fluctuations in the video data transmitted from the network camera 2 caused by going through the network (for example, a decrease in the data amount due to a lack of bandwidth, etc.). The average calculation unit 114 notifies the delay time calculation unit 115 of the average data amount information indicating the calculated average data amount. In this embodiment, the average amount of data stored in the video buffer 121 at each point in time within a preset average calculation target period is calculated, but for example, the average calculation unit 114 may calculate the amount of data stored in the video buffer 121 at that point in time when it acquires event occurrence notification information from the trigger acquisition unit 113.

The delay time calculation unit 115 calculates the delay time from the first time point at the second time point, that is, the average delay time which is the average of the delay times at the time point when the video extraction unit 112 extracts the video data relative to the time point when the event occurs, based on the average data amount indicated by the average data amount information notified by the average calculation unit 114 and the bit rate when the video data is transmitted from the network camera 2 to the recording device 1. Specifically, the delay time calculation unit 115 calculates the average delay time by dividing the average data amount by the bit rate. For example, if the average data amount is 6 to 9 Mbits and the bit rate is 3 Mbps, the average delay time calculated by the delay time calculation unit 115 is 2 to 3 sec. The delay time calculation unit 115 notifies the extraction period specification unit 116 of the average delay time information indicating the calculated average delay time.

When an event occurs in the image capture target of the network camera 2, the extraction period determination unit 116 determines a period including a time when the delay time due to passing through the video buffer 121 has elapsed from the time when the event occurred as an extraction period for extracting a part of the video data stored in the video storage unit 131. Specifically, when an event occurs in the image capture target of the network camera 2, the extraction period determination unit 116 determines a period including a time when the average delay time has elapsed from the time when the event occurred as an extraction period for extracting a part of the video data stored in the video storage unit 131. When an event occurs in the image capture target of the network camera 2 and the trigger acquisition unit 113 notifies the extraction period determination unit 116 of event occurrence notification information, the extraction period determination unit 116 determines a period including a time when the average delay time dT1 indicated by the average delay time information notified by the delay time calculation unit 115 has elapsed from the time when the event occurrence notification information was notified as an extraction period. Here, as shown in FIG. 4, for example, the extraction period is set to the period between a point (Tt-dT2) a preset fixed time dT2 before the point (Tt+dT1) which is the average delay time dT1 described above from the event occurrence point Tt, and a point (Tt+dT2) the preset fixed time dT2 after the point. This fixed time dT2 can be preset by the user of the recording device 1 depending on the purpose of video recording, and is set to, for example, 30 seconds. The extraction period determination unit 116 notifies the video extraction unit 117 of extraction period information indicating the determined extraction period.

Returning to FIG. 3, the video extraction unit 117 extracts a portion of the video data corresponding to the extraction period indicated by the extraction period information notified by the extraction period specification unit 116 from the video data stored in the video storage unit 131. Specifically, the video extraction unit 117 refers to time information associated with each of the multiple image data constituting the video data stored in the video storage unit 131, and extracts image data from the multiple image data whose time indicated by the time information is included in the extraction period indicated by the extraction period information. For example, as shown in FIG. 4, the video extraction unit 117 extracts video data corresponding to a time before and after the time (Tt+dT1) that is the time when the average delay time dT1 has elapsed from the event occurrence time Tt, that is, the period from (Tt+dT1-dT2) to (Tt+dT1+dT2). Then, the video extraction unit 117 stores a portion of the extracted video data in the extracted video storage unit 132. As shown in FIG. 4, for example, the video extraction unit 117 extracts video data corresponding to a period between a time point (Tt+dT1-dT2) a preset fixed time dT2 before the event occurrence time Tt and a time point (Tt+dT1+dT2) the fixed time dT2 after the event occurrence time Tt, including the time point (Tt+dT1) when the average delay time dT1 described above has elapsed, and stores the extracted video storage unit 132. As a result, video data captured by the network camera 2 during the period from a time point (Tt-dT2) a fixed time dT2 before the event occurrence time Tt to a time point (Tt+dT2) the fixed time dT2 after the event occurrence time Tt can be stored in the extracted video storage unit 132.

Next, the recording process executed by the recording device 1 according to the present embodiment will be described in detail with reference to FIG. 5. This recording process is started, for example, when the recording device 1 is powered on. In parallel with this recording process, the video acquisition unit 111 acquires video data transmitted from the network camera 2 and stores it in the video buffer 121, and the video extraction unit 112 extracts image data constituting the video data from the video buffer 121 and stores it in the video storage unit 131 in association with the extraction time information. First, the trigger acquisition unit 113 determines whether or not trigger information has been acquired (step S101). Here, as long as the trigger acquisition unit 113 determines that trigger information has not been acquired (step S101: No), the process of step S101 is repeatedly executed. On the other hand, when the trigger acquisition unit 113 determines that trigger information has been acquired (step S101: Yes), the average calculation unit 114 calculates the average data amount in the average calculation target period immediately before the video data stored in the video buffer 121 (step S102). Next, the delay time calculation unit 115 calculates an average delay time, which is the average of the delay times from the time an event occurs in the imaging subject to the time the video extraction unit 112 extracts the video data, based on the average data amount calculated by the average calculation unit 114 and the bit rate at which the video data is transmitted from the network camera 2 to the recording device 1 (step S103).

Then, the extraction period determination unit 116 determines a predetermined period including the time when the calculated average delay time has elapsed from the time when the above-mentioned event occurred as an extraction period from which to extract a portion of the video data stored in the video storage unit 131 (step S104). After that, the video extraction unit 117 extracts a portion of the video data corresponding to the extraction period determined by the extraction period determination unit 116 from the video data stored in the video storage unit 131, and stores the extracted portion of the video data in the extracted video storage unit 132 (step S105). Next, the process of step S101 is executed again.

Now, suppose that the video extraction unit 117, for example as shown in FIG. 6, refers to the time information corresponding to each of the multiple image data constituting the video data stored in the video storage unit 131 and simply extracts a portion of the video data corresponding to the period from a time (Tt-dT2) before the event occurrence time Tt to a time (Tt+dT2). In this case, the portion of the extracted video data is video data captured by the network camera 2 during the period from a time (Tt-dT1-dT2) a certain time dT1+dT2 before the event occurrence time Tt to a time (Tt-dT1+dT2) a certain time dT2-dT1 after the event occurrence time Tt, and the video data is stored in the extracted video storage unit 132. That is, the length of time corresponding to the extracted video data before and after the event occurrence time Tt differs. In particular, if the delay time dT1 is longer than the certain time dT2 set by the user, the image data at the event occurrence time Tt cannot be extracted.

In contrast, according to the imaging system of this embodiment, the extraction period determination unit 116 determines a period including a time point (Tt+dT1) that is the average delay time dT1 determined based on the delay time from the event occurrence time Tt as an extraction period from which to extract a portion of the video data stored in the video storage unit 131. This makes it possible to determine the extraction period based on the delay time so that at least the event occurrence time Tt is included in the extraction period, and therefore makes it possible to appropriately record video data before and after the event occurrence time.

(Embodiment 2)
The imaging system of this embodiment differs from embodiment 1 in that the recording device 1 has a sequence number extraction unit that extracts sequence number information indicating a sequence number assigned to a packet containing image data at the time an event occurs, which constitutes part of the video data, and calculates the aforementioned delay time based on the sequence number.

The hardware configuration and functional configuration of the imaging system according to this embodiment are the same as those of the imaging system according to the first embodiment. Therefore, the hardware configuration of the imaging system according to this embodiment will be described using the symbols shown in FIG. 2 as appropriate. As shown in FIG. 7, the CPU 101 of the recording device 2001 according to this embodiment reads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as a video acquisition unit 2111, a video extraction unit 2112, a trigger acquisition unit 113, a delay time calculation unit 2115, an extraction period specification unit 116, a video extraction unit 117, and a sequence number extraction unit 2118. Note that in FIG. 7, the same components as those in the first embodiment are given the same symbols as those in FIG. 3. In addition, the main storage unit 102 of the recording device 2001 has a video buffer 2121 that temporarily stores video data acquired from the network camera 2 in association with a sequence number assigned to a packet including image data constituting the video data transmitted from the network camera 2. Furthermore, the auxiliary storage unit 103 of the recording device 2001 has a video storage unit 131 and an extracted video storage unit 132.

The video acquisition unit 2111 stores image data at the time of the event occurrence, which constitutes part of the video image data transmitted from the network camera 2, in the video buffer 2121 in association with the sequence number assigned to the packet containing the image data. Here, the packet containing the image data transmitted from the network camera 2 and acquired by the video acquisition unit 2111 conforms to the above-mentioned RTP and contains RTP header information as shown in FIG. 8. Here, "V" indicates the version number of RTP, and is set to "2", for example. "P" indicates whether the payload portion corresponding to the RTP header of the packet is padded or not, and is set to "1" if it is padded, and is set to "0" if it is not padded. "X" indicates whether an extension header is added or not, and is set to "1" if an extension header is added, and is set to "0" if an extension header is not added. "CC" indicates the length of the list of information indicating multiple media streams contained in the packet when the payload portion corresponding to the RTP header of the packet is a composite of multiple media streams. "M" indicates an event defined in RTP, such as the start, restart, and end of a media stream. "PT" is information that identifies the media type of the data stored in the payload corresponding to the RTP header. The sequence number is expressed as an unsigned 16-bit integer value that is incremented by one each time a packet is sent from the network camera 2. The timestamp indicates the time when the image data included in the packet was generated. This is approximately equal to the time when the image was captured. When the video acquisition unit 2111 acquires a packet containing image data sent from the network camera 2, it extracts the image data from the acquired packet and stores the extracted image data in the video buffer 2121 in association with sequence number information indicating the sequence number included in the RTP header of the acquired packet. Note that, although the sequence number written in the RTP header is used for the explanation here, it is sufficient that the image data and the sequence number are associated in the packet sent from the network camera, and is not limited to RTP.

Returning to FIG. 7, when the trigger acquisition unit 113 acquires trigger information indicating that the aforementioned event has occurred in the image capture target of the network camera 2, the trigger acquisition unit 113 notifies the extraction period determination unit 116, the delay time calculation unit 2115, and the sequence number extraction unit 2118 of event occurrence notification information notifying that the event has occurred.

When the sequence number extraction unit 2118 is notified of the event occurrence notification information from the trigger acquisition unit 113, it extracts sequence number information indicating the sequence number assigned to a packet including image data at the time of the event occurrence that constitutes part of the video image data. Specifically, when the sequence number extraction unit 2118 is notified of the event occurrence notification information from the trigger acquisition unit 113, it extracts sequence number information indicating the sequence number included in the RTP header of the packet acquired by the video acquisition unit 2111 from the network camera 2. Since the delay at the time of reception by the video acquisition unit 2111 is almost negligible, the image data loaded into the packet with this sequence number is almost equal to that captured at the time the event occurred. The sequence number extraction unit 2118 notifies the video extraction unit 2112 of the extracted sequence number information.

As with the video fetching unit 112 according to embodiment 1, the video fetching unit 2112 fetches from the video buffer 2121 image data constituting the video data stored in the video buffer 2121 and stores it in the video storage unit 131. At this time, the time at which the video data is fetched from the video buffer 2121 is delayed by the delay time caused by passing through the video buffer 2121. Then, when the video fetching unit 2112 fetches from the video buffer 2121 image data associated with the sequence number information notified by the sequence number extraction unit 2118, it notifies the delay time calculation unit 2115 of event occurrence time image fetching notification information notifying that the image data at the time of the event occurrence has been fetched.

Based on the sequence number, the delay time calculation unit 2115 identifies the retrieval time when the video retrieval unit 2112 retrieves the image data at the time of event occurrence from the video buffer 2121, and calculates the time difference between the identified retrieval time and the time of event occurrence as the delay time. Specifically, the delay time calculation unit 2115 identifies the time when the video retrieval unit 2112 notifies the event occurrence time image retrieval notification information as the retrieval time. The delay time calculation unit 2115 then notifies the extraction period determination unit 116 of delay time information indicating the calculated delay time. When the event occurrence notification information is notified from the trigger acquisition unit 113, the extraction period determination unit 116 identifies the period including the time when the delay time indicated by the delay time information notified from the delay time calculation unit 2115 has elapsed from the time when the event occurrence notification information was notified as the extraction period.

Next, the recording process performed by the recording device 2001 according to this embodiment will be described in detail with reference to FIG. 9. In parallel with this recording process, the video acquisition unit 2111 acquires video data transmitted from the network camera 2 and stores it in the video buffer 121 in association with sequence number information, and the video extraction unit 2112 extracts image data constituting the video data from the video buffer 121 and stores it in the video storage unit 131 in association with the extraction time information. First, the trigger acquisition unit 113 determines whether or not trigger information has been acquired (step S201). Here, as long as the trigger acquisition unit 113 determines that trigger information has not been acquired (step S201: No), the process of step S201 is repeatedly executed. On the other hand, if the trigger acquisition unit 113 determines that it has acquired trigger information (step S201: Yes), the sequence number extraction unit 2118 extracts sequence number information indicating the sequence number assigned to the packet from the RTP header information of the packet that contains image data at the time the event occurred, which constitutes part of the video data (step S202).

Next, the delay time calculation unit 2115 identifies the time when the video extraction unit 2112 retrieved the image data at the time of the event occurrence from the video buffer 2121 based on the sequence number information extracted by the sequence number extraction unit 2118 (step S203). Next, the delay time calculation unit 2115 calculates the time difference between the identified retrieval time and the time of the event occurrence as the delay time (step S204). More specifically, the delay time calculation unit 2115 calculates the time difference between the timestamp included in the packet at the identified retrieval time and the timestamp included in the packet at the time of the event occurrence based on the sequence number information. After that, the extraction period identification unit 116 identifies the period including the time when the calculated delay time has elapsed from the aforementioned event occurrence time as the extraction period for extracting a part of the video data stored in the video storage unit 131 (step S205). Next, the video extraction unit 117 extracts a portion of the video data corresponding to the extraction period identified by the extraction period identification unit 116 from the video data stored in the video storage unit 131, and stores the extracted portion of the video data in the extracted video storage unit 132 (step S206). Then, the process of step S201 is executed again.

As described above, in the imaging system according to the present embodiment, the sequence number extraction unit 2118 extracts sequence number information indicating the sequence number assigned to the packet including the image data at the time of the event occurrence. The delay time calculation unit 2115 also identifies the retrieval time at which the image data at the time of the event occurrence was retrieved from the video buffer 2121 by the video retrieval unit 2112 based on the sequence number information, and calculates the time difference between the identified retrieval time and the time of the event occurrence as the delay time. The extraction period determination unit 116 then identifies the period including the time when the calculated delay time has elapsed from the time of the event occurrence as the extraction period. This makes it possible to more accurately calculate the delay time corresponding to the image data at the time of the event occurrence, and therefore to appropriately determine the extraction period.

(Embodiment 3)
The imaging system of this embodiment differs from embodiment 1 in that the recording device 1 has a time information extraction unit that extracts time information (timestamp) indicating the time when an event occurred that is contained in a packet containing image data at the time when the event occurred, and identifies the aforementioned extraction period based on the extracted time information.

The hardware configuration and functional configuration of the imaging system according to this embodiment are the same as those of the imaging system according to the first embodiment. Therefore, the hardware configuration of the imaging system according to this embodiment will be described using the symbols shown in FIG. 2 as appropriate. As shown in FIG. 10, the CPU 101 of the recording device 3001 according to this embodiment reads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as a video acquisition unit 3111, a video extraction unit 3112, a trigger acquisition unit 113, an extraction period specification unit 3116, a video extraction unit 117, and a time information extraction unit 3118. Note that in FIG. 10, the same components as those in the first embodiment are given the same symbols as those in FIG. 3. In addition, the main storage unit 102 of the recording device 3001 has a video buffer 3121 that temporarily stores video data acquired from the network camera 2 in association with time information indicating the time when the image data constituting the video data transmitted from the network camera 2 was generated. Furthermore, the auxiliary storage unit 103 of the recording device 3001 has a video storage unit 3131 and an extracted video storage unit 132. The video storage unit 3131 stores multiple pieces of image data constituting the video data retrieved from the video buffer 3121 by the video retrieval unit 3112, each piece corresponding to time information indicating the time at which the image data was generated.

The video acquisition unit 3111 stores image data at the time of event occurrence, which constitutes part of the video data transmitted from the network camera 2, in the video buffer 3121 in association with time information indicating the time when the image data was generated. When the video acquisition unit 3111 acquires a packet containing image data, it extracts the image data and generates time information indicating the time when the image data was generated by referring to the above-mentioned timestamp included in the RTP header of the packet. The video acquisition unit 3111 then stores the extracted image data and the generated time information in association with each other in the video buffer 3121.

When the trigger acquisition unit 113 acquires trigger information indicating that the aforementioned event has occurred in the subject imaged by the network camera 2, it notifies the time information extraction unit 3118 of event occurrence notification information notifying that the event has occurred.

When the time information extraction unit 3118 is notified of the event occurrence notification information from the trigger acquisition unit 113, it extracts time information indicating the time when image data at the time of the event occurrence that constitutes part of the video image data was generated. Specifically, when the time information extraction unit 3118 is notified of the event occurrence notification information from the trigger acquisition unit 113, it extracts time stamp information indicating the time stamp included in the RTP header of the packet acquired by the video acquisition unit 2111 from the network camera 2, i.e., time information. The time information extraction unit 3118 notifies the extraction period determination unit 3116 of the extracted time information.

The video extraction unit 3112 extracts from the video buffer 3121 the image data constituting the video data stored in the video buffer 3121 and the time information corresponding to the image data, and stores these in the video storage unit 3131 in association with each other.

The extraction period determination unit 3116 determines a predetermined period including the time point at which an event occurs as an extraction period from which to extract a portion of the video data stored in the video storage unit 3131, based on time information indicating the time point at which an event occurs that is notified by the time information extraction unit 3118. The extraction period determination unit 3116 notifies the video extraction unit 117 of extraction period information indicating the determined extraction period. The video extraction unit 117 references time information corresponding to the image data that constitutes the video data stored in the video storage unit 3131, and extracts a portion of the video data where the time point indicated by the time information is included within the period indicated by the extraction period information notified by the extraction period determination unit 3116.

Next, the recording process performed by the recording device 3001 according to this embodiment will be described in detail with reference to FIG. 11. In parallel with this recording process, the video acquisition unit 3111 acquires video data transmitted from the network camera 2 and stores the data in the video buffer 3121 in association with time information, and the video extraction unit 3112 extracts image data constituting the video data from the video buffer 121 and the corresponding time information, and stores the data in the video storage unit 3131 in association with each other. First, the trigger acquisition unit 113 determines whether or not trigger information has been acquired (step S301). Here, as long as the trigger acquisition unit 113 determines that trigger information has not been acquired (step S301: No), the process of step S301 is repeatedly executed. On the other hand, if the trigger acquisition unit 113 determines that it has acquired trigger information (step S301: Yes), the time information extraction unit 3118 extracts time information indicating the time when the image data was generated from the RTP header information of a packet that contains image data at the time when the event occurred, which constitutes part of the video data (step S302).

Next, the extraction period determination unit 3116 determines the period including the time when the image data at the time of the event occurrence indicated by the time information extracted by the time information extraction unit 3118 was generated as the extraction period from which to extract a portion of the video data stored in the video storage unit 3131 (step S303). Next, the video extraction unit 117 extracts a portion of the video data corresponding to the extraction period determined by the extraction period determination unit 3116 from the video data stored in the video storage unit 3131, and stores the extracted portion of the video data in the extracted video storage unit 132 (step S304). Then, the process of step S301 is executed again.

As described above, according to the imaging system of this embodiment, the time information extraction unit 3118 extracts time information indicating the time when the image data at the time of event occurrence was generated, which is included in a packet including image data at the time of event occurrence when an event constituting part of the video image data occurred. The extraction period determination unit 3116 then determines, as the extraction period, the period including the time when the image data at the time of event occurrence indicated by the time information extracted by the time information extraction unit 3118 was generated. This allows the time when the image data at the time of event occurrence was generated to be accurately determined, and therefore the extraction period can be appropriately determined.

Although each embodiment of the present invention has been described above, the present invention is not limited to the configuration of each of the above-mentioned embodiments. For example, the extraction period determination unit may determine, when an event occurs in an object captured by the network camera 2, a period including a point in time that is a preset fixed delay time after the event occurrence as the above-mentioned extraction period.

The hardware configuration and functional configuration of the imaging system according to this modification are the same as those of the imaging system according to the first embodiment. In the imaging system according to this modification, as shown in FIG. 12, the CPU 101 of the recording device 4001 loads a program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as the video acquisition unit 111, the video extraction unit 112, the trigger acquisition unit 113, the extraction period determination unit 4116, and the video extraction unit 117. Note that in FIG. 12, the same components as those in the first embodiment are given the same reference numerals as those in FIG. 3. The auxiliary storage unit 103 of the recording device 3001 has a video storage unit 131, an extracted video storage unit 132, and a delay time storage unit 4133. The delay time storage unit 4133 stores delay time information indicating a predetermined fixed delay time as the delay time due to passing through the video buffer 121 based on past experience, for example, the delay time from the first time point to the second time point described in the first embodiment.

When an event occurs in the subject captured by the network camera 2, the extraction period determination unit 4116 determines a period including a point in time from the time the event occurs that is a certain delay time indicated by the delay time information stored in the delay time memory unit 4133, as an extraction period from which to extract a portion of the video data stored in the video memory unit 131.

With this configuration, the processing executed by the recording device 4001 is simplified, thereby reducing the processing load on the recording device 4001.

As another method for determining the delay time, a flag may be added to the video data obtained from the network camera 2 when trigger information is received, and the delay time may be determined from the difference between the time when the flagged video data is retrieved from the video buffer 121 and the time when the trigger information is received.

In each embodiment, an example has been described in which the video transmission unit 211 transmits the video data stored in the captured image buffer 221 to the recording device 1 by RTP, but the protocol used to transmit the video data is not limited to this. For example, the video transmission unit 211 may transmit the video data stored in the captured image buffer 211 to the recording device 1 by HLS (HTTP Live Streaming), which is a streaming protocol that is a technology for distributing video over the Internet.

The various functions of the recording devices 1, 2001, 3001, and 4001 according to the present invention can be realized by using a computer system equipped with a wireless communication module, rather than a dedicated system. For example, a program for executing the above operations can be stored on a non-transitory recording medium (such as a CD-ROM) that can be read by the computer system and distributed to a computer connected to a network, and the program can be installed in the computer system to configure the recording devices 1, 2001, 3001, and 4001 to execute the above-mentioned processes.

The method of providing the program to the computer is arbitrary. For example, the program may be uploaded to a server on a communication line and distributed to the computer via the communication line. The computer then starts up the program and executes it under the control of the OS in the same way as other applications. In this way, the computer functions as the recording device 1, 2001, 3001, 4001 that executes the above-mentioned processing.

Although the embodiments and modifications of the present invention have been described above, the present invention is not limited to these. The present invention includes suitable combinations of the embodiments and modifications, and suitable modifications thereto.

The present invention is suitable as an imaging system that performs event-driven recording using a network camera.

1, 2001, 3001, 4001: Recording device, 2: Network camera, 101, 201: CPU, 102, 202: Main memory unit, 103, 203: Auxiliary memory unit, 106, 206: Wired communication interface, 108: Timekeeping unit, 109, 209: Bus, 111, 2111, 3111: Video acquisition unit, 112, 2112, 3112: Video extraction unit, 113: Trigger acquisition unit, 114: Average value calculation unit, 115 , 2115: Delay time calculation unit, 116, 3116, 4116: Extraction period determination unit, 117: Video extraction unit, 121, 2121, 3121: Video buffer, 131, 3131: Video storage unit, 132: Extracted video storage unit, 204: Imaging unit, 211: Video transmission unit, 221: Captured image buffer, 2118: Sequence number extraction unit, 3118: Time information extraction unit, 4133: Delay time storage unit, NW: Network

Claims (9)

An imaging device;
a recording device connected to the imaging device via a network,
The recording device is
a video buffer for temporarily storing video data;
a video acquisition unit that acquires the video image data transmitted from the imaging device and stores the video image data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period determination unit that, when a preset event occurs in an object to be imaged by the imaging device, determines , with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculation unit has elapsed, as an extraction period for extracting a portion of the moving image data stored in the moving image storage unit;
a moving image extracting unit that extracts a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit,
Imaging system. The recording device is
an average calculation unit that calculates an average data amount of the video data stored in the video buffer,
The delay time calculation unit calculates an average delay time , which is an average of the delay times, based on the average data amount and a bit rate at the time of transmitting the video data from the imaging device to the recording device.
The imaging system according to claim 1 . the imaging device transmits the moving image data to the recording device using a protocol used for transmitting the moving image data;
The recording device is
a sequence number extracting unit that extracts sequence number information indicating a sequence number assigned to a packet including the image data at the time point when the event occurs, the sequence number extracting unit extracting ...
the video acquisition unit stores the image data at the time point when the event occurs in the video buffer in association with the sequence number information corresponding to a packet including the image data;
the delay time calculation unit specifies, based on the sequence number information, fetch time information indicating a time when the image data at the time point when the event occurs was fetched from the video buffer by the video fetch unit, and calculates , as the delay time, a time difference between the specified fetch time information and the time point when the event occurs.
The imaging system according to claim 1 . The delay time calculation unit sets a predetermined time as the delay time .
The imaging system according to claim 1 . An imaging device;
a recording device connected to the imaging device via a network,
the imaging device transmits moving image data of an object to the recording device according to a protocol used for transmitting moving image data;
The recording device is
a video buffer for temporarily storing video data;
a video acquisition unit that acquires the video data and stores it in a video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video extracting unit that extracts image data constituting the video data stored in the video buffer from the video buffer and stores the image data in the video storage unit;
a timestamp extracting unit that, when a preset event occurs in an object to be imaged by the imaging device, extracts a timestamp indicating a time when image data at the time point of the event occurrence was generated, the timestamp being included in a packet including image data at the time point of the event occurrence, the packet constituting a part of the video data;
an extraction period determination unit that determines a period around the event occurrence time point indicated by the time stamp by a preset certain time as an extraction period for extracting a part of the video data stored in the video storage unit;
a moving image extracting unit that extracts a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit,
Imaging system. The recording device is
A trigger acquisition unit that acquires trigger information indicating that the event has occurred.
The imaging system according to claim 1 . A recording device connected to an imaging device via a network,
a video buffer for temporarily storing video data;
a video acquisition unit that acquires the video image data transmitted from the imaging device and stores the video image data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period determination unit that , when a preset event occurs in an object to be imaged by the imaging device, determines , with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculation unit has elapsed , as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
a moving image extracting unit that extracts a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit,
Recording device. A method for recording moving image data received from an imaging device via a network, comprising:
acquiring video data transmitted from the imaging device and storing the video data in a video buffer that temporarily stores the video data;
fetching image data constituting the moving image data stored in the moving image buffer from the moving image buffer , and storing the image data in a moving image storage unit that stores the moving image data obtained from the imaging device in association with fetching time information indicating a time point when the image data was fetched;
a delay time calculation step of calculating a delay time caused by passing through the video buffer;
a step of, when a preset event occurs in an object to be imaged by the imaging device, specifying, with respect to the extraction time information, a period around a preset fixed time centered on a point in time when the delay time calculated by the delay time calculation step has elapsed from a time point when the event occurred, as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
extracting a portion of the video data corresponding to the extraction period from the video data stored in the video storage unit.
How to record. Computer,
a video buffer for temporarily storing video data;
a video acquisition unit that acquires video image data transmitted from an imaging device and stores the video data in the video buffer;
a moving image storage unit that stores the moving image data acquired from the imaging device;
a video fetching unit fetching image data constituting the video data stored in the video buffer from the video buffer , and storing the image data in the video storage unit in association with fetching time information indicating a time point at which the image data was fetched;
a delay time calculation unit that calculates a delay time caused by passing through the video buffer;
an extraction period specifying unit that, when a preset event occurs in an object to be imaged by the imaging device , specifies, with respect to the retrieval time information, a period around a preset fixed time centered on a point in time when the event occurs and the delay time calculated by the delay time calculating unit has elapsed, as an extraction period for extracting a part of the moving image data stored in the moving image storage unit;
a moving image extracting unit that extracts a portion of the moving image data corresponding to the extraction period from the moving image data stored in the moving image storage unit;
A program to function as a

Images