JP5246978B2 - Image sensing device - Google Patents

Description

  The present invention relates to an image sensing device that is used in a security monitoring device and controls operations of a plurality of image sensors.

  2. Description of the Related Art Conventionally, there is an image sensing system that changes a sensing method of an image sensor according to a situation of a sensing area sensed by the image sensor and a restriction of a device that uses sensing data. Some of these image sensing systems change the recording mode (recording time ratio) to a VTR (Video Tape Recorder) when a change in video brightness is detected for analog video signals of each of a plurality of image sensors (for example, , See Patent Document 1). According to the technique disclosed in Patent Document 1, for example, an image of a sensor that detects an abnormality can be automatically switched to long-time recording.

  Another conventional technology is to change the picture structure of digital video data MPEG (Moving Picture Experts Group) according to the detection result of the presence or absence of intruders in order to save the recording capacity while ensuring the amount of information at the time of abnormality. (For example, refer to Patent Document 2). Specifically, the I frame and the P frame are usually left out, and the other frames are thinned out so as not to be thinned out when an abnormality such as intruder detection is detected.

  As another conventional technique, one camera image is stored in a plurality of image frame memories, and information on the entire sensing target area stored in one image frame memory is displayed on one monitor, Some of the other plurality of monitors cut out and display only the designated area portion from the other plurality of frame memories (see, for example, Patent Document 3). Japanese Patent Application Laid-Open No. H10-228707 discloses that an image stored in a memory is distributed to each monitoring terminal by specifying a region from each of the plurality of monitoring terminals.

JP-A-8-205133 JP 2002-203022 A JP 2002-262276 A

  However, Patent Document 1 discloses that the recording time mode of each camera is switched when an alarm signal is detected, but it is considered that the accumulation schedule is variable for each partial region within the field of view of the camera. Not.

  Further, Patent Document 2 discloses that frames are thinned out for each camera when no abnormality is detected. However, data thinning is performed for each partial region within the field of view of the camera in cooperation with a plurality of cameras. It is not considered to change the way.

  Further, in Patent Document 3, an entire image of a monitoring target area and an image of a partial area are simultaneously provided for one monitoring camera image, and an arbitrary designated portion by a plurality of terminals is distributed for one camera image. This is disclosed, but image data reading when the entire sensing target area is monitored by a plurality of image sensors is not considered.

  The present invention has been made in view of such circumstances, and while securing information on the entire sensing target area and information on a plurality of objects by a plurality of image sensors, power consumption and transfer image data on the image sensor side. An object of the present invention is to provide an image sensing device capable of reducing the amount and storing data for a long time on the image data recording device side.

  In order to achieve the above object, an image sensing device of the present invention is an image sensing device configured by a plurality of image sensors communicating with each other through a network, and the plurality of image sensors are overlapped by the plurality of image sensors. When the overlapping sensing area to be sensed and the single sensing area that is sensing independently are held, and the object is detected in the overlapping sensing area in the overlapping sensing area, each of the image sensors corresponds to the overlapping sensing area. When image data is read and there is no object in the overlapping sensing area, the plurality of image sensors communicate with each other, so that at least one image sensor has an image of the selected image sensor with respect to the overlapping sensing area. Read data, and the plurality of Image sensor is characterized by reading image data at all times of the single sensing region.

  The image sensing device of the present invention is an image sensing device including a plurality of image sensors and a sensing data storage unit that communicate with each other through a network, and the plurality of image sensors are overlapped by the plurality of image sensors. When the object is detected in the overlapping sensing area in the overlapping sensing area, each of the image sensors detects the overlapping sensing area. And when the object is not present in the overlapping sensing area, the plurality of image sensors communicate with each other so that at least one image sensor is in the overlapping sensing area. For the selected image It reads out the image data of the service transmitted to the sensing data storage unit, also, the plurality of image sensors and transmits to the sensing data storage unit the image data is read out at all times of the single sensing region.

  In the above configuration, when there is no object in the overlapping sensing area, the image sensor performs sensing of the overlapping sensing area at a predetermined interval.

  In the above configuration, the image sensor has a first level of detail when sensing the entire overlapping sensing area or the entire single sensing area, and within at least one sensing area of the overlapping sensing area and the single sensing area. The second detail level when sensing a part of the image sensor is brought close to the first target detail level and the second target detail level, respectively, and the image sensor detects the entire overlapping sensing area or single sensing area. When a part of the peripheral area of the sensing area to be sensed is sensed with the second level of detail by another image sensor, the first level of the other image sensor is set to the first level of detail of the image sensor. The total amount of read data per unit time from the plurality of sensing areas is predetermined. Controlling said first and second level of detail so that the target value.

  According to the present invention, the level of detail for selecting and / or sensing data in a region that is sensed redundantly by a plurality of image sensors is selected and / or sensed according to the presence / absence or position of an object in the sensing region. Therefore, it is possible to reduce the amount of data to be read or stored while preserving the information of the entire sensing target area with priority on the information of the target object. More priority video monitoring and long-time data can be recorded.

1 is a block diagram showing a schematic configuration of an image sensing system according to Embodiment 1 of the present invention. The figure which shows an example of the sensing area | region in the image sensing system which concerns on Embodiment 1 of this invention. The flowchart showing operation | movement of the image sensing system which concerns on Embodiment 1 of this invention. The figure which shows an example of the data reduction schedule in the image sensing system which concerns on Embodiment 1 of this invention. The figure which shows an example of the movement path | route of the target object in the sensing area | region in the image sensing system which concerns on Embodiment 1 of this invention. The figure which shows an example of the target priority schedule in the image sensing system which concerns on Embodiment 1 of this invention. The figure explaining several detail in the image sensing system which concerns on Embodiment 2 of this invention. The figure which shows the concept of the process of the image sensing system which concerns on Embodiment 2 of this invention. The block diagram which shows schematic structure of the image sensing system which concerns on Embodiment 2 of this invention. The flowchart showing operation | movement of the image sensing system which concerns on Embodiment 2 of this invention. The figure which shows an example of the setting of the target data amount in the image sensing system which concerns on Embodiment 2 of this invention. The figure which shows an example of the calculation result of the partial sensing area whole frame detail level and local frame detail level in the image sensing system which concerns on Embodiment 2 of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments for carrying out the invention will be described in detail with reference to the drawings.

(Embodiment 1)
The image sensing system according to Embodiment 1 of the present invention changes a schedule for reading data of a region that is sensed redundantly by a plurality of image sensors according to the presence or absence and position of an object in the sensing region. As a result, the amount of accumulated data is reduced while securing information on the entire sensing target area and information on the target object.

FIG. 1 is a block diagram showing a schematic configuration of the image sensing system according to the present embodiment. In FIG. 1, the image sensing system according to the present embodiment includes an image sensing device 100, a plurality of image sensors 101, and a sensing data storage unit 107. A plurality of image sensors 101 are installed for the sensing target area. In FIG. 2, four image sensors 101 (C ₁ , C ₂ , C ₃ , C ₄ ) having a fixed angle of view are installed so as to monitor the entire sensing target area 200 (rectangular ADPM) that is a monitoring area without blind spots. It is an example. Here, areas sensed by the image sensors C ₁ , C ₂ , C ₃ , and C ₄ are a rectangle ACKI, a rectangle BDLJ, a rectangle EGOM, and a rectangle FHPN, respectively.

The image sensing device 100 includes a frame memory 102, a partial sensing region storage unit 103, a partial sensing region-specific event detection unit 104, a partial sensing region data accumulation schedule storage unit 105, and an accumulated data selection unit 106. Image data (hereinafter referred to as sensing data) obtained by the image sensor 101 is stored in the frame memory 102 for each image sensor 101. In FIG. 2, regions R ^{1 to} R ⁹ divided by the sensing region of the image sensor 101 are referred to as a partial sensing region 201 and are stored in the partial sensing region storage unit 103. The partial sensing region 201 includes a single sensing region 203 that is sensed by only one image sensor 101 such as R ¹ , R ³ , R ⁷ , and R ⁹ , and R ² , R ⁴ , R ⁵ , R ⁶ , R ^{As shown in FIG. 8} , there is an overlapping sensing area 204 sensed by a plurality of image sensors 101.

  The partial sensing area-specific event detection unit 104 detects movement and presence / absence of the object 202 for each partial sensing area 201. The partial sensing area data accumulation schedule storage unit 105 stores a schedule for each image sensor 101 to sense each partial sensing area 201. The accumulated data selection unit 106 is based on the information of the object 202 detected by the event detection unit 104 for each partial sensing region and the sensing schedule of the partial sensing region 201 stored in the partial sensing region data accumulation schedule storage unit 105. The sensing data of the partial sensing area 201 to be accumulated in the sensing data accumulation unit 107 is determined.

The operation of the image sensing system of the present embodiment configured as described above will be described. FIG. 3 shows information on the entire sensing target area 200 and the target object 202 by sorting out sensing data in the sensing area that is sensed by a plurality of image sensors 101 in accordance with the detection status of the target object 202. It is a flowchart which shows the procedure which accumulate | stores, securing this information. Here, the sensing target region 200 as shown in FIG. 2 is sensed by a plurality of image sensors C _i (i = 1, 2,..., N; n is the number of image sensors 101, here four). A case will be described in which the entire target region 200 is sensed simultaneously at discrete times t (t = 0, 1, 2,...).

(Step S101: Storage of sensing data by image sensor C _i )
Each image sensor 101 senses a sensing area in charge at a predetermined frame rate. Sensing data obtained by sensing by each image sensor 101 is stored in the frame memory 102. In the case of FIG. 2, the areas sensed by the image sensors C ₁ , C ₂ , C ₃ , and C ₄ are a rectangle ACKI, a rectangle BDLJ, a rectangle EGOM, and a rectangle FHPN, respectively. In addition, G _Ci ^{Rk in} FIG. 2 is a node set that overlaps and senses the partial sensing region R ^k (k = 1, 2,..., N ^R ; n ^R is the number of partial sensing regions). The numbers i are stored in ascending order (for example, G _Ci ^R1 = {C ₁ }, G _Ci ^R4 = {C ₁ , C ₃ }, G _Ci ^R5 = {C ₁ , C ₂ , C ₃ , C ₄ } ).

(Step S102: Detect presence / absence of object)
The partial sensing area-specific event detection unit 104 detects the presence or absence of the object 202 for each partial sensing area 201 stored in the partial sensing area storage unit 103, and temporarily stores the result. The detection of the object 202 may be detected by using inter-frame differences or other known image processing methods using sensing data stored in time series in the frame memory 102. Instead of using the sensing data in the frame memory 201, a heat source such as a person may be sensed by installing a detection sensor such as an infrared sensor corresponding to the sensing area of the image sensor.

(Step S103: Select accumulation schedule)
The accumulated data selection unit 106 stores in the partial sensing region storage unit 103 based on the accumulation schedule stored in the partial sensing region data accumulation schedule storage unit 105 and the detection result of the object 202 by the event detection unit 104 by partial sensing region. Sensing data is read from the frame memory 102 and stored in the sensing data storage unit 107 for each of the partial sensing areas 201 that have been set. At this time, if there is no object or the object 202 is detected in the single sensing area 203, the process proceeds to step S104. If the object 202 is detected in the overlapping sensing area 204, the process proceeds to step S105.

(Step S104: Accumulation by data reduction schedule)
The accumulated data selection unit 106 reads out sensing data from the frame memory 102 for each partial sensing area and accumulates it in the sensing data accumulation unit 107 according to a data reduction schedule described later. In the data reduction schedule, the sensing data of the plurality of image sensors 101 sensing the overlapping sensing area is alternately read from the frame memory 102 on the time axis. The data reduction schedule is determined as follows. Assume that the image sensor C _{i in} charge of sensing the region R ^{k at} time t is G _Ci ^Rk (m), (m = 0, 1,...). G _Ci ^Rk (m) represents the (m + 1) th element from the left of the image sensor set G _Ci ^Rk , and m is determined by m = (t) mod (| G _Ci ^Rk |). However, mod represents a remainder operation, and | G _Ci ^Rk | represents the number of elements of the set G _Ci ^Rk , that is, the number of image sensors C _{i in} charge of sensing in the partial sensing region R ^k . m takes an integer value between 0 and (| G _Ci ^Rk | −1). For example, for the region R ⁵ , the set of image sensors 101 that can be sensed is G _Ci ^R5 = {C ₁ , C ₂ , C ₃ , C ₄ }, and when time t = 0, m = 0 and C ₁ When time t = 1, m = 1 and C ₂ , when time t = 2, m = 2 and C ₃ , and when time t = 3, C ₄ is selected with m = 3. Thereafter, when time t = 4 s, m = 0 and C ₁ , when time t = 4 s + 1, m = 1 and C ₂ , when time t = 4 s + 2, m = 2 and C ₃ , and when time t = 4 s + 3, m = 3 in _{C 4} is responsible for sensing. However, s = 0, 1, 2,...

An example of the data reduction schedule is shown in FIG. Here, the total data amount D ^TOTAL (t) read by all the image sensors 101 by time t is obtained by (Equation 1). However, D with R _Ci ^k is superscript represents the total amount of data node C _i is read by time t for the partial sensing region R _ci ^k, quo represents the quotient. Further, the α has a R _Ci ^k is superscript, the imaging device of the node C _i for sensing partial sensing region R ^k (e.g., CCD (Charge Coupled Device)) as a percentage of the image pickup element entire area of the region on is there.

  As shown in FIG. 2, when the four image sensors 101 share 1/4 (25%) or 1/2 (50%) of the sensing area, the data reduction schedule shown in FIG. When the total data amount is calculated by reading by (Equation 1), the read total data amount can be reduced by 43.75% compared to the case where the sensing data of all the frames is always read. This is because when the sensing data of all the frames is always read at each time, the data amount is 16 pieces of the partial sensing area 201. On the other hand, the number of partial sensing areas 201 necessary for sensing the entire sensing target area 200 is nine, and reading of sensing data in the overlapping sensing area is performed according to the data reduction schedule shown in FIG. By performing only the image sensor 101, the data amount for 16−9 = 7 can be reduced at each time, and the reduction rate is 7/16 = 0.4375.

(Step S105: Accumulation by object priority schedule)
In FIG. 3, the accumulated data selection unit 106 reads out sensing data from the frame memory 102 for each partial sensing area and accumulates it in the sensing data accumulation unit 107 according to an object priority schedule described later. In the object priority schedule, when the object 202 exists in the overlapping sensing area, the sensing data of at least two or more image sensors 101 among the plurality of image sensors 101 sensing the overlapping sensing area are simultaneously framed. The data is read from the memory 102. For example, if the sensing data of all the image sensors 101 that are sensing the overlapping sensing area is read, as shown in FIG. 5, the object 202 is in the partial sensing area R between t = 4 and t = 10. ¹ , R ² , R ⁵ , R ⁸ , R ⁹ are moved, the object priority schedule is as shown in FIG. 6 (the portion surrounded by the thick line frame is compared with the data reduction schedule shown in FIG. 4). In addition, a portion for further reading data for securing information on the object 202 is shown). The transition of the data reduction rate at this time is 43.75% (t = 4) ⇒ 43.75% (t = 5) ⇒ 37.5% (t = 6) ⇒ 25.00% (t = 7) ⇒ 37.5% (t = 8) → 43.75% (t = 9) → 43.75% (t = 10). Thereby, it is possible to accumulate the sensing data so as to secure the data of the object from a plurality of viewpoints while reducing the read data amount as compared with the case where the sensing data of all the frames of all the image sensors 101 are constantly read. it can.

  As described above, according to the image sensing system of the present embodiment, sensing data of a region that is sensed by a plurality of image sensors 101 is used as the presence or absence or position of the object 202 in the sensing region. By selecting accordingly, it is possible to reduce the amount of data to be accumulated while securing information on the entire sensing target area and information on the target object 202.

  In the present embodiment, when the object 202 exists in the overlapping sensing area, the sensing data of all the image sensors 101 sensing the overlapping sensing area has been described as an example. When the object 202 is a person and the target is face identification, sensing data without a face may not be accumulated. In addition, even if there is a person in the single sensing area, if the direction of movement is approaching the overlapping sensing area, the sensing data in the overlapping sensing area is pre- The accumulation of sensing data of the image sensor 101 may be started.

  Further, in the present embodiment, all sensing data obtained from each image sensor 101 is temporarily stored in the frame memory 102 on the data accumulation side, and the accumulated sensing data is stored in the partial sensing area according to the presence or absence of the object 202. As described in the example of cutting out from the frame memory 102 every time, the partial sensing region storage unit 103, the partial sensing region-specific event detection unit 104, the partial sensing region data accumulation schedule storage unit 105, and the accumulated data selection unit 106 are provided on the image sensor 101 side. The amount of data read from the image sensor of the image sensor 101 is reduced by communicating the presence / absence and position information of the object in the sensing area by the event detection unit 104 by the partial sensing area between the image sensors 101. Reduce power consumption and transfer of image sensor 101 It goes without saying that can reduce the sensing data amount.

  Moreover, although this Embodiment demonstrated by the example which selects the sensing data of an overlapping sensing area | region based on the presence or absence of the target object 202, or positional information, this invention is not limited to this. In addition to detecting the target object 202, the sensor or sensing area where the user wants to obtain more detailed information is specified by GUI (Graphical User Interface), etc., and the information of the overlapping sensing area is automatically tracked following the specification. You may make it choose to.

  Further, in the present embodiment, in addition, when the plurality of image sensors 101 have a sound recording and transmission function, or when the sound in the sensing area can be recorded and transmitted with another microphone or the like, not only the image data, By controlling the audio data in combination, finer multi-sensor control is possible. For example, it is possible to perform control such that video data is accumulated at a place where the object 202 is detected and audio recording is started. Further, when the object 202 is detected in the overlapping sensing area but it is difficult to store video data in terms of data capacity, only audio data may be stored.

(Embodiment 2)
In the image sensing system according to the first embodiment described above, the schedule for reading sensing data in a region that is sensed redundantly by a plurality of image sensors 101 is changed according to the presence or absence and position of the object 202 in the sensing region. Thus, the image sensing apparatus that reduces the amount of accumulated data while securing information on the entire sensing target area and information on the target object 202 has been described. The image sensing system according to the second embodiment not only changes the data read schedule for each partial sensing area, but also stores the data to be read for each partial sensing area so that the total read data amount becomes a predetermined target value. The degree of detail is also considered to be controlled. Specifically, the plurality of details are set to a plurality of target details so that each partial sensing area is sensed with a plurality of details and the total amount of data read from the plurality of partial sensing areas is a predetermined target value. It controls to be close to

First, a plurality of details in the image sensing system according to the second embodiment will be described with reference to FIG. As shown in FIG. 7, a region where the entire partial sensing region 201 (for example, R ^{1 to} R ⁹ in FIG. 2) is sensed with a predetermined level of detail is referred to as a partial sensing region whole frame (partial sensing region whole frame 701). The predetermined level of detail is referred to as a partial sensing region whole frame detail level, and more specifically, the partial sensing region whole frame horizontal resolution and the number of pixels in the vertical direction, which are the number of pixels in the horizontal direction of the partial sensing region whole frame pixel 703. The partial sensing area is the entire frame vertical resolution.

  In addition, a region in which a partial region in the partial sensing region 201 is sensed with a predetermined level of detail is referred to as a partial sensing region local frame (partial sensing region local frame 702). Further, the predetermined detail level is referred to as a partial sensing area local frame detail level, and more specifically, the partial sensing area local frame horizontal resolution and the vertical direction number of pixels in the horizontal direction of the partial sensing area local frame pixel 704. It is a partial sensing area local frame vertical resolution which is the number of pixels. The partial sensing region local frame 702 is used for sensing the object 202 and the like in detail.

Next, an outline of the image sensing device according to the second embodiment will be described with reference to FIGS. 8 and 2. In the image sensing device of the second embodiment, the partial sensing region whole frame 701 and the partial sensing region local in each partial sensing region are adjusted by adjusting the whole partial sensing region frame detail and the partial sensing region local frame detail. The read data amount by the frame 702 is brought close to each target data amount (partial sensing region whole frame target data amount 801 and partial sensing region local frame target data amount 802) set for each partial sensing region, and FIG. Referring to the adjacency relationship of the partial sensing areas as shown in FIG. 4B, the amount of data read by the partial sensing area whole frame 701 of each partial sensing area and the partial sensing area of the partial sensing area adjacent to the partial sensing area While reducing the difference between the read data amount of the entire frame 701, and at the same time, to control as close to the total amount of data that the total amount of data read out in all parts sensing area becomes a predetermined target (target total data amount 803). At this time, the values of the partial sensing region entire frame target data amount 801 and the partial sensing region local frame target data amount 802 of the partial sensing region including the object 202 are set as the partial sensing region entire frame target data amount 801 not including the object 202. As shown in FIG. 8, the surrounding information (R ¹ , R ² ) of the object itself and the object 202 is not included in the area where the object 202 does not exist (R ⁸ , R ^9, etc.). It can be secured more preferentially.

  That is, the image sensing device according to Embodiment 2 controls not only the data readout schedule of each partial sensing area but also the level of detail to be sensed, thereby reducing the amount of readout data so as to approach the target value, while the object itself In addition, the information amount of the entire neighborhood is secured with the highest priority, and at the same time, information on the entire area around the object by the plurality of image sensors 101 is also preferentially secured compared to the image sensor 101 not including the object 202, This is so considered that the images of the plurality of image sensors 101 can be accumulated.

  FIG. 9 is a block diagram illustrating a schematic configuration of the image sensing system according to the second embodiment. In FIG. 9, the same reference numerals as those in FIG. The image sensing apparatus 900 in FIG. 9 includes a variable detail level accumulated data selection unit 901 instead of the accumulated data selection unit 106 in FIG. 1, and further includes a target data amount setting storage unit 902 and a partial sensing area multiple detail storage unit 903. And a partial sensing area multiple detail calculation unit 904.

  Based on the accumulation schedule stored in the partial sensing region data accumulation schedule storage unit 105 and the detection result of the object 202 by the event detection unit 104 for each partial sensing region, the variable detail degree storage data selection unit 901 is a partial sensing region storage unit. For each partial sensing region 201 stored in 103, sensing data is read from the frame memory 102 and stored in the sensing data storage unit 107 based on the detail stored in the partial sensing region multiple detail storage unit 903. At this time, before reading the sensing data from the frame memory 102, the partial sensing area multiple detail calculation unit 904 is activated to calculate the detail of the sensing data to be read.

  The target data amount setting storage unit 902 stores the partial sensing region whole frame target data amount 801, the partial sensing region local frame target data amount 802, and the target total data amount 803 for all the partial sensing region whole frames and the partial sensing region local frames. Set and remember. For example, when the user sets, it is input and stored from a keyboard (not shown) of the target data amount setting storage unit 902. Further, a predetermined partial sensing area whole frame target data amount 801 and a partial sensing local frame target data amount 802 preset by the manufacturer may be stored.

  Further, partial sensing is proportional to or inversely proportional to the storage capacity of the sensing data storage unit 107, the data reading speed from the frame memory 102, the number of image sensors 101, the number of partial sensing regions 201, the number of objects 202, and the like. The entire region frame target data amount 701 and the partial sensing region local frame target data amount 702 may be dynamically changed. For example, the partial sensing area whole frame target data amount 801 and the partial sensing area local frame target data amount 802 are given in proportion to the storage capacity or the data reading speed, or the partial sensing area whole frame is in inverse proportion to the number of objects to be processed. A target data amount 801 and a partial sensing region local frame target data amount 802 may be given.

  The partial sensing area multiple detail storage unit 903 stores the partial sensing area local frame 702 and the partial sensing area local frame detail for all the image sensors 101 in charge of photographing the partial sensing area 201 where the object 202 is detected. The number of objects 202 is set and stored. Also, the partial sensing area whole frame 701 and the partial sensing area whole frame detail level are set for all the image sensors 101 in charge of photographing the partial sensing area 201 for the partial sensing area 201 in which the object 202 is detected. The partial sensing area 201 in which the object 202 is not detected is set according to the accumulation schedule stored in the partial sensing area data accumulation schedule storage unit 105. These settings will be described with reference to FIG.

Since the partial sensing region R ¹ is a single sensing region of the image sensor C ₁ , the partial sensing region entire frame 701 and the partial sensing region entire frame detail level are always set for the image sensor C ₁ , and the partial sensing region local The frame 702 and the partial sensing area local frame detail level are set to two, which is the number of objects 202. As for the partial sensing area R ^2, because it is redundant sensing region by the image sensor C ₁ and C _2, the image sensor C ₁ and C _2, respectively, partial sensing area local frame 702 and partial sensing area local frame details The partial sensing area whole frame 701 and the partial sensing area whole frame detail level are set.

Furthermore, for the partial sensing area R ^8, is a redundant sensing region by the image sensor C ₃ and C _4, since the object 202 is not present, the entire portion sensing area frame 701 and the partial sensing area across the frame level of detail, partial accordance accumulation schedule stored in the sensing area data storage schedule storage unit 105, it sets alternately to one of the image sensor C ₃ or C _4.

  The partial sensing area multiple detail calculation unit 904 includes a partial sensing area whole frame target data amount 801, a partial sensing area local frame target data amount 802, and a target total data amount 803 set in the target data amount setting storage unit 902, and Based on the adjacency relationship between the partial sensing areas stored in the partial sensing area storage unit 103, the entire frame detail level of the partial sensing area and the local frame detail level are calculated for each partial sensing area 201, and the partial sensing area multiple details are calculated. This is stored in the degree storage unit 903.

Here, an example of the spatial adjacency relationship of the partial sensing area 201 stored in the partial sensing area storage unit 103 is shown. For example, in the example of FIG. 2, part sensing region ^{R 1} is adjacent to the partial sensing region ^{R 2,} R 4, ^{R 5,} part sensing region ^{R 5} is adjacent to the partial sensing region ^R 1 to R ⁹ .

  The partial sensing area multiple detail calculation unit 904 is a part determined by the difference between the partial sensing area local frame data read-out amount determined by the partial sensing area local frame detail level and the local frame target data amount 802 and the entire frame detail level of the partial sensing area. The difference between the sensing area entire frame data read amount and the partial sensing area entire frame target data amount 801 is reduced, and the adjacent relationship of the partial sensing area 201 stored in the partial sensing area storage unit 103 is referred to. While reducing the difference between the partial sensing area whole frame data read amount of the sensing area 201 and the partial sensing area whole frame data read amount of the peripheral partial sensing area of the partial sensing area, The entire partial sensing region in which the difference between the total data read amount in the entire partial sensing region and the target total data amount 803, which is the sum of the partial frame read out amount of the partial sensing region and the local frame data read amount of the partial sensing region, is reduced. The frame detail level and the partial sensing area local frame detail level are calculated.

The operation of the image sensing system of the second embodiment configured as described above will be described using the flowchart of FIG. FIG. 10 is a flowchart showing a procedure for calculating the overall frame detail level of the partial sensing area and the local frame detail level of the partial sensing area 201 shown in FIG. 2 based on the concept shown in FIG. Here, the sensing target area 200 as shown in FIG. 2 is set to a plurality of image sensors C _i (i = 1, 2,..., N; n is the number of image sensors 101, here four), and the degree of detail is set. An example of sensing in consideration will be described.

(Step S1001: storing the sensing data by the image sensor _{C i)}
Each image sensor 101 senses a sensing area in charge at a predetermined frame rate, and stores the sensing data in the frame memory 102. In the case of FIG. 2, the areas sensed by the image sensors C ₁ , C ₂ , C ₃ , and C ₄ are a rectangle ACKI, a rectangle BDLJ, a rectangle EGOM, and a rectangle FHPN, respectively. In FIG. 2, G _Ci ^Rk is a node set that senses the partial sensing region R ^k (k = 1, 2,..., N ^R , n ^R is the number of the partial sensing regions). The node numbers i are stored in ascending order (for example, G _Ci ^R1 = {C ₁ }, G _Ci ^R4 = {C ₁ , C ₃ }, G _Ci ^R5 = {C ₁ , C ₂ , C ₃ , C ₄ }).

(Step S1002: detection of presence or absence of an object)
For each partial sensing region 201 stored in the partial sensing region storage unit 103, the event detection unit 104 for each partial sensing region detects the presence or absence of the object 202, and the event detection unit 104 for each partial sensing region temporarily detects the result. Remember. The detection of the object 202 may be detected by using inter-frame differences or other known image processing methods using sensing data stored in time series in the frame memory 102. Instead of using the sensing data of the frame memory 201, a heat source such as a person may be sensed by installing a detection sensor such as an infrared sensor corresponding to the sensing area of the image sensor 101.

(Step S1003: Setting of frame detail level and local frame detail level of partial sensing area to be calculated based on accumulation schedule)
Based on the accumulation schedule stored in the partial sensing region data accumulation schedule storage unit 105 and the detection result of the object 202 by the event detection unit 104 for each partial sensing region, the variable variable degree of detail data selection unit 901 stores the partial sensing region storage. For each partial sensing area 201 stored in the unit 103, the partial sensing area overall frame 701 and the partial sensing area overall frame detail level, and the partial sensing area local frame 702 and the partial sensing area local frame detail level to be calculated are calculated. This is stored in the partial sensing area multiple detail storage unit 903.

Here, the partial sensing area whole frame horizontal resolution and the partial sensing area whole frame vertical resolution, and the partial sensing area local frame horizontal resolution and partial sensing, which are parameters representing the partial sensing area local frame detail and partial sensing area whole frame detail. The region local frame vertical resolution is stored in the partial sensing region multiple detail storage unit 903. However, if the aspect ratio of the horizontal resolution to the vertical resolution of the entire partial sensing region frame or the partial sensing region local frame is known, only one of the parameters need be stored. Here, the partial sensing region R ^k partial sensing area entire frame horizontal resolution of the w _{k, i} ^F, the partial sensing area local frame horizontal resolution w _{k, h,} and _i ^H. Here, (h = 1, 2,..., M), m is the number of partial sensing area local frames. Also hereinafter, the aspect ratio of the partial sensing region _{^{R k a k, i: b}} k, described as _i.

The level of detail of the partial sensing region local frame 702 is set and stored for all the image sensors 101 in charge of photographing the partial sensing region 201 in which the target object 202 is detected. The level of detail of the entire partial sensing area frame 701 is set for all of the image sensors 101 in charge of photographing the partial sensing area 201 for the partial sensing area 201 in which the object 202 is detected. About the partial sensing area | region 201 in which 202 is not detected, it sets according to the accumulation schedule memorize | stored in the partial sensing area data accumulation schedule memory | storage part 105. FIG. This setting will be described with reference to FIG. Since the partial sensing region R ¹ is a single sensing region of the image sensor C ₁ , the partial sensing region entire frame 701 and the partial sensing region entire frame detail level are always set for the image sensor C ₁ , and the partial sensing region local The frame 702 and the partial sensing area local frame detail level are set to two, which is the number of objects 202.

As for the partial sensing area R ^2, because it is redundant sensing region by the image sensor C ₁ and C _2, the image sensor C ₁ and C _2, respectively, partial sensing area local frame 702 and partial sensing area local frame details The partial sensing area whole frame 701 and the partial sensing area whole frame detail level are set.

Furthermore, for the partial sensing area R ^8, is a redundant sensing region by the image sensor C ₃ and C _4, since the object 202 is not present, the entire portion sensing area frame 701 and the partial sensing area across the frame level of detail, partial In accordance with the accumulation schedule stored in the sensing area data accumulation schedule storage unit 105, the image sensor C ₃ or C ₄ is alternately set.

(Step S1004: setting of target data amount)
By using the target data amount setting storage unit 902, the partial sensing region whole frame target data amount 801, the partial sensing region local frame target data amount 802, and the target total data amount 803 are set for all the partial sensing region whole frames and the partial sensing region local frames. Set and remember.

  FIG. 11 shows a specific setting example of the partial sensing area whole frame target data amount 801 and the partial sensing area local frame target data amount 802. In FIG. 11, when the data amount of each partial sensing area 201 as a whole is 1, a partial sensing area whole frame target data amount 801 can be expressed as, for example, ¼. Actually, if the data of the image sensor 101 is a data amount of 1600 × 1200 (horizontal resolution × vertical resolution) pixels, the data amount of one partial sensing area is 800 × 600 pixels, for example, 1/2 of that. The data amount of 400 × 600 pixels or 1/4 of 400 × 300 pixels is given as the frame target data amount for the entire partial sensing area.

  In the example of FIG. 11, in order to further secure the amount of information around the area where the object 202 exists, the partial sensing area whole frame target data amount 801 of the partial sensing area where the object 202 exists is used as the partial sensing where the object 202 does not exist. The partial sensing area of the area is set to be higher than the entire frame target data amount 801. As for the partial sensing area local frame target data amount 802, the partial sensing area local frame 702 is provided so as to include the area of the object 202, but 80% (0.8 of the area of the object 202 having the assumed size). Double the amount of data).

  Alternatively, for example, a data amount of 120% (1.2 times) may be specified to further secure detailed information on the area near the object 202. As specific values, for the partial sensing area whole frame target data amount 801, the minimum data amount necessary to detect the moving object 202 or the like is set to 320 × 240 pixels or 160 × 120 pixels, for example. Give it.

  The partial sensing area local frame target data amount 802 depends on the application, such as the amount of data that can identify someone when the human sees the image data, or the amount of data required by the face authentication device, etc. Thus, for example, 256 × 256 pixels or 128 × 128 pixels may be provided. The partial sensing area entire frame target data amount 801 and the partial sensing area local frame target data amount 802 set in this way are just target values, and the data amount actually read from the frame memory 102 is the details of the partial sensing area entire frame details. Degree and partial sensing area depends on local frame detail.

  The partial sensing area overall frame detail level and the partial sensing area local frame detail level are close to the value of the target total data amount 803, which is the target value of the total amount of data read from all the partial sensing regions, in step S1005 described below. The value is calculated by the partial sensing area multiple detail calculation unit 904 corresponding to the increase / decrease of the number of objects 202 so as to be a value. Regarding the target total data amount 803, the storage capacity of the sensing data storage unit 107, the storable data transfer rate, the reading speed from the frame memory 102, and the image data are monitored by the monitor when the present invention is used as a monitoring device. Are set in consideration of the minimum amount of data required for the above, the minimum amount of data required for detection of the moving object and face recognition described above.

For example, in FIG. 11, the total data amount read from the entire partial sensing area when the partial sensing area achieves the total frame target data amount 801 and the partial sensing area local frame target data amount 802 is the total data of the partial sensing area 201. about becomes 3.5 pieces of (for partial sensing region R ⁴ is an image sensor C ₁ and the object 202 a overlap sensing region of the C ₂ is present, partial sensing each image sensor C ₁ and C ₂ of Since the entire region frame and the partial sensing region local frame are allocated), if the target total data amount 803 is one partial sensing region 201, the partial sensing is performed in step S1005 described below so as to satisfy the restriction. The region multiple detail calculation unit 904 receives each partial sense Calculating the entire partial sensing area frame granularity and partial sensing area local frame detail of the region.

(Step S1005: Calculation of the partial frame detail level of the partial sensing area and the local frame detail level of the partial sensing area)
The detail variable storage data selection unit 901 activates the partial sensing region multiple detail calculation unit 904 after the completion of steps S1003 and S1004. The partial sensing area multiple detail calculation unit 904 includes a partial sensing area whole frame target data amount 801, a partial sensing area local frame target data amount 802, and a target total data amount 803 set in the target data amount setting storage unit 902, and The part set for each partial sensing region 201 based on the accumulation schedule stored in the partial sensing region data accumulation schedule storage unit 105 based on the adjacent relationship of the partial sensing regions stored in the partial sensing region storage unit 103 The total sensing frame local resolution and the partial sensing domain local frame detail are set to the following (Equation 2) to minimize the partial sensing domain whole frame horizontal resolution w _{k, i} ^F and the partial sensing domain local frame horizontal resolution w _{k, h. , i} ^H is calculated. However, the aspect ratio of each image sensor C _{i in} the partial sensing region R ^k is a _{k, i} : b _{k, i} . In (Expression 2), α ₁ , α ₂ , β, γ, and ε are weighting factors of the first to fifth terms, respectively, and control the trade-off of the five terms.

The first term and the second term of (Equation 2) are respectively the total frame target data amount F _{k of the} image sensor C _i in which the object 202 is detected in the partial sensing region R ^k and the partial sensing region local frame is set. _{, i} ^M and the partial sensing area whole frame horizontal resolution is w _{k, i} ^F , the difference of the partial sensing area whole frame data amount is reduced, and the image sensor C _{j in} which the partial sensing area local frame is not set This represents that the difference between the partial sensing area whole frame target data amount F _{k, j} ^NM and the partial sensing area whole frame data amount when the horizontal resolution of the partial sensing area whole frame is w _{k, j} ^F is reduced. At this time, by setting the value of F _{k, i} ^M to be larger than F _{k, j} ^{NM and} the value of α ₁ to be larger than α ₂ , the entire partial sensing area frame of the sensing area in which the object 202 is included The level of detail can be increased and a larger amount of information can be secured. F _{k, i} ^M is desirably the data amount of the entire partial sensing area, but the target total data amount 803 is largely limited, and importance is placed on the partial sensing area local frame data amount, that is, securing detailed data in the object area. In this case, a target value with a reduced data amount, such as 1/4 or 1/16 of the actual frame size, is set.

The third term of (Expression 2) represents that the difference in the data amount of the entire partial sensing area frame generated in the adjacent partial sensing areas is reduced. That is, the resolution of the entire frame in the neighboring sensing area is made uniform. As a result, the amount of data read from the image sensor 101 sensing the periphery where the object 202 exists is relatively increased, and the image sensor 101 sensing the periphery of the area where the object 202 does not exist is read. Since the amount of data is relatively low, information on a wider peripheral area where the object 202 exists can be secured more. Here, representing the S _p is a set of partial sensing region adjacent to the partial sensing region R ^p in (Equation 2). Further, the adjacency relationship between the imaging areas represents spatial adjacency of the imaging area of the sensor. For example, an area within a certain distance range may be the adjacent area, or the distance may be separated, but Topologically adjacent ones such as both ends may be designated, or the user may arbitrarily set them.

The fourth term of (Expression 2) is the difference between the partial sensing region local frame target data amount H _h and the partial sensing region local frame data amount when the partial sensing region local frame horizontal resolution is w _{k, h, i} ^H. Represents a reduction in. By making the value of γ larger than other weighting factors, the information of the object itself can be secured most, for example, the resolution equal to the area of the local frame is secured, and further, the data more than the area of the object By setting the amount to the partial sensing area local frame target data amount, it is possible to secure information with high resolution up to the detailed information of the vicinity of the object 202.

  The partial sensing area local frame can be manually set not only to secure the information amount of the object 202 but also to a specific area in a statically important sensing area. In the case of securing information on the object 202, the size of the object 202 is set as a reference. For example, when setting a target value of 1.0 times or more, such as 2.0 times, with respect to the reference size, all the data of the area of the object 202 is secured and information about the surrounding area is also acquired. (See the partial sensing area local frame target data amount 802 in FIG. 11). When the data amount is largely suppressed, when a target value less than 1.0 such as 0.5 is set, the information is thinned out with respect to the size of the detected object 202.

The fifth term of (Expression 2) represents that the difference between the target total data amount D ^TOTAL and the total read data amount in all partial sensing areas is reduced. The target total data amount D ^TOTAL is given in consideration of the capacity of the recorder that accumulates the image data of the plurality of image sensors 101, the data transfer rate, and the like.

The calculation of the partial sensing area whole frame horizontal resolution w _{k, i} ^F and the partial sensing area local frame horizontal resolution w _{k, h, i} ^H minimizing (Equation 2) is simply the number of horizontal pixels in the partial sensing area. Is an integer value of 0 to 800 for the partial partial sensing region whole frame horizontal resolution w _{k, i} ^F , and 0 to 256 for the partial sensing region local frame horizontal resolution w _{k, h, i} ^R. Assuming an integer value, (Equation 2) is calculated for the combination of the horizontal resolution of all partial sensing areas and the horizontal resolution of all partial sensing areas and the local resolution of all partial sensing areas and m local frames. The combination of w _ki ^F and w _{k, h, i} ^R that minimizes the value may be selected.

Here, if this is a range where the possible integer values are as small as 0 to 30, such as optimization of the frame rate, a calculation method of obtaining the minimum value for all the combinations in this way may be used. There is no problem in particular when sufficient time is available for the optimization calculation at the time of design. However, when a wide range of image data from a large number of image sensors 101 is read out in a short time, the calculation of the partial frame horizontal resolution w _ki ^F for each partial sensing area is performed by calculating the own total frame target data amount F _{k, i.} ^M , partial sensing area local frame horizontal resolution w _{k, h, i} ^H in own sensing area, partial sensing area whole frame horizontal resolution and partial sensing area local frame horizontal resolution adjacent to own partial sensing area, target total Considering only the data amount D ^{TOTAL, the} horizontal resolution w _ki ^F of the partial sensing area of (Equation 2) is regarded as a continuous value, and the gradient obtained by partial differentiation of (Equation 2) shown in (Equation 3) with w _ki ^F and the initial value of the appropriate w _ki ^F set by the random number, to sequentially updated in appropriate step width (e.g. 0.001, etc.) based on Te, can be obtained at high speed the approximate solution of w _ki ^F satisfying (Equation 2).

The partial sensing area whole frame horizontal resolution w _{k, j} ^F where the partial sensing area local frame is not set and the partial sensing area local frame horizontal resolution w _{k, h, i} ^H are also (Equation 4) and (Equation 5), respectively. Can be calculated based on the gradient shown in FIG.

FIG. 12 shows an example of the data amounts of the entire partial sensing region frame and the partial sensing region local frame when the target total data amount D ^TOTAL is 2 (when the data amount for one partial sensing region is 1).

(Step S1006: Data accumulation based on detail level)
The degree of detail variable accumulation data selection unit 901 includes a partial sensing region multiplexing detail for each of the accumulation schedule stored in the partial sensing region data accumulation schedule storage unit 105 and the partial sensing region 201 stored in the partial sensing region storage unit 103. Based on the degree of detail stored in the degree storage unit 903, the sensing data is read from the frame memory 102 and accumulated in the sensing data accumulation unit 107.

  As described above, according to the image sensing system of the second embodiment, the amount of read data is reduced to approach the target value by controlling not only the data read schedule of each partial sensing area but also the level of detail to be sensed. However, the information amount of the object itself and the entire vicinity thereof is secured with the highest priority, and at the same time, the information on the entire area around the object 202 by the plurality of image sensors 101 is given priority over the image sensor 101 that does not include the object 202. Therefore, the images of the plurality of image sensors 101 can be efficiently accumulated.

  In the second embodiment, the example has been described in which the object 202 is detected to increase the detail level of the area of the object itself and its surrounding area, and the detail level of the area away from the area is decreased. Is not limited to this. In addition to the detection of the target object 202, the user specifies a sensor or region where more detailed information is to be obtained, or the user directly manipulates the level of detail of an arbitrary partial sensing region, and the adjacent region The level of detail may be automatically adjusted. In this way, it is needless to say that an easy-to-use monitoring system can be realized in which the user does not have to directly manipulate the level of detail of the plurality of partial sensing areas and can easily secure the amount of information around the area of interest. Absent. An operation for designating the sensor, area, and level of detail by the user may be directly input by the user through a GUI or the like.

  In the second embodiment, the example in which the image sensing device 900 centrally controls the plurality of image sensors 101 has been described. However, each image sensor 101 includes the image sensing device 900 and is directly connected to the image sensor via a network. It goes without saying that the 101 can directly send and receive information on the sensing area to identify the adjacent shooting area, and calculate and control the degree of detail of each partial sensing area according to the detection state of the object and abnormality. This has the effect of reducing the driving power consumption of the sensor 101 and the amount of transferred data, and can control more image sensors 101 autonomously than the centralized management method, and can also be used for the time required for tracking an object. The effect of improving the responsiveness such as a decrease in delay can be obtained.

  The present invention can be applied to a remote monitoring system using a plurality of image sensors, particularly in an environment where the transfer rate of image data and the storage data capacity are limited, while giving priority to the imaging region where the object is detected. The present invention can be applied to a device that simultaneously monitors the entire monitoring target area or a device that accumulates monitoring video.

DESCRIPTION OF SYMBOLS 100 Image sensing apparatus 101 Image sensor 102 Frame memory 103 Partial sensing area memory | storage part 104 Event detection part according to partial sensing area 105 Partial sensing area data accumulation schedule memory | storage part 106 Accumulated data selection part 107 Sensing data accumulation part 202 Target object 900 Image sensing apparatus 901 Detail level variable accumulation data selection unit 902 Target data amount setting storage unit 903 Partial sensing region multiple detail level storage unit 904 Partial sensing region multiple detail level calculation unit

Claims (4)

An image sensing device composed of a plurality of image sensors communicating with each other through a network,
The plurality of image sensors hold an overlapping sensing area that is sensed redundantly by the plurality of image sensors and a single sensing area that is sensing alone,
When an object is detected in the overlapping sensing area in the overlapping sensing area, each of the image sensors reads image data for the overlapping sensing area,
When there is no object in the overlapping sensing area, the plurality of image sensors communicate with each other so that at least one image sensor reads the image data of the selected image sensor with respect to the overlapping sensing area,
The image sensing device is characterized in that the plurality of image sensors always read out image data of the single sensing region. An image sensing device comprising a plurality of image sensors and a sensing data storage unit that communicate with each other through a network,
The plurality of image sensors hold an overlapping sensing area that is sensed redundantly by the plurality of image sensors and a single sensing area that is sensing alone,
When an object is detected in the overlapping sensing area in the overlapping sensing area, each of the image sensors reads image data for the overlapping sensing area and transmits it to the sensing data storage unit,
When there is no object in the overlapping sensing area, the plurality of image sensors communicate with each other so that at least one image sensor reads the image data of the selected image sensor with respect to the overlapping sensing area, and Sent to the sensing data storage,
The plurality of image sensors may constantly read image data in the single sensing area and transmit the image data to the sensing data storage unit.   3. The image sensor according to claim 1, wherein when there is no object in the overlapping sensing area, at least one image sensor performs sensing of the overlapping sensing area at a predetermined interval. The image sensing device described in 1. The image sensor senses a part of at least one of the overlapping sensing area and the single sensing area at a first level of detail when sensing the overlapping sensing area or the entire single sensing area. The second level of detail close to the first target detail level and the second target detail level,
The image sensor is configured such that when a part of a peripheral region of the sensing region in which the image sensor senses the entire overlapping sensing region or the single sensing region is sensed with the second level of detail by another image sensor, the image sensor Bringing the first level of detail of the sensor closer to the first level of detail of the other image sensor;
4. The first and second detail levels are controlled such that a total amount of read data per unit time from the plurality of sensing areas becomes a predetermined target value. 5. The image sensing device described.

Images