JP7612353B2 - Image capture device, image capture system, and method for controlling image capture device - Google Patents

Description

The present invention relates to an imaging device, an imaging system, and a control method and program for an imaging device.

In recent years, surveillance systems using network cameras have become widespread. Network cameras are used in a wide range of fields, such as as surveillance cameras in large public institutions and mass retailers, and there are various ways of operating them. In addition, there are network cameras with various functional features to suit the operating form. Specifically, there are network cameras that can freely change the shooting direction by panning and tilting, and omnidirectional cameras that can monitor a wide area at once, and surveillance is carried out by installing multiple cameras of these.
Some surveillance systems using installed cameras have a function of protecting privacy by masking a portion of the image captured by the camera so that it cannot be seen. Patent Document 1 discloses a surveillance system that has a function of storing setting information for whether or not to perform masking for each camera and displaying a masked image and an unmasked image separately on a monitor.

JP 2017-228854 A

The conventional technology disclosed in Patent Document 1 above proposes a method for easily setting whether or not to perform mask processing on each of multiple different types of captured images when the captured images are divided and displayed on a monitor.
However, with the above method, although masking is performed on the image from a camera that has a mask area set, there is a problem that when other cameras that do not have a mask area set are driven to pan or tilt in the direction of the mask area, they still capture the image without being masked.

The present invention aims to prevent the problem of a masked area being photographed and made public by another device when a masked area is set in video captured by a certain device.

The imaging device of the present invention is an imaging device that constitutes an imaging system together with an external device that sets a mask area that performs mask processing in a captured image, and has an imaging means, a judgment means that judges whether or not at least a portion of the mask area is included in the image acquired by the imaging means based on information related to the mask area set by the external device, and a restriction means that restricts the shooting operation when the judgment means judges that at least a portion of the mask area is included in the acquired image, and is characterized in that the restriction means changes a method of restricting the shooting operation depending on whether or not automatic tracking that tracks a specific subject is being executed .

The present invention makes it possible to prevent the problem that, when a mask area is set in an image captured by a certain device, the mask area is captured by another device and made public.

FIG. 1 is a diagram illustrating an example of a system configuration according to a first embodiment. 1 is a block diagram showing an example of a functional configuration of an imaging system according to a first embodiment. 1 is a schematic diagram showing an example of a camera installation configuration of an imaging system according to a first embodiment and an example of an area of a privacy mask set on a wide-angle surveillance camera. 10 is a diagram showing an example of a setting screen for setting a mask area in the wide-angle surveillance camera 102 according to the first embodiment. FIG. FIG. 2 is a schematic diagram showing a mask area according to the first embodiment as a three-dimensional region. FIG. 4 is a diagram illustrating an example of mask area information in a table according to the first embodiment. 10 is a flowchart illustrating an example of a process for transmitting mask area information between cameras according to the first embodiment. 10 is a flowchart showing an example of the flow of a restriction process according to the first embodiment. 10 is a flowchart showing an example of a flow of a restriction execution process according to the first embodiment; 4 is a diagram showing an example of a restriction pattern priority table used for selecting a video restriction method according to the first embodiment; FIG. 5A to 5C are schematic diagrams showing an example of images from both cameras when the image total blackening process according to the first embodiment is executed; 5A to 5C are schematic diagrams showing the movement of a PTZ type surveillance camera when a restriction is executed to stop pan, tilt and zoom control according to the first embodiment. 10 is a schematic diagram showing an example of how a warning is displayed on an image captured by another camera when a mask area according to the first embodiment is set; FIG. 11 is a schematic diagram showing an example of presenting a reason why a display restriction has occurred when video restriction is executed in the PTZ type surveillance camera according to the first embodiment; FIG. 13 is a flowchart showing an example of a process for transmitting mask area information between cameras according to the second embodiment. 13 is a flowchart illustrating an example of a process for transmitting mask area information between cameras according to the third embodiment.

(First embodiment)
A first embodiment of the present invention will now be described with reference to FIGS.
FIG. 1A is a diagram showing an example of an operational configuration of an imaging system according to this embodiment.
The imaging system in this embodiment is composed of a PTZ (Pan Tilt Zoom) type surveillance camera 101 having pan, tilt and zoom mechanisms, a wide-angle type surveillance camera 102, a client device 103, and a PTZ type surveillance camera 104. The PTZ type surveillance camera 101, the wide-angle type surveillance camera 102, the client device 103, and the PTZ type surveillance camera 104 are connected to a network 105 and are capable of communicating with each other.
The network 105 is not particularly limited as long as it enables communication between the PTZ type surveillance camera 101, the wide-angle type surveillance camera 102, the client device 103, and the PTZ type surveillance camera 104. In addition, although two PTZ type surveillance cameras are used in this embodiment, one camera may be installed, or three or more cameras may be installed.

Next, the hardware configuration of the imaging system in this embodiment will be described with reference to Fig. 1B. Note that the hardware configuration of the PTZ type surveillance camera 104 is similar to that of the PTZ type surveillance camera 101, so the hardware configuration of the PTZ type surveillance camera 101 will be described below as a representative example.
The PTZ type surveillance camera 101 has a CPU 11, a ROM 12, a RAM 13, an input I/F 16, and a communication unit 17. The wide-angle type surveillance camera 102 has a CPU 21, a ROM 22, a RAM 23, an input I/F 26, and a communication unit 27. The client device 103 has a CPU 31, a ROM 32, a RAM 33, a HDD 34, a display unit 35, an input I/F 36, and a communication unit 37.
The CPU 11 reads out a control program stored in the ROM 12 and executes various processes. The same applies to the CPUs 21 and 31. The ROM 12 is a rewritable non-volatile memory, and also serves to store various data and various programs. The same applies to the ROM 22. The RAM 13 is used as a temporary storage area such as the main memory and work area of the CPU 11. The same applies to the RAMs 23 and 33. The HDD 34 stores various data and various programs. The display unit 35 displays various information. The display unit 35 may be a display device integrated with a touch panel. The input I/F 16 is an interface for inputting operation information. The same applies to the input I/Fs 26 and 36. The communication unit 17 performs communication processing with an external device such as a wide-angle surveillance camera 102 via a network, either wired or wirelessly. The same applies to the communication units 27 and 37.

The functions and processing of the PTZ surveillance camera 101, which will be described later, are realized by the CPU 11 reading out a program stored in the ROM 12 and executing this program. As another example, the CPU 11 may read out a program stored in a recording medium such as an SD card instead of the ROM 12. The same applies to the functions and configurations of the wide-angle surveillance camera 102 and the client device 103.

In this embodiment, one processor (CPU) uses one memory (ROM) to execute each process shown in the flowcharts described below, but other configurations are also possible. For example, multiple processors, multiple RAMs, ROMs, and storages can work together to execute each process shown in the flowcharts described below. Also, some processes may be executed using hardware circuits. Also, the functions and processes described below may be realized using a processor other than a CPU (for example, a GPU (Graphics Processing Unit) may be used instead of a CPU).

FIG. 2 is a block diagram showing an example of the functional configuration of the imaging system according to this embodiment.
The PTZ type surveillance camera 101 includes an imaging unit 201, an image processing unit 202, a system control unit 203, a pan driving unit 204, a tilt driving unit 205, a zoom driving unit 206, a pan/tilt/zoom control unit 207, a restriction processing unit 208, a storage unit 209, and a communication unit 210. The PTZ type surveillance camera 104 also includes functions similar to those of the PTZ type surveillance camera 101.
The image capturing unit 201 is configured with a lens and an image sensor, captures an image of a subject, converts the image into an electrical signal, and transmits the electrical signal to the image processing unit 202.
The image processing unit 202 performs image processing and compression encoding processing on the electrical signal converted by the imaging unit 201 to generate image data, and transmits the image data to the system control unit 203 .
The pan drive unit 204 is made up of a mechanical drive system that performs panning operations, a motor as the drive source, and an angle sensor that detects the angle of the pan drive unit 204, and when the motor is driven, the pan drive mechanism drives in the pan direction.
The tilt drive unit 205 is made up of a mechanical drive system that performs tilt operations, a motor as the drive source, and an angle sensor that detects the angle of the drive unit. When the motor is driven, the tilt drive mechanism drives in the tilt direction.

The zoom driver 206 includes a focus lens, a driver and a motor for the zoom lens, and a sensor for detecting the position of the zoom lens, and when the motor is driven, the zoom lens moves in the optical axis direction to change the focal length. When the motor is driven, the focus lens moves in the optical axis direction to perform focus adjustment.
The pan/tilt/zoom control unit 207 as a control means controls the pan driving unit 204, tilt driving unit 205, and zoom driving unit 206 based on an instruction signal transmitted from the system control unit 203. The pan/tilt/zoom control unit 207 can also analyze sensor information of the pan driving unit 204, tilt driving unit 205, and zoom driving unit 206 to obtain information on the pan angle, tilt angle, and zoom angle. The pan/tilt/zoom control unit 207 transmits information on the pan angle, tilt angle, and zoom angle to the system control unit 203 based on an instruction signal transmitted from the system control unit 203.

The system control unit 203 transmits image data generated by the image processing unit 202 to the client device 103 via the communication unit 210. The system control unit 203 also analyzes a camera control command received from the client device 103 or the wide-angle surveillance camera 102 via the communication unit 210, and performs processing according to the command. For example, the system control unit 203 instructs the image processing unit 202 to set image quality adjustment, and the pan/tilt/zoom control unit 207 to set shooting parameters such as pan, tilt, and zoom operations. The system control unit 203 can also set an automatic tracking processing flag to ON for the pan/tilt/zoom control unit 207 when tracking of a subject is instructed from the client device 103 or the like. When the automatic tracking processing flag is ON, the pan/tilt/zoom control unit 207 controls pan, tilt, and zoom to track the subject.
The restriction processing unit 208 acquires mask area information from a camera in which a mask area, which is a feature of this embodiment, is set, and performs restriction processing required to realize restrictions on the camera itself. Specifically, the restriction processing unit 208 performs restriction processing on shooting operations such as restricting pan, tilt, and zoom driving, and restricting captured images.
The storage unit 209 stores setting values such as image quality adjustment parameters and network settings, etc. Even if the device is rebooted, it is possible to start up using the previously set values.
A communication unit 210 corresponds to the communication unit 17 in FIG. 1B, and transmits and receives camera control commands, image data, and the like.

Next, the wide-angle surveillance camera 102 includes an imaging unit 211 , an image processing unit 212 , a system control unit 213 , a mask processing unit 214 , a storage unit 215 , and a communication unit 216 .
The imaging unit 211 is composed of a lens and an imaging element. The imaging unit 211 captures an image of a subject, converts the image into an electrical signal, and transmits the signal to the image processing unit 212. The imaging unit 211 also includes a wide-angle lens having a hemispherical viewing angle.
The image processing unit 212 performs image processing and compression encoding processing on the electrical signals converted by the imaging unit 211 to generate image data, and transmits the image data to the system control unit 213 .
The system control unit 213 distributes image data generated by the image processing unit 212 to the client device 103 via the communication unit 216. The system control unit 213 also analyzes camera control commands received from the client device 103, the PTZ type surveillance camera 101, and the PTZ type surveillance camera 104 via the communication unit 216, and performs processing according to the command. For example, the system control unit 213 instructs the image processing unit 212 to set shooting parameters such as image quality adjustment settings.
The mask processing unit 214 sets a mask area in the captured image and performs mask processing on the captured image for the set area. The mask processing unit 214 also performs processing to transmit mask area information to other cameras in cooperation with the mask processing unit 214 via the communication unit 216.
The storage unit 215 stores setting values such as image quality adjustment parameters and network settings, etc. Even if the device is rebooted, it is possible to start up using the previously set values.
A communication unit 216 corresponds to the communication unit 27 in FIG. 1B, and transmits and receives camera control commands, image data, and the like.

Next, the client device 103 is an information processing device including a display unit 217 , an input unit 218 , a system control unit 219 , a communication unit 220 , and a storage unit 221 .
The display unit 217 is a liquid crystal display device or the like equivalent to the display unit 35 in FIG. 1B, and displays image data received from the PTZ type surveillance camera 101, the wide-angle type surveillance camera 102, and the PTZ type surveillance camera 104. The display unit 217 also displays a graphic user interface (hereinafter referred to as GUI) or the like for controlling the PTZ type surveillance camera 101, the wide-angle type surveillance camera 102, and the PTZ type surveillance camera 104. The camera control is, for example, control for setting the zoom ratio of the PTZ type surveillance camera 101 and the PTZ type surveillance camera 104, and control for changing the pan/tilt direction. The camera control also includes control for instructing the PTZ type surveillance camera 101, the PTZ type surveillance camera 104, and the wide-angle type surveillance camera 102 to capture images, etc.
The input unit 218 as a designation means is a pointing device such as a keyboard or a mouse, and the user of the client device 103 operates the input unit 218 to operate the GUI via the input I/F 36 .
The communication unit 220 corresponds to the communication unit 37 in FIG. 1B, and transmits and receives camera control commands, image data, and the like.
The storage unit 221 stores information such as the IP address of the camera, settings such as network settings, etc. Even if the camera is rebooted, it is possible to start up using the previously set values.

Next, the installation configuration of the cameras of the imaging system according to this embodiment and the mask settings of each camera will be described with reference to FIG.
3 is a schematic diagram showing the installation configuration of the cameras in this system and the mask area set in the wide-angle surveillance camera 102. The PTZ surveillance camera 101, the PTZ surveillance camera 104, and the wide-angle surveillance camera 102 are capable of communicating with each other and with the client device 103 via the network 105. A mask area 301 is set in the wide-angle surveillance camera 102, and the mask area 301 in the captured video is subjected to mask processing such as filling in or mosaic processing on the entire mask area 301 or the area of the subject 302 by the mask processing unit 214. In this way, the subject 302 cannot be identified from the captured video by being masked in the captured video by setting the mask area 301.
On the other hand, no mask area is set for the PTZ surveillance cameras 101 and 104. Therefore, when the direction of the subject 302 is photographed by panning, tilting, and zooming, the subject 302 is photographed without being masked. However, by performing the restriction process shown in the flow chart of Fig. 9, which will be described later, it is possible to avoid a situation in which the subject 302 is photographed without being masked.

Next, the setting of the mask area will be described with reference to FIG.
4 is a diagram showing an example of a setting screen for setting a mask area in the wide-angle surveillance camera 102. An image 401 is an image being captured by the wide-angle surveillance camera 102, and is displayed on the display unit 217 of the client device 103 via the network 105. A mask area 301 is a region of the mask area set by the user, and is set by the user specifying a range by dragging a mouse pointer 402 from the input unit 218 of the client device 103. Information on the set mask area 301 is transmitted from the communication unit 220 of the client device 103 to the wide-angle surveillance camera 102, received by the communication unit 216 of the wide-angle surveillance camera 102 via the network 105, and stored in the storage unit 215.
In the present embodiment, an example has been shown in which the mask area 301 is set as a fixed area, but the mask area may be an area that changes dynamically in accordance with the area of the subject being photographed.

Next, the mask area information will be described with reference to FIG. 5 and FIG.
Fig. 5 is a schematic diagram illustrating the mask area 301 set by the wide-angle surveillance camera 102 as a three-dimensional area. The mask area 301 set by the input unit 218 described above in Fig. 4 is two-dimensional information because the mask area is set in the two-dimensional image captured by the wide-angle surveillance camera 102, but the area that is actually masked is spread out in the depth direction as shown in Fig. 5.
Three-dimensional coordinates 501 of wide-angle surveillance camera 102 are already stored in storage unit 215 of wide-angle surveillance camera 102, and the vertex coordinates on the floor surface when mask area 301 is projected from the three-dimensional coordinates 501 onto the floor surface are coordinates 502, 503, 504, and 505. A quadrangular pyramid area consisting of a total of five vertices, the three-dimensional coordinates 501 and the four vertex coordinates 502 to 505 on the floor surface, becomes the three-dimensional area to be masked. Information on these five vertices becomes part of the mask area information and is used as information to be transmitted to other cameras.

The three-dimensional coordinates 501 of the wide-angle surveillance camera 102 stored in the storage unit 215 may be set in advance as three-dimensional coordinates with a certain point on the floor as the origin when the camera is installed. A method for setting the three-dimensional position of the camera may include a method in which a two-dimensional marker indicating the origin coordinate is installed on the floor and photographed from the camera to obtain the three-dimensional coordinates of the camera. Alternatively, a camera installation map may be created using a tool such as 3D-CAD before the camera is installed, and the three-dimensional coordinates may be obtained using the tool and input to the camera. Either method may be used.
In addition, the vertex coordinates on the floor surface when the mask area 301 is projected onto the floor surface from the three-dimensional coordinates 501 can be calculated using trigonometric functions and vectors pointing toward each vertex of the mask area, the camera's three-dimensional coordinates, and height information from the floor.

Next, FIG. 6 is a diagram showing an example of mask area information in the form of a table.
In the mask area information table 600, first, character strings of the model name and serial number are stored in a "Model Name" item 601 and a "Serial No" item 602 as main body identification information of the wide-angle surveillance camera 102. Furthermore, coordinate values of five vertices are stored in items 603 to 607 of "Coordinates_A" to "Coordinates_E" as the mask area information described above in Fig. 5. The mask area information is stored in the storage unit 215 of the wide-angle surveillance camera 102. By transmitting the values stored in this mask area information table 600 to other cameras, it is possible to indicate to the other cameras the area to be masked by the wide-angle surveillance camera 102.

Next, an example of a process for transmitting mask area information from the wide-angle surveillance camera 102 to another camera will be described with reference to the flowchart of FIG.
First, the process in the wide-angle surveillance camera 102 will be described.
In step S701, the system control unit 213 calculates the coordinate values of the mask area as described above with reference to FIG. 5, and stores the calculated coordinate values in the mask area information table 600 described with reference to FIG.
Next, in step S702, the system control unit 213 transmits the mask area information, whose coordinate values have been stored in step S701, to the PTZ type surveillance camera 101 via the communication unit 216. In this embodiment, an example is shown in which the mask area information is transmitted to the PTZ type surveillance camera 101, but if the PTZ type surveillance camera 104 is also a link target, the mask area information is transmitted to the PTZ type surveillance camera 104 as well.

Next, in step S703 on the PTZ surveillance camera 101 side, the system control unit 203 waits for reception of mask area information from the wide-angle surveillance camera 102.
In step S704, the system control unit 203 determines whether or not mask area information has been received from the wide-angle surveillance camera 102. If the system control unit 203 determines that mask area information has been received, the process proceeds to step S705. On the other hand, if the system control unit 203 determines that no mask area information has been received, the process returns to step S703 and periodically waits for reception.
In step S705, the restriction processing unit 208 performs restriction processing based on the mask area information received from the wide-angle surveillance camera 102. The received mask area information is stored in the storage unit 209. If the PTZ surveillance camera 104 is also a link target, the PTZ surveillance camera 104 also performs restriction processing according to a similar processing flow.

Next, the restriction process by the restriction processor 208 will be described with reference to FIGS.
FIG. 8 is a flowchart showing an example of a detailed flow of the restriction process by the restriction processing unit 208 in the PTZ type surveillance camera 101 shown in step S705 of FIG.
First, in step S801, the restriction processing unit 208 refers to the mask area information stored in the storage unit 209.
Next, in step S802, the restriction processing unit 208 calculates the coordinates of the vertices of the shooting space area currently being shot by the PTZ surveillance camera 101. Note that the system control unit 203 acquires information on the current pan angle, tilt angle, and zoom angle from the pan/tilt/zoom control unit 207 to obtain the coordinates of the shooting area, and the restriction processing unit 208 calculates the coordinates of the vertices of the shooting space area based on this information.
Next, in step S803, the restriction processing unit 208 judges whether or not there is an overlapping portion between the spatial area of the quadrangular pyramid surrounded by the coordinate values of the mask area information and the shooting space area of the PTZ type surveillance camera 101. Regarding the method of judging the overlap, the surface and the bottom surface of each triangle of the quadrangular pyramid of the mask area are regarded as polygons, and the surface and the bottom surface of each triangle of the quadrangular pyramid of the shooting space area are also regarded as polygons, and whether or not there is an overlap is judged by performing collision judgment between each polygon of the quadrangular pyramid and the quadrangular pyramid. If the restriction processing unit 208 judges that there is an overlapping portion, the process proceeds to step S805. On the other hand, if the restriction processing unit 208 judges that there is no overlapping portion, the process proceeds to step S804.
In step S804, the restriction processing unit 208 waits for at least one of panning, tilting, and zooming to be performed, and if any driving is performed, the process returns to step S802 to recalculate the coordinates of the vertices of the shooting space area of the camera itself.
In step S805, the restriction processing unit 208 actually executes the restriction. The details of the restriction execution process will be described later with reference to FIG.

FIG. 9 is a flowchart showing an example of the detailed process flow of step S805 in FIG.
First, in step S901, the restriction processing unit 208 determines whether or not automatic tracking is currently being performed by the PTZ surveillance camera 101. Whether or not automatic tracking is being performed can be confirmed by referring to the automatic tracking processing in progress flag of the pan-tilt-zoom control unit 207. If the restriction processing unit 208 determines that automatic tracking is being performed, the process proceeds to step S902. On the other hand, if the restriction processing unit 208 determines that automatic tracking is not being performed, the process proceeds to step S904.
In step S902, the restriction processing unit 208 selects an image restriction method for blocking the captured image within the mask area while continuing automatic tracking of the subject. Details of the selection of the image restriction method will be described later with reference to FIG. 10.
In step S903, the restriction processing unit 208 executes video restriction processing in accordance with the video restriction method selected in step S902.
In step S904, the limiting processor 208 judges whether the overlapping amount between the shooting area of the PTZ type surveillance camera 101 and the mask area of the wide-angle type surveillance camera 102 is greater than a threshold value Ts. The method of calculating the overlapping amount may be a method of calculating the volume of the overlapping portion between the quadrangular pyramids composed of the polygons described above, or a simple and fast method of judging the overlapping amount may be a method of calculating the area of the overlapping portion on a two-dimensional plane of the floor surface. If the limiting processor 208 judges that the overlapping amount is greater than the threshold value Ts, the process proceeds to step S905. On the other hand, if the limiting processor 208 judges that the overlapping amount is equal to or less than the threshold value Ts, the process proceeds to step S906.
In step S905, the restriction processing unit 208 instructs the pan and tilt angles to the pan/tilt zoom control unit 207 via the system control unit 203 so that the PTZ surveillance camera 101 faces outside the mask area. Then, the pan/tilt zoom control unit 207 controls the pan driving unit 204 and the tilt driving unit 205 based on the pan and tilt angles instructed by the restriction processing unit 208.
In step S906, the restriction processing unit 208 transmits an instruction to the pan/tilt/zoom control unit 207 via the system control unit 203 to stop pan, tilt, and zoom control so as not to capture more of the mask area than is currently the case. Then, the pan/tilt/zoom control unit 207 performs control to stop the operations of the pan driving unit 204 and the tilt driving unit 205 based on the instruction from the restriction processing unit 208. Details of the operation to stop pan, tilt, and zoom control will be described later with reference to FIG.

Next, the selection of the video control method described above with reference to FIG. 9 will be described with reference to FIG.
FIG. 10 illustrates a restriction pattern priority table used for selecting a video restriction method.
The restriction pattern priority table 1000 consists of three elements: priority, restriction type, and execution capability. In this embodiment, four levels of priority, from priority 1 to priority 4, are defined in the priority item 1001. In Fig. 10, each element value of priority 1 is stored in column 1004, and similarly each element value of priority 2 is stored in column 1005, each element value of priority 3 is stored in column 1006, and each element value of priority 4 is stored in column 1007. The smaller the priority value, the higher the priority.
The restriction type item 1002 indicates the type of image restriction method, and the types of partial masking, defocusing, blacking out the image, and distribution suspension are set according to priority. Note that partial masking refers to applying a restriction such as a mask to a specific part of the image, and defocusing refers to applying a restriction so that the subject in the image is blurred.
In the execution capability item 1003, a value indicating whether or not the PTZ surveillance camera 101 can execute the restriction type is set to "impossible," and "impossible" is set for the restriction type that is not possible for that camera. The restriction processing unit 208 searches for video restriction methods, selects the video restriction method that is executable and has the highest priority, and executes the restriction. In the example of Fig. 10, priority 3 is the highest priority among the executable types, so the restriction processing unit 208 selects and executes the process of completely blacking out the video.
Regarding the relationship between the restriction type and the execution capability, partial masking can be performed by any camera that has a function for masking only a specific part of the image, and similarly, shifting of focus can be performed by any camera that is capable of adjusting the depth of field. In addition, the system control unit 203 can stop the process of transmitting the image to the communication unit 210, thereby stopping the distribution of the image to the client device 103.
Furthermore, the restriction pattern priority table is stored in advance in the storage unit 209, and the contents of the execution capability item 1003 are also determined in advance depending on the function of the PTZ type surveillance camera.

Next, a method of completely blacking out an image, which is one example of an image restriction method, will be described with reference to FIG.
FIG. 11 is a schematic diagram showing an example of the images of the wide-angle surveillance camera 102 and the PTZ surveillance camera 101 when the image blackening process is executed as an example of the image restriction method. Images 1101 and 1104 shown in FIG. 11 show images displayed on the display unit 217 of the client device 103. In the image 1101 of the wide-angle surveillance camera 102, the subject 1102 in the mask area is masked and displayed, and the subject 1103 slightly invading the mask area is displayed as it is except for the feet. On the other hand, the image 1104 of the PTZ surveillance camera 101 is blackened as described above in the explanation of FIG. 10, so the image on the screen is completely black and nothing can be recognized. Note that, as will be described in detail later, the reason for the image restriction process may be superimposed on the completely blackened image.
In this way, when the image of the PTZ type surveillance camera 101 enters the mask area, it is possible to protect the privacy of the subject by applying image restriction to the image of the PTZ type surveillance camera 101. Although an example of completely blacking out the image is shown in Fig. 11, as explained above in Fig. 10, the restriction method is not limited to this, and partial masking, shifting the focus, distribution suspension, and other methods may also be used.

Next, the operation of stopping the pan, tilt and zoom control described above with reference to FIG. 9 will be described with reference to FIG.
FIG. 12 is a schematic diagram showing the movement of the PTZ type surveillance camera 101 when restrictions are imposed to stop pan, tilt and zoom controls.
A subject 302 exists within a mask area 301 set in the wide-angle surveillance camera 102, and a subject 1201 exists separately from the subject 302. The subject 1201 is moving toward the subject 302, and the PTZ surveillance camera 101 is tracking the subject 1201 by manually panning and tilting the PTZ surveillance camera 101. However, just before the subject 1201 enters the mask area 301, a part of the shooting area of the PTZ surveillance camera 101 overlaps with the mask area. As a result, an instruction to stop the pan, tilt, and zoom control described above in FIG. 9 is executed, and the movement 1202 by the pan driving unit 204 and the tilt driving unit 205 is stopped.
This stopping process restricts the PTZ surveillance camera 101 from capturing images within the masked area, making it possible to protect the privacy of the subject.

When a mask area is set in the wide-angle surveillance camera 102, a warning may be superimposed on the image captured by the PTZ surveillance camera 101. Next, with reference to Fig. 13, an example of displaying a warning on the image captured by the PTZ surveillance camera 101 when a mask area is set in the wide-angle surveillance camera 102 will be described.
FIG. 13 is a schematic diagram showing an example of a warning being superimposed on the image captured by the PTZ type surveillance camera 101 on the display unit 217 of the client device 103. In FIG.
As described above with reference to FIG. 4, the user drags the mouse pointer 1303 with the mouse connected to the input unit 218 of the client device 103 to specify the range of the wide-angle surveillance camera image 1301, thereby setting the mask area 1302. When the mask area 1302 is set, the wide-angle surveillance camera 102 transmits the mask area information to the PTZ surveillance camera 101 by communication of the mask area information, and the restriction processing unit 208 instructs the system control unit 203 to output a warning. Then, the system control unit 203 instructs the image processing unit 202 to superimpose a warning message 1305 on the captured image, and further transmits the captured image with the warning message superimposed to the client device 103 via the communication unit 210. The message also includes the model number and serial number of the camera. The character string of the warning message is stored in the storage unit 209, and the system control unit 203 instructs the superimposition by reading out the character string. This allows the user to know on the client device 103 which camera connected to the system has been used to set the mask area.

The warning message may be superimposed on the client device 103 side. In this case, the process is performed, for example, in the following procedure. First, the system control unit 203 instructs the client device 103 to superimpose a warning message on the captured image. Then, the system control unit 219 of the client device 103 superimposes the warning message on the captured image received from the PTZ surveillance camera 101 and displays the image on the display unit 217.

Next, with reference to FIG. 14, an example of displaying a notification of the reason for video restriction processing when the video restriction processing is executed in the PTZ surveillance camera 101 will be described.
FIG. 14 is a schematic diagram showing an example of displaying a notice of the reason for performing image restriction processing on the image captured by the PTZ type surveillance camera 101. In FIG.
When the image restriction process such as the all-black image described above in the description of FIG. 9 is executed, a restriction reason message 1402 is added to the PTZ type surveillance camera image 1401 in order to show the user why the screen display has been restricted. In this process, the restriction processing unit 208 first instructs the system control unit 203 to output a message. The system control unit 203 then instructs the image processing unit 202 to superimpose the restriction reason message 1402 on the all-black image, and further transmits the image with the restriction reason message superimposed to the client device 103 via the communication unit 210. The character string of the message is stored in the storage unit 209, and the system control unit 203 instructs the superimposition by reading out this character string. This allows the user to know why the screen has been restricted.

Note that the restriction reason message may be superimposed on the client device 103 in the same manner as the warning message. In this case, the restriction reason message may be superimposed in the same manner as the warning message is superimposed on the client device 103. In addition, while FIG. 14 describes an example in which the restriction reason message is superimposed on a completely blacked-out image, the message may be superimposed on an image that has been partially masked or defocused as shown in the table in FIG. 10.

As described above, according to this embodiment, the wide-angle surveillance camera 102 transmits mask area information to the PTZ surveillance camera 101, and the PTZ surveillance camera 101 performs control based on the mask area information so that the mask area is not displayed. In other words, in a situation where the pan, tilt, and zoom control of the PTZ surveillance camera 101 causes the shooting range to overlap with the mask area, the pan, tilt, and zoom control is stopped or control is performed so that the shooting range is moved out of the mask area. In addition, when automatic tracking is performed, some kind of processing is performed on the captured image in which the mask area is captured. This makes it possible to avoid privacy violations caused by the mask area being displayed on the client device 103.

Second Embodiment
Next, a second embodiment of the present invention will be described with reference to Fig. 15. The same components as those in the first embodiment will be denoted by the same reference numerals as those in the first embodiment, and detailed descriptions thereof will be omitted. The description will focus on the difference between the first embodiment and this embodiment in the method of transmitting mask area information. This method of omitting descriptions will also be used in the other embodiments described later.
In the first embodiment, the wide-angle surveillance camera 102 transmits mask area information to the PTZ surveillance camera 101. In contrast, in the present embodiment, the PTZ surveillance camera 101 requests the wide-angle surveillance camera 102 to obtain mask area information, and then the wide-angle surveillance camera 102 transmits the mask area information to the PTZ surveillance camera 101.

FIG. 15 is a flowchart showing an example of a processing procedure for transmitting mask area information between cameras in response to a request from the PTZ type surveillance camera 101.
First, in step S 1501 , the system control unit 203 of the PTZ surveillance camera 101 transmits a request to acquire mask area information to the wide-angle surveillance camera 102 .
Next, in step S1502, the system control unit 203 waits to receive a response from the wide-angle surveillance camera 102 in response to the acquisition request in step S1501.
On the other hand, on the wide-angle surveillance camera 102 side, in step S1503, the system control unit 213 of the wide-angle surveillance camera 102 waits to receive a request from another camera.
In step S1504, the system control unit 213 determines whether or not a request to obtain mask area information has been received from the PTZ surveillance camera 101. If the system control unit 213 determines that a request has been received, the process proceeds to step S1505. On the other hand, if the system control unit 213 determines that a request has not been received, the process returns to step S1503 and repeats waiting for reception.
In step S1505, the system control unit 213 refers to the mask area information that has already been calculated in advance and stored in the storage unit 215. In step S1506, the system control unit 213 transmits a response of the mask area information to the PTZ type surveillance camera 101 via the communication unit 216.
Next, in step S1507, in the PTZ surveillance camera 101, the system control unit 203 determines whether or not a response has been received from the wide-angle surveillance camera 102. If the system control unit 203 determines that a response has been received, the process proceeds to step S1507. On the other hand, if the system control unit 203 determines that no response has been received, the process returns to step S1502 and repeats waiting for reception.
In step S1508, the restriction processing unit 208 performs restriction processing based on the received mask area information. The content of the restriction processing is the same as that in the first embodiment.
As described above, in response to a request to obtain mask area information from the PTZ type surveillance camera 101, the wide-angle surveillance camera 102 sends mask area information, allowing the PTZ type surveillance camera 101 to obtain the mask area information and perform restriction processing.

Third Embodiment
Next, referring to Fig. 16, a third embodiment of the present invention will be described, mainly focusing on a case where communication of mask area information is performed between cameras via a client device 103. In this embodiment, the difference between the first and second embodiments will be described, focusing on a difference in the method of transmitting mask area information. In the first embodiment, the mask area information is transmitted directly from the wide-angle surveillance camera 102, while in the second embodiment, an acquisition request is issued from the PTZ surveillance camera 101 and mask area information is received as a response. In this embodiment, the client device 103 requests the wide-angle surveillance camera 102 to acquire mask area information, and transmits the acquired mask area information to the PTZ surveillance camera 101, thereby achieving communication of mask area information.

FIG. 16 is a flowchart showing an example of a processing procedure for transmitting mask area information between cameras in response to a request from the client device 103.
First, in step S 1601 , the system control unit 219 of the client device 103 transmits a request to acquire mask area information to the wide-angle surveillance camera 102 .
Next, in step S1602, the system control unit 219 waits to receive a response from the wide-angle surveillance camera 102 in response to the request transmitted in step S1601.
On the other hand, on the wide-angle surveillance camera 102 side, in step S 1603 , the system control unit 213 of the wide-angle surveillance camera 102 waits to receive a request from the client device 103 .
In step S1604, the system control unit 213 determines whether or not a signal has been received. If the system control unit 213 determines that a signal has been received, the process proceeds to step S1605. On the other hand, if the system control unit 213 determines that a signal has not been received, the process returns to step S1603 and the system control unit 213 repeats waiting for reception.
In step S 1605 , the system control unit 213 refers to the mask area information that has been calculated in advance and stored in the storage unit 215 .
In step S<b>1606 , the system control unit 213 transmits a response including the mask area information to the client apparatus 103 via the communication unit 216 .
Then, in step S1607, the system control unit 219 of the client device 103 determines whether or not a response has been received. If the system control unit 219 determines that a response has been received, the process proceeds to step S1608. On the other hand, if the system control unit 219 determines that no response has been received, the process returns to step S1602 and the process repeats waiting for reception.
In step S<b>1608 , the system control unit 219 transmits the mask area information received from the wide-angle surveillance camera 102 to the PTZ surveillance camera 101 via the communication unit 220 .
On the other hand, on the PTZ type surveillance camera 101 side, in step S1609, the system control unit 203 of the PTZ type surveillance camera 101 waits to receive mask area information.
In step S1610, the system control unit 203 determines whether or not a signal has been received. If it is determined that a signal has been received, the process proceeds to step S1611. On the other hand, if it is determined that a signal has not been received, the process returns to step S1609 and the system control unit 203 repeats waiting for reception.
In step S1611, the restriction processing unit 208 performs restriction processing based on the mask area information received from the client device 103. The content of the restriction processing is the same as in the first embodiment.
As described above, when communicating mask area information between the PTZ type surveillance camera 101 and the wide-angle type surveillance camera 102, the PTZ type surveillance camera 101 can obtain the mask area information and perform restriction processing by going through the client device 103.

Other Embodiments
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

203 System control unit, 207 Pan/tilt/zoom control unit, 208 Restriction processing unit

Claims (17)

An imaging device constituting an imaging system together with an external device that sets a mask area in a captured image,
An imaging means;
a determination means for determining whether or not at least a part of the mask area is included in the image acquired by the imaging means, based on information related to the mask area set by the external device;
a limiting means for limiting a photographing operation when the determining means determines that at least a part of the mask area is included in the acquired image;
having
The imaging device according to claim 1, wherein the restriction means changes a method for restricting a photographing operation depending on whether or not an automatic tracking operation for tracking a specific subject is being executed . The camera further includes a control unit for controlling pan, tilt, and zoom of the imaging unit,
the restriction means restricts a photographing operation by restricting control by the control means;
2. The imaging device according to claim 1, the restriction means restricts the photographing operation by restricting the control means to perform pan, tilt, and zoom control so that the photographing range of the imaging means is outside the mask area when the automatic tracking is not being performed ;
3. The imaging device according to claim 2, the restriction means restricts a photographing operation by applying a restriction to the acquired image while the automatic tracking is being performed ;
2. The imaging device according to claim 1, the restriction means restricts a photographing operation by applying a restriction to a specific portion of the acquired image;
5. The imaging device according to claim 4, the restriction means restricts the photographing operation by applying a restriction so that a subject in the acquired image is blurred;
5. The imaging device according to claim 4, the restriction means restricts a photographing operation by applying a restriction to the entire captured image;
5. The imaging device according to claim 4, Further comprising an adding means for adding a notice including a reason for applying the restriction by the restricting means to the acquired video.
8. The imaging device according to claim 4, wherein the restriction means restricts the acquired image by stopping transmission of the image to an information processing device that displays the image;
2. The imaging device according to claim 1, the restriction means selects a method for imposing a restriction on the acquired video based on a priority from among a plurality of methods for imposing a restriction on the acquired video;
5. The imaging device according to claim 4, a superimposing unit that superimposes a warning that the mask area has been set by the external device on the acquired image;
11. The imaging device according to claim 1, wherein The masking area may further include a receiving unit for receiving information related to the masking area.
the receiving means receives information relating to the mask region from the external device;
12. The imaging device according to claim 1, wherein a transmission unit for transmitting a request for information on the mask region to the external device;
The imaging device according to claim 12 . the receiving means receives information relating to the mask region from the external device via an information processing device that displays the acquired image;
The imaging device according to claim 12 . An imaging system including a first imaging device that sets a mask area in a captured image and a second imaging device that restricts a shooting operation when at least a part of the mask area is included in the captured image,
The first imaging device is
A first imaging means;
A setting means for setting an area to be subjected to mask processing in the image generated by the first imaging means;
an output means for outputting information regarding the mask area set by the setting means;
having
The second imaging device is
A second imaging means;
a determining means for determining whether or not at least a part of the mask area is included in the image captured by the second imaging means, based on information related to the mask area set by the setting means;
a limiting means for limiting a photographing operation when the determining means determines that at least a part of the mask area is included in the image captured by the second imaging means;
having
The imaging system according to the present invention, wherein the restriction means changes a method for restricting a photographing operation depending on whether or not an automatic tracking operation for tracking a specific subject is being executed . A method for controlling an imaging device that constitutes an imaging system together with an external device that sets a mask area in which mask processing is performed in a captured image, comprising:
a determination step of determining whether or not at least a part of the mask area is included in the image acquired by the imaging means based on information related to the mask area set by the external device;
a limiting step of limiting a photographing operation when it is determined in the determining step that at least a part of the mask area is included in the acquired image;
having
The control method for an imaging apparatus, wherein the limiting step changes a method for limiting a photographing operation depending on whether an automatic tracking for tracking a specific subject is being executed . A program for controlling an imaging device constituting an imaging system together with an external device that sets a mask area in which a mask process is performed in a captured image,
a determination step of determining whether or not at least a part of the mask area is included in the image acquired by the imaging means based on information related to the mask area set by the external device;
a limiting step of limiting a photographing operation when it is determined in the determining step that at least a part of the mask area is included in the acquired image;
on the computer ,
The program, wherein the limiting step changes a method for limiting a photographing operation depending on whether or not an automatic tracking operation for tracking a specific subject is being executed .

Images