JP6700845B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing device, an information processing method, and a program.

  In recent years, surveillance cameras are increasingly installed in buildings for the purpose of improving security. When installing surveillance cameras in a building, how to arrange the cameras is an important issue. Patent Document 1 describes a method in which a surveillance camera and an object to be photographed are virtually arranged on a design drawing, and a video image captured by the arranged surveillance camera is displayed on a display screen.

  Moreover, as the number of surveillance cameras installed in public areas increases, the issue of privacy becomes a major issue. Some camera bodies or surveillance camera systems are equipped with a function of masking areas that should not be photographed so that they are not displayed. As a method of masking the area that should not be photographed, a rectangular or circular browsing restriction area (area for restricting browsing) designated by the user with respect to the captured image of each camera is filled or mosaiced. The method of doing so is conventionally known.

JP, 2006-148406, A

  In the conventional technology, the viewing restriction area set in the surveillance camera is effective in the captured image in the fixed direction at the fixed position of the surveillance camera. Therefore, when two or more surveillance cameras shoot the same place from different angles, the restricted viewing area set for one surveillance camera cannot be shared with other monitoring cameras, and the restricted viewing area for each surveillance camera Had to set.

  The present invention has been made in view of the above problems, and an object of the present invention is to set an unphotographable range that can be shared by a plurality of cameras.

As one means for achieving the above object, the information processing apparatus of the present invention has the following configuration. That is, the mask is set so that the subject set as a masked target in the image captured by the first image capturing device is also masked in the image captured by the second image capturing device whose position is different from that of the first image capturing device. A first information processing device that performs a first virtual object corresponding to a subject in a virtual space corresponding to a real space in which the first photographing device, the second photographing device, and the subject exist. A generation unit, a first setting unit that sets a mask area according to a user operation so that the subject included in an image captured by the first image capturing apparatus is masked, and a first setting unit that corresponds to the set mask area. Second generation means for generating a second virtual object in the virtual space, the virtual mask being provided in the virtual space, and a part of the first virtual object, and included in the space of the second virtual object. Information about the position in the virtual space of the surface of the part of the first virtual object, a virtual imaging device provided in the virtual space corresponding to the second imaging device, and the first virtual object. And a second setting unit that sets a mask area based on the positional relationship so that the subject is masked in the image captured by the second image capturing apparatus .

  According to the present invention, it is possible to set a non-shootable range that can be shared by a plurality of cameras.

FIG. 3 is a block diagram showing a module configuration example of the information processing apparatus according to the first embodiment. FIG. 3 is a block diagram showing a hardware configuration example of an information processing device according to the first embodiment. The figure which shows the example of the display screen of the display apparatus in 1st embodiment. FIG. 6 is a diagram for explaining a process of specifying a browsing restricted area or a process of creating a mask object. The figure for demonstrating the designation|designated process of a browsing restriction area. The figure for demonstrating the creation process of a mask object. 6 is a flowchart of masking area setting processing according to the first embodiment. The figure for demonstrating the visual field area of a camera. The figure for demonstrating the collision surface of a mask object and a to-be-photographed object. The figure for demonstrating a mask object and a collision surface. The figure for demonstrating a masking condition selection screen. The figure for demonstrating the image display window of each camera after application of a masking area. 7 is a flowchart of masking area setting processing according to the second embodiment. FIG. 4 is a diagram for explaining a video display window when there is a subject object to be transparent.

  Hereinafter, the present invention will be described in detail based on its embodiments with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[First embodiment]
The configuration and operation of the information processing device 100 according to the first embodiment will be described. First, the hardware configuration of the information processing apparatus 100 will be described, and then the module configuration will be described.

<Hardware configuration>
FIG. 2 is a block diagram showing a hardware configuration example of the information processing apparatus 100 according to this embodiment. As a hardware configuration, the information processing apparatus 100 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Randam Access Memory) 203, a secondary storage device 204, an input device 205 and a display device 206, a network. It has an I/F (interface) 207 and a bus 208.

  The CPU 201 executes an instruction according to a program stored in the ROM 202 or the RAM 203 to perform processing and control of each unit of the camera layout simulation device. The ROM 202 is a non-volatile memory and stores programs and data. The RAM 203 is a volatile memory and temporarily stores data such as images. The secondary storage device 204 is a non-volatile rewritable storage device such as a flash memory, and stores a camera list, an object list, layout information, which will be described later, and various setting contents. These pieces of information are transferred to the RAM 203, and the CPU 201 executes the program and uses the data. The network I/F 207 is an interface for transmitting/receiving digital data, and is connected to a network such as the Internet or an intranet. The bus 208 connects the above-mentioned components to each other to input/output data.

  The input device 205 is a keyboard, a mouse, a touch panel, or the like for a user to input an operation command. The display device 206 is one or more liquid crystal displays or projectors having a display screen, and performs various displays. Although the display device 206 is included in the information processing device 100 in the present embodiment, the display device 206 may be configured as a device different from the information processing device 100. Further, the information processing device 100 may be a device that includes a printing device and is capable of printing an image or a simulation result on a medium such as paper.

<Module configuration>
FIG. 1 is a block diagram showing the module configuration of the information processing apparatus 100 according to this embodiment. As a module configuration, the information processing apparatus 100 includes a drawing data acquisition unit 101, a camera placement unit 102, an object placement unit 103, a restricted viewing area setting unit 104, a mask object creation unit 105, a layout information management unit 106, and a collision surface acquisition unit 107. A display control unit 108, a masking area determination unit 109, a camera list storage unit 111, an object list storage unit 112, and a layout information storage unit 113.

  The drawing data acquisition unit 101 acquires image data (drawing data) of a design drawing that is a base on which a camera is arranged. The drawing data is displayed in the layout window (see FIG. 3) on the display screen of the display device 206 by the display control unit 108 described later. The layout window is a window for displaying the drawing data, the arranged cameras and the field of view of the cameras, and objects other than the arranged cameras. Further, the drawing data acquisition unit 101 changes the scale of the drawing data, the position/orientation of the arranged camera, the angle/orientation/size of the view field of the camera, other than the camera, based on the operation of the input device 205 by the user. It is possible to accept changes in the position/angle/size of the object. The drawing data acquisition unit 101 reflects the received change in the layout window via the display control unit 108. The object other than the camera is a person, a car, a wall, a pillar, or the like that can be arranged on the same drawing as the camera in order to confirm how the camera looks. Further, in the following, an object that is a subject of the camera is called a subject object.

  The camera list storage unit 111 stores a list having detailed specifications such as one or more camera models, camera shapes, camera angles of view, and camera drivable ranges (PTZ). The camera placement unit 102 acquires a camera list from the camera list storage unit 111. In addition, the camera placement unit 102 receives the selection of the camera model desired by the user based on the operation of the input device 205 by the user, and, via the display control unit 108, selects the camera model corresponding to the selected camera model. Display the symbol in the layout window. The camera symbol has the shape of the camera body (dome shape, box shape, etc.) and is displayed together with the symbol indicating the field of view of the camera.

  FIG. 6 is a diagram for explaining the visual field range of the camera. FIG. 6A shows a case where the camera body has a box shape, and FIG. 6B shows a case where the camera body has a dome shape. In FIG. 6A, the visual field area of the camera 61 is represented by a visual field area 62. Further, in FIG. 6B, the visual field area of the camera 63 is represented by a visual field area 64. The shape of the camera is determined by the model of the camera, and other than the shape shown in FIG.

  The cameras 61 and 63 and the visual field regions 62 and 64 are displayed in the window as symbols each having a three-dimensional shape. In the present embodiment, the floor is set to Z=0 in the drawing data displayed in the layout window. As shown in FIG. 6, the visual field area of the camera in the three-dimensional space is represented by a quadrangular pyramid visual field area F-ABCD when the camera faces the XY plane. When the camera faces the XY plane, the center point exists in the direction of the camera in the box type and in the direction of the lens of the camera in the dome type. The viewing area F-ABCD has a viewing angle θ. In FIG. 6A, when the viewpoint of the camera is F and the point passing through the center of the visual field is O, the vector FO represents the direction of the camera. In the example of FIG. 6A, since the visual field area F-ABCD hits the floor without reaching the object, the bottom surface ABCD is a collision surface between the visual field area F-ABCD and the floor. The collision surface is calculated from the three-dimensional model and can be confirmed visually and numerically.

  The object list storage unit 112 stores a list including one or more subject object types, object shapes, object sizes, and the like. The symbol of the subject object has a three-dimensional shape like the symbol of the camera. The object placement unit 103 acquires a list of subject objects from the object list storage unit 112. Further, the object placement unit 103 selects the type of the subject object based on the user's operation on the input device 205, and displays the symbol of the selected subject object on the layout window via the display control unit 108. ..

  The viewing restriction area setting unit 104 accepts a user's designation (operation input) via the input device 205 in the video display window (see FIG. 3) of the camera displayed by the display control unit 108, and based on the designation. , Set viewing restriction area. For example, the user designates a partial area of the image by operating input through a mouse, a touch panel, or a keyboard. In the present embodiment, the browsing restriction area is an area for restricting browsing. Note that the browsing restriction area may be an area for restricting browsing to a user having a specific attribute, or may be an area that can be browsed by a management user. The mask object creation unit 105 receives the setting of the viewing restricted area by the viewing restricted area setting unit 104, creates a virtual object (mask object), and displays the mask object on the layout window via the display control unit 108. .. The mask object is a symbol having a three-dimensional shape, and after being displayed on the layout window, the position and the like can be changed based on the user's operation.

  The layout information storage unit 113 relates to layouts such as drawing data information, camera position and orientation, camera field of view size, subject object position, orientation, and size, mask object position, orientation, and size. Information is stored. The layout information management unit 106 receives a layout change based on the user's operation on the input device 205, and stores the change in the layout information storage unit 113. Changing the layout means changing the camera position/orientation, changing the angle of view of the field of view, changing the position/orientation of the subject object, changing the symbol size, changing the position, orientation, or size of the mask object. .. The layout information storage unit 113 stores information that is updated each time a layout of cameras, a layout of subjects, and a layout change occur in the layout window.

  The collision surface acquisition unit 107 acquires a surface on which the mask object created by the mask object creation unit 105 collides with a subject object arranged in the layout window. The display control unit 108 controls display on the display screen of the display device 206. The masking area determination unit 109 determines a masking area, which is an example of the viewing restriction area, based on the collision surface acquired by the collision surface acquisition unit 107. Details of the operations of the collision surface acquisition unit 107 and the masking area determination unit 109 will be described later.

  Here, the viewing restriction area and the masking area in the present embodiment will be described. The display device 206 displays an image captured by the camera selected by the user. Here, the image captured by the camera in the browse-restricted area set by the browse-restricted area setting unit 104 is not displayed (masked). The restricted viewing area is an effective area when the camera faces the area, and the masked area is released when the camera has a different angle. On the other hand, the masking area is not displayed in the captured image regardless of the angle at which the camera captures the image. The masking area refers to an area of a mask object generated based on the restricted viewing area after removing unnecessary portions.

  FIG. 3 is an example of a display screen of the display device 206 according to the first embodiment. A window 300 on the display screen of the display device 206 is shown in each of FIGS. 3A and 3B. The window 300 has a layout window 301, a video display window 302, a camera placement button 303, an object placement button 304, a viewing restriction area setting button 305, and a mask determination button 306. The layout window 301 is a window for displaying image data (drawing data) of a design drawing of a public facility or a building where a camera is to be arranged, for example. Further, the video display window 302 is a window for displaying a video captured by one of the cameras selected in the layout window 301. FIG. 3A is a view of the layout window 301 in which a camera and a subject object are arranged, as viewed from above, and FIG. 3B is a view of the layout window as viewed obliquely from above.

  The operation of the information processing device 100 for displaying the window 300 shown in FIG. 3 will be described. The drawing data acquisition unit 101 reads image data (drawing data) in which a two-dimensional design drawing is saved in an image format such as a bitmap or JPEG, and displays it on the layout window 301 via the display control unit 108. For example, the drawing data acquisition unit 101 acquires image data selected based on a user operation from a file menu (not shown). Then, the display control unit 108 displays the acquired image data on the layout window.

  When the user operates the camera placement button 303, the camera placement unit 102 acquires a list of selectable cameras from the camera list storage unit 111, and displays the list of cameras in the window 300 via the display control unit 108. Display (not shown). The camera placement unit 102 selects one model of camera from the camera list based on the operation by the user, and displays the symbol of the selected camera at a desired position on the layout window 301 via the display control unit 108. To do. For example, the user selects one model of camera from the camera list, selects the selected camera with a mouse as the input device 205, and then drags and drops it to a desired position in the layout window 301 or selects it. Then, the layout window 301 is pressed to a predetermined position. Thereby, the symbol of the camera of the desired model and the field of view of the camera is displayed at the desired position of the layout window 301. The camera and the field of view of the camera differ depending on the camera model. Displaying the symbol of the camera and the field of view of the camera at a predetermined position in the drawing displayed in the layout window is called camera arrangement.

  When the user operates the object placement button 304, the object placement unit 103 acquires a list of selectable objects from the object list storage unit 112, and displays the list of objects via the display control unit 108. It is displayed on 300 (not shown). The object placement unit 103 selects an attribute (for example, a person, a car, a wall, a pillar, etc.) of one object from the list of the objects based on the operation by the user, and selects the attribute via the display control unit 108. The symbols having the different attributes are displayed at desired positions in the layout window 301. For example, the user selects one attribute from the list of objects, selects the selected attribute with the mouse or the like as the input device 205, and then drags and drops it to a desired position in the layout window 301, or in the selected state. The layout window 301 is pressed to a predetermined position. As a result, the symbol of the desired object is displayed at the desired position in the layout window 301.

  Although the drawing data is two-dimensional, the camera, the field of view of the camera, and the object object have three-dimensional information, and are arranged in the layout window with a three-dimensional shape. The mask object described later also has a three-dimensional shape.

  By such selection and display processing, as shown in FIG. 3A, in the layout window 301, the camera 3001, the visual field range (shooting range) 3002 of the camera 3001, and the pillar 3003 which is the subject object are displayed. In FIG. 3, the object attribute of the pillar 3003 that is the subject object is a pillar. In the image display window 302, an image captured by the camera 3001 is displayed, and a pillar 3003 and a poster 3004 attached to the pillar 3003 are displayed.

  The position, orientation, and size of the camera, the field of view of the camera, the subject object, and the mask object arranged in the layout window can be changed after the arrangement based on an operation using the mouse or the like as the input device 205. The layout information is updated by changing the cameras and objects placed in the layout window. The layout information is a file in a format such as XML and stores drawing data of a layout window, camera layout information, and object layout information. The layout information is updated at the timing when the layout is changed. The layout information storage unit 113 can store the layout information at the timing when the layout is changed. The layout information storage unit 113 updates and stores the layout information based on a predetermined operation by the user. You can go. The browsing restricted area setting button 305 and the mask determination button 306 will be described later.

<Masking area setting processing>
Next, with reference to FIG. 5, a masking area setting process in the present embodiment will be described. FIG. 5 is a flowchart of the masking area setting processing in this embodiment. The process of FIG. 5 is realized by the CPU 201 of the information processing device 100 reading and executing a predetermined program. However, part or all of the processing in FIG. 5 may be realized by dedicated hardware or the like.

  First, in step S501, the drawing data acquisition unit 101 acquires image data (drawing data) of a design drawing and displays it on the layout window 401 via the display control unit 108. Next, in step S502, the information processing apparatus 100 arranges one or more cameras in the layout window. Specifically, as described above, the camera selected based on the user's operation and the symbol of the camera are displayed in an overlapping manner on the layout window. After the display, the position and orientation of the camera may be changed, the orientation of the field of view of the camera, and the angle of view may be changed based on a user operation.

  Next, in step S503, the information processing apparatus 100 arranges one or more subject objects in the layout window. Specifically, as described above, the symbols of the objects selected based on the user's operation are displayed in an overlapping manner on the layout window. After the display, the position, orientation, and size of the subject object may be changed based on the user's operation.

  Next, in step S504, the drawing data acquisition unit 101 selects a symbol of any one of the cameras arranged in the layout window in step S502 based on the operation by the user. Next, in step S505, the display control unit 108 displays the captured image (capturing view) of the camera selected in step S504 in the image display window 402. Next, in step S506, the browsing-restricted area setting unit 104 receives designation of the browsing-restricted area by a user operation on the captured video displayed in the video display window 402. For example, the user can use the mouse as the input device 205 to enclose a region to be partially hidden in the video display window 402 with a rectangle or an ellipse, and then accept the designation of the browse-restricted region setting button 305. Become.

  Next, in step S<b>507, the mask object creation unit 105 creates a mask object from the set browsing restricted area according to the selection of the browsing restricted area setting button 305 by the user. The created mask object is displayed in the layout window by the display control unit 108. The mask object creation unit 105 can also change the mask object based on a user operation.

  Next, in step S508, the collision surface acquisition unit 107 determines whether or not there is a subject object (collision object) that collides with the mask object created in step S507. If no collision object exists (NO in step S508), the process proceeds to step S509. In step S509, the masking area determination unit 109 determines the mask object created in step S507 as a masking area.

  On the other hand, if a collision object exists (YES in step S508), the process proceeds to step S510. In step S510, the collision surface acquisition unit 107 acquires the collision surface between the mask object created in step S507 and the subject object placed in step S503. Subsequently, in step S511, when the masking area determination unit 109 receives the selection of the mask determination button 306, the masking area determination unit 109 determines the masking area based on the collision surface acquired in step S510. Specifically, the masking area determination unit 109 uses the collision surface acquired in step S510 to fit the mask object created in step S507 to the subject object to determine the masking area. The process of step S511 may be performed regardless of whether or not the mask determination button 306 is selected.

  The information processing apparatus 100 repeats the processes of steps S510 to S511 while the collision object exists (Yes in step S512→step S510). If there is no collision object, the process proceeds to step S513. In step S511, the information processing apparatus 100 sets (applies) the masking area determined in step S509 or step S511 to the cameras arranged in the layout window.

  Next, the process of designating the viewing restriction area, generating the mask object, acquiring the collision surface, and determining the masking area will be described in detail in order.

<Specify browsing restricted area>
The designation of the viewing restriction area in step S506 of FIG. 5 will be described. FIG. 4A and FIG. 4B are diagrams for explaining a method of designating a restricted viewing area using the video display window 402. Here, a method of setting a poster attached to a pillar 4003 in the field of view 4002 of the camera 4001 as a non-photographable range will be described.

  In FIG. 4A, the captured image of the camera 4001 arranged in the layout window 401 is displayed in the image display window 402. In the image display window 402, an image 4004 of the poster attached to the pillar 4003 is shown as an unphotographable range set based on the user's operation. In the state where the video image captured by the camera 4001 is displayed, the viewing restriction area setting unit 104 receives an input from the user to the video display window 402 by using the input device 205 such as a mouse. For example, the user surrounds the area with a rectangle 4005 so that the poster image 4004 is hidden. Here, it is assumed that the input is performed by dragging the mouse as the input device 205. The rectangle 4005 may be a rectangle generated by designating two points with the mouse, or may be a rectangle designated by the pointillism of mouse click. The user may also specify an area having a shape other than a rectangle.

  The display control unit 108 temporarily displays the rectangle 4005 designated by the user on the video display window 402 by a dotted line or the like. In this state, in response to the selection of the viewing restriction area setting button 305, the viewing restriction area setting unit 104 determines the temporary area (rectangle 4005) as the viewing restriction area. Note that the browse-restricted area setting unit 104 can accept the temporary area while the browse-restricted area setting button 305 is not selected. Also, a plurality of viewing restriction areas may be determined.

  When the browse-restricted area is confirmed, the provisional display of the rectangle 4005 is visually indicated to be a non-displayed area. FIG. 4B is a diagram showing a state in which the rectangle 4005 designated in the video display window in FIG. 4A is confirmed as the browsing restricted area 4006 by the browsing restricted area setting button 305.

<Generation of mask object>
The creation of the mask object in step S507 will be described. 4A and 4C are views for explaining a method of creating a mask object using the video display window 402. In response to the selection of the viewing restriction area setting button 305, the mask object creation unit 105 creates a mask object so as to cover the viewing restriction area, and displays the mask object in the layout window via the display control unit 108. FIG. 4C shows an example in which the mask object is displayed in the layout window 401 based on the restricted viewing area determined in FIG. 4B. The mask object creation unit 105 creates an object 4007 in the field of view area 4002 of the camera so that the viewing restriction area 4006 determined in FIG. 4B is hidden, and displays it over the layout window 401 via the display control unit 108. To do. Such an object in which the restricted viewing area is hidden is called a mask object.

  Here, the mask object will be described. In the present embodiment, the mask object represents a view-restricted area that is originally invisible in the drawing as an object that can be virtually operated in the layout window. FIG. 8 is a diagram for explaining the mask object and the collision surface. FIG. 8 shows a bird's eye view of the camera 81, the visual field area 82, and the mask object 83 in the layout window.

  The mask object 83 is a three-dimensional object displayed in the layout window so that the viewing restriction area set in the video display window is hidden. Although a quadrangular pyramid will be described here, any quadrangular pyramid or a cube may be used as long as the viewing restriction area set in the video display window is hidden. If the viewing restriction area is set in a circle in the video display window, it may be a cone or a sphere.

  In FIG. 8A, the mask object is an area represented by Mf-MaMbMcMd. The mask object is created so as to include the restricted viewing area specified in the video display window. It suffices that there is a point that passes through the view-restricted area between the line segments MfMa, MfMb, MfMc, and MfMd forming the mask object or as an extension. It may be.

  In the example of FIG. 8A, when the center point of the bottom surface of the mask object is Me, the direction of MfMe has the same direction as the lens direction of the camera 81, and the point Mf is included in the visual field region 82. Is set to. Then, the mask object 83 is formed by the point Mf and the bottom surface MaMbMcMd. The surface where the mask object and the floor collide is SaSbScSd. Although the mask object is a virtual object, it has information about a three-dimensional object that is the same as the subject object. Therefore, when displayed in the layout window, the point Mf and the bottom surface of the mask object can be changed in the layout window after the mask object is created. Since the bottom surface of the mask object (bottom surface MaMbMcMd) can be moved to an arbitrary place by the user's selection, it may be in a state of floating in the air without hitting the floor or the object object, or the bottom surface of the mask object may cover the floor or the object. In some cases, it will penetrate. The mask object does not have to be displayed in the layout window, and may be only displayed in the video display window of each camera. Further, since the mask object is a virtual object, it may be printed out of the print target when printing the layout window.

  The created mask object is an object having an area that only satisfies the condition that the restricted viewing area is included. When there is no subject object in the periphery (NO in step S508 of FIG. 5), the created mask object is determined as the masking area as it was when it was created. Alternatively, an area excluding unnecessary portions may be determined as a masking area by accepting a change in the point Mf and the bottom surface from the user.

<Collision surface acquisition>
When the mask object collides with the subject object (YES in step S508 of FIG. 5 ), the masking area is determined based on the collision surface acquired by the collision. Here, the collision surface refers to a surface where the mask object and the subject object collide. When the mask object, which is a virtual object, exists through the subject object, the collision surface refers to the surface of the subject object in contact with the mask object.

  FIG. 8B shows an example for explaining the collision surface between the mask object 84 and the subject object. Here, it is assumed that the subject object is a pillar and the mask object is vertically colliding with the pillar. The mask object 84 has a shape of Mf-MaMbMcMd, and the collision surface between the mask object and the pillar is SaSbScSd. For detecting the collision surface, a known method such as CAD for calculating the collision surface between the three-dimensional objects is used.

  As shown in FIG. 8B, the mask object can penetrate through even after colliding with the subject object, and the bottom surface of the mask object can be extended to the tip of the subject object. This makes it possible to set the masking area not only for the first collision surface, but for all surfaces that are in contact with the collision object that the mask object collides with, or for the entire collision object, or even for multiple collision objects. Become.

  FIG. 7 is a diagram for explaining a collision surface between the mask object and the subject object. FIG. 7 shows the relationship between the collision surface and the masking area when the mask object collides with the pillar. The diagram on the left is a side view of the mask object, and the diagram on the right is a view from the lens side of the camera. The mask objects 72, 73, 74 are Mf-MaMbMcMd. The first collision surface between the mask objects 72, 73, and 74 and the subject object 71 having the pillar attribute is SaSbScSd. In the example of FIG. 7, since the mask object collides perpendicularly with the pillar, the first colliding surface is SaSbScSd (that is, one surface). On the other hand, when the mask object collides obliquely with the pillar, the first collision surfaces are two surfaces. In this case, the collision surface acquisition unit 107 acquires each of the two collision surfaces as a collision surface.

  In FIG. 7, the masking areas 75, 76 and 77 are shown by hatching. FIG. 7(A) shows a case where the surface where the mask object first collides with the subject object (collision surface existing in the visual field range from the viewpoint of the camera in which the viewing restriction area is set) is used as the masking area SaSbScSd. Only the masking area becomes. FIG. 7B shows a case where the entire surface where the mask object collides with the subject object (that is, the entire side surface) is the masking area. When the first collision surface between the mask object 72 and the subject object 71 is SaSbScSd and the last collision surface is NaNbNcNd, the collision surfaces in FIG. 7B are four surfaces of SaSbScSd, SaSbNaNb, ScSdNcNd, and NaNbNcNd. These four surfaces serve as masking areas in the case of FIG. 7(B). FIG. 7C shows a case where the entire subject object with which the mask object collides is used as the masking area.

  Although the case where the collision surface is flat has been described here, even when the collision surface has unevenness or a curved surface, the collision surface is acquired to set a masking area that covers the surface of the subject object that collides. It is possible to Further, in the above description, the collision surface acquisition unit 107 shows an example in which the mask object detects a collision with respect to only one subject object. However, when the collision object collides with a plurality of subject objects, the collision is similarly detected. It is possible to

<Maskking area determination>
The determination of the masking area in step S511 of FIG. 5 will be described. The masking area determination unit 109 uses the collision surface acquired in step S510 to perform the fitting of the mask object to the subject object, and determines the final masking area. When the user selects the mask determination button 306, the display control unit 108 displays a screen on the display device 206 that allows selection of which part of the colliding subject object is to be used as the masking region.

  FIG. 9 is a diagram showing an example of a condition selection screen when determining the masking area. For example, the masking condition selection screen 900 allows three types of masking region determination conditions to be specified. The user can select any one of the radio buttons 901, 902, and 903, and the setting of the masking area of the mask object is switched according to the selection. When the user selects the radio button 901, the masking area determination unit 109 sets only the first colliding surface as the masking area. When the user selects the radio button 902, the masking area determination unit 109 sets all the surfaces that collide when the mask object passes the subject object as the masking area. When the user selects the radio button 903, the masking area determination unit 109 sets the entire subject object with which the mask object has collided as the masking area.

  FIGS. 9A, 9B, and 9C are diagrams showing the masking areas set by the masking area determination unit 109 in the video display window when the radio buttons 901, 902, and 903 are selected, respectively. .. By selecting such a masking condition, the masking desired by the user can be performed without masking unnecessary areas. Note that the condition for determining the masking area may be set in advance without requiring the determination by the mask determination button 306.

  Subsequently, application of the masking area to the camera in step S513 of FIG. 5 will be described. The masking area determined by the above method is set (applied) so that it can be viewed by all the cameras arranged in the layout window. Here, the information processing apparatus 100 sets the viewing restriction area in the video imaged by the plurality of cameras based on the information indicating the positions where the plurality of arranged cameras are installed and the operation input by the user.

  FIG. 10 is a diagram for explaining the video display window of each camera after applying the masking area. In FIG. 10, the masking area 1006 is shown by hatching, and the masking area 1007 is painted black. According to such a display, the camera can visually recognize the masking area regardless of the position of the layout window or the direction of the lens. Further, the masking area can be visually recognized even by a camera other than the camera in which the masking area is set/determined. Further, in FIG. 10, the masking area is displayed in the layout window in the same manner as the object, but the masking area is not displayed in the layout window, and the masking area is displayed only in the display area of the captured image when the camera is selected. You may do it.

  In FIG. 10, cameras 1001 and 1002 and a subject object having a pillar attribute are displayed in the layout window. The video display window 1004 is a video image captured by the camera 1001, and a viewing restriction area 1005 is set. 10A, 10B, and 10C are images captured by the camera 1002 when the condition for fitting the mask object to the subject object is set by the method of FIG. FIG. 10A shows an image captured by the camera 1002 when the radio button 901 is selected. FIG. 10B shows a video image captured by the camera 1002 when the radio button 902 is selected. FIG. 10C is an image captured by the camera 1002 when the radio button 903 is selected. As described above, the mask object can be visually recognized by the camera 1002 other than the camera 1001 that applied the masking area.

  As described above, according to this embodiment, the masking area set in a specific camera is effective even when the shooting direction of the camera changes. Further, the masking area can be shared by other cameras. Therefore, it is possible to safely protect the area set by the user that should not be photographed, even in an environment in which a plurality of cameras are arranged. The screen display form shown in the present embodiment is an example, and other screen display forms may be applied as long as the process shown in FIG. 5 can be realized. The number and shape of the masking area may be set variably. In addition, a plurality of cameras may be selected in S504 and masking areas may be set in parallel at the same time.

(Second embodiment)
As a second embodiment, a method of applying a masking region by ignoring a subject object that is not masked and hidden by the attribute of the colliding subject object will be described. The points different from the first embodiment will be described.

  Hereinafter, even if the mask object collides with the subject object, the collision surface is not acquired and the subject object is ignored. This means that the mask object is transparent to the subject object. For example, if the attribute of the subject object with which the mask object collides is a person or a car, it may not appear in the captured image at the next time even if it appears in the captured image at a certain time. .. In such a case, it is effective to set the masking area determination process not to be performed on the movable object object even if the object collides, and to be performed only when the object object is transparent and collides with the immovable object object. is there.

  When the subject object is placed in the layout window by the user's operation, the layout information is associated with the position information and stored in the layout information storage unit 113. Therefore, the collision surface acquisition unit 107 can acquire the attribute of the object of the subject by referring to the layout information from the position information of the subject object with which the mask object has collided when acquiring the collision surface.

  FIG. 11 is a flowchart of the masking area setting process in this embodiment. The process of FIG. 11 is realized by the CPU 201 of the information processing device 100 reading and executing a predetermined program. However, part or all of the processing in FIG. 11 may be realized by dedicated hardware or the like. The processing of steps S1101 to S1109 is the same as the processing of steps S501 to S509 in FIG.

  In step S1110, the collision surface acquisition unit 107 acquires the attribute of the collision subject object. Next, in step S1111, the masking area determination unit 109 determines whether or not the colliding subject object is transparent based on the object attribute of the subject object. If it is determined in step S1111 that the subject object is not transparent (NO in step S1111), the process proceeds to step S1112. In step S1112, the masking area determination unit 109 fits the mask object to the subject object using the collision surface acquired in step S1110 to determine the masking area. This process is similar to the process described in the first embodiment.

  On the other hand, if it is determined in step S1111 that the subject object is transparent (YES in step S1110), the process proceeds to step 1113. In step 1113, the masking area determination unit 109 determines the masking area in the transparent state. That is, when there is one collision object and it is determined that the collision object is transparent, the masking area determination unit 109 determines the mask object created in step S1107 as the masking area. In other cases, the masking area determination unit 109 determines the already determined masking area as the masking area in S1113. The processing of steps S1114 to S1115 is the same as the processing of steps S512 to S513 in FIG.

  FIG. 12 is a diagram for explaining a video display window when there is a subject object to be transparent. A subject object having the attributes of a person and a car is movable, a subject object having other attributes is immovable, and a movable subject object is treated as something that should be transparent. That is, a mask object is transparent to a subject object having attributes of a person and a car, and a mask object collides with subject objects having other attributes.

  In FIG. 12, it is assumed that a pillar exists in the field of view of the camera 1201 arranged in the layout window and a person stands in front of it. A video image captured by the camera 1201 is displayed in the video display window 1207. A pillar 1206 is an area that is arranged in the layout window and shows an image of a pillar in which the camera 1201 projects a subject object having a pillar attribute. The viewing restriction area setting unit 104 sets the viewing restriction area 1204 in the video display window 1207 so as to hide a part of the pillar 1206. Here, it is assumed that the user operates the mask determination button 306 to select the setting for masking the entire collision object 903 on the masking condition selection screen of FIG. 9. At this time, when the person 1202 stands in front of the pillar 1203 as shown in FIG. 12A, in the first embodiment, the collision surface of the subject object having the human attribute is acquired as shown in FIG. 12B. , Hides the human object as a masking area. On the other hand, when the second embodiment is used, since the subject object having the attribute of the person with whom the mask object first collides is transparent, the mask object is transparent to the person 1202 and then to the pillar 1203 of the subject object that collides next. collide. Since the subject objects having the pillar attribute collide with each other, the entire pillar 1203 is set as a masking area as shown in FIG.

  As described above, according to the present embodiment, the masking area can be set according to the attribute of the subject object. Therefore, in addition to the effect described in the first embodiment, it is possible to set the masking area more in accordance with the user's intention.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 information processing device, 101 drawing data acquisition unit, 102 camera arrangement unit, 103 object arrangement unit, 104 view restriction area setting unit, 105 mask object creation unit, 106 layout information management unit, 107 collision surface acquisition unit, 108 display control unit , 109 masking area determination unit, 111 camera list storage unit, 112 object list storage unit, 113 layout information storage unit

Claims (13)

The mask is set so that the subject set as a masked target in the image captured by the first image capturing apparatus is also masked in the image captured by the second image capturing apparatus whose position is different from that of the first image capturing apparatus. An information processing device,
A first generation unit that generates a first virtual object corresponding to the subject in the virtual space corresponding to the real space in which the first photography device, the second photography device, and the subject exist;
To the subject included in the image captured by the first imaging device is masked in accordance with a user operation, a first setting means for setting a mask region,
Second generation means for generating a second virtual object in the virtual space, the virtual object being provided in the virtual space corresponding to the set mask area, and a part of the first virtual object;
Information on the position in the virtual space of the surface of the part of the first virtual object included in the space of the second virtual object, and a virtual image provided in the virtual space corresponding to the second imaging device. Second setting means for setting a mask area so that the subject is masked in the image captured by the second image capturing device based on the positional relationship between the image capturing device and the first virtual object ;
An information processing device comprising: The second setting unit is a surface of the part of the first virtual object included in the space of the second virtual object, the surface of the surface facing the virtual imaging device in the virtual space. The information processing apparatus according to claim 1, wherein a mask area is set so that the subject is masked in a video image captured by the second image capturing apparatus, based on position information. The information processing apparatus according to claim 1 or 2, wherein the first generation unit generates the first virtual object in the virtual space according to a user operation. The second generation unit generates the second virtual object including the virtual mask and a part of the first virtual object in the virtual space based on information indicating the position of the first imaging device. The information processing apparatus according to any one of claims 1 to 3, wherein: Before Stories second setting means, the position of the information on the position of the in the virtual space of the first said part of the virtual object surface, and the virtual imaging device and the first virtual object having a predetermined attribute based on the relationship, the to second to have you to a video image captured by the photographing apparatus the subject is masked, information processing according to any one of claims 1 to 4, characterized in that setting the mask area apparatus. Wherein the predetermined attribute, the information processing apparatus according to claim 5, wherein the first virtual object, characterized in that it comprises an attribute that indicates that it is a non-moving. The mask is set so that the subject set as a masked target in the image captured by the first image capturing apparatus is also masked in the image captured by the second image capturing apparatus whose position is different from that of the first image capturing apparatus. An information processing method,
A first generation step of generating a first virtual object corresponding to the subject in a virtual space corresponding to a real space in which the first photographing device, the second photographing device, and the subject exist,
A first setting step of setting a mask area according to a user operation so that the subject included in an image captured by the first imaging device is masked ;
A second generation step of generating in the virtual space a second virtual object including a virtual mask provided in the virtual space corresponding to the set mask area and a part of the first virtual object;
Information on the position in the virtual space of the surface of the part of the first virtual object included in the space of the second virtual object, and a virtual image provided in the virtual space corresponding to the second imaging device. A second setting step of setting a mask area so that the subject is masked in the image captured by the second image capturing device based on the positional relationship between the image capturing device and the first virtual object ;
An information processing method comprising:   In the second setting step, in the virtual space of the surface of the part of the first virtual object included in the space of the second virtual object, the surface positioned facing the virtual imaging device in the virtual space. The information processing method according to claim 7, wherein the mask area is set so that the subject is masked in the image captured by the second image capturing device, based on the position information.   9. The information processing method according to claim 7, wherein in the first generation step, the first virtual object is generated in the virtual space according to a user operation.   In the second generation step, the second virtual object including the virtual mask and a part of the first virtual object is generated in the virtual space based on the information indicating the position of the first imaging device. The information processing method according to any one of claims 7 to 9.   In the second setting step, information on the position in the virtual space of the surface of the part of the first virtual object having a predetermined attribute, and the positional relationship between the virtual imaging device and the first virtual object. 11. The information processing method according to claim 7, wherein the mask area is set so that the subject is masked in the image captured by the second image capturing device based on the above.   The information processing method according to claim 11, wherein the predetermined attribute includes an attribute indicating that the first virtual object is immovable.   A program for operating a computer as each unit of the information processing apparatus according to claim 1.