JP4915464B2 - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable mask processing to be executed in a remote place. <P>SOLUTION: A server device 200 comprises: a server processing section 240 to generate, in response to a request from a terminal device 100, a mask ID and a two dimensional code which is a two dimensionally encoded mask ID; a server communication section 250 to transmit the mask ID and the two dimensional code generated by the server processing section 240 to the terminal device 100; a camera 290 to image a video displayed on the terminal device 100; a display recognition section 210 to recognize the two dimensional code to be attached to the video imaged by the camera 290; a display processing section 220 to determine a mask position on the video based on the two dimensional code recognized by the image recognition section 210 and to execute a mask processing to the mask position in accordance with the mask ID. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for performing mask processing.

  A system has been developed that uses a bird's-eye view video to share a space between a person at a remote location and a person at the source of the bird's-eye view video. Such a system is composed of a camera that shoots an image overlooking the office at two offices, a video distribution device that bi-directionally distributes the video from the camera, and a terminal that displays the video distributed from the other party Has been.

  In this system, the person in the other party's office may be shown, but the content displayed on the monitor of the PC (Personal Computer) may not be seen by a distant person. In this case, an information portion that is not desired to be seen is masked by image processing.

  In recent years, a function called a privacy mask has been regarded as important for protecting personal information (for example, see Non-Patent Document 1). Non-Patent Document 1 is a manual for each device for setting a privacy mask. The privacy mask is put on a portion of the surveillance camera image that is difficult to capture. However, in Non-Patent Document 1, it is cumbersome to set an area to be privacy masked by manually setting the position of a rectangular area on an image using a GUI or the like. It puts a burden.

  In view of this, there has been proposed a technique for performing mask processing on a portion that is difficult to appear by comparing and collating a video from a camera with an image registered in a database and performing mask processing on a matching portion ( For example, see Patent Document 1).

Hitachi Surveillance Color Video Camera VK-C876 Instruction Manual

Japanese Patent Application No. 2008-84410

  However, in Patent Document 1, a face of a person registered in the database in advance for comparison and comparison is compared with the video from the camera, and only the person registered in the database is masked. In this process, it is difficult to identify the same shape or different objects, and it is not always possible to realize an automated mask process with good accuracy.

  In view of the above problems, an object of the present invention is to provide a new and improved image processing apparatus and image processing method capable of easily performing mask processing even in a remote place.

In order to solve the above problems, according to an aspect of the present invention, a server processing unit that generates a mask ID and a two-dimensional code obtained by two-dimensionally encoding the mask ID in response to a request from a terminal device; A server communication unit that transmits the mask ID and the two-dimensional code generated by the server processing unit to the terminal device, an imaging unit that captures an image displayed on the terminal device,
An image recognition unit for recognizing the two-dimensional code added to the video imaged by the imaging unit; a mask position of the video image is determined based on the two-dimensional code recognized by the image recognition unit; And an image processing unit that executes a mask process according to the mask ID.

  According to such a configuration, the image displayed on the terminal device is imaged by the imaging unit, the two-dimensional code added to the image captured by the image recognition unit is recognized, and image processing is performed based on the recognized two-dimensional code. The mask position of the video is determined by the unit, and mask processing is performed on the mask position. Thereby, the two-dimensional code on the video displayed on the terminal device side is taken in by the imaging unit, the mask position is determined according to the position of the two-dimensional code, and the mask processing according to the mask ID is performed at the mask position. It can be performed. For this reason, the mask processing can be easily executed even in a remote place.

  When one of the two-dimensional codes is recognized by the image recognition unit, the image processing unit determines a mask position of the video based on the position and size of the two-dimensional code recognized by the image recognition unit, Mask processing may be performed at the mask position.

  When a plurality of the two-dimensional codes are recognized by the image recognition unit, the image processing unit determines a mask position of the video based on the arrangement of the two-dimensional codes recognized by the image recognition unit, and sets the mask position. Mask processing may be performed.

  The server processing unit generates a temporary ID temporarily used as a mask ID and a two-dimensional code obtained by two-dimensionally encoding the temporary ID, and the server communication unit generates a temporary ID and a temporary ID generated by the server processing unit. The two-dimensional code is transmitted to the terminal device, the image recognition unit recognizes the two-dimensional code added to the video captured by the imaging unit, and the server processing unit masks the temporary ID. An image ID may be generated, and the image processing unit may determine a mask position of the video based on the recognized two-dimensional code, and perform a mask process corresponding to the mask ID on the mask position.

  When the terminal processing unit determines that there is a shielding object between the imaging unit and the video displayed on the terminal device, the terminal processing unit determines that the part containing the shielding object is a mask exclusion region. The mask process may not be performed on the mask exclusion area.

  The image processing unit may determine whether to perform mask processing according to an attribute to which the video belongs.

  The image processing unit may determine the presence / absence of a mask according to a type of an application that activates the video.

  The server communication unit causes the terminal device to hold the generated mask ID as a mask ID (TID), and the image processing unit sends a mask completion notification and a mask ID (SID) to the terminal device after the mask processing. ), The terminal device performs authentication processing based on the held mask ID (TID) and the transmitted mask ID (SID), and the authentication processing is successful. In this case, the image after mask processing may be transmitted to the terminal device.

  In order to solve the above-described problem, according to another aspect of the present invention, a step of generating a mask ID and a two-dimensional code obtained by two-dimensionally encoding the mask ID in response to a request from a terminal device Transmitting the generated mask ID and the two-dimensional code to the terminal device, imaging the video displayed on the terminal device, and the two-dimensional code added to the captured video And a step of determining a mask position of the video based on the recognized two-dimensional code, and executing a mask process corresponding to the mask ID at the mask position. An image processing method is provided.

  A step of holding the generated mask ID as a mask ID (TID) in the terminal device, and a mask completion notification and a mask ID as a mask ID (SID) are transmitted to the terminal device after the mask processing. Accordingly, a step of causing the terminal device to perform an authentication process based on the held mask ID (TID) and the transmitted mask ID (SID), and if authenticated by the terminal device, Transmitting video to the terminal device.

  As described above, according to the present invention, mask processing can be easily executed even in a remote place, and security can be improved.

1 is an overall configuration diagram of an image processing system according to first and third embodiments of the present invention. FIG. 3 is a flowchart illustrating an operation of the image processing system according to the first embodiment. It is the figure which showed an example of the mask information holding part which concerns on 1st Embodiment. It is a whole block diagram of the image processing system which concerns on 2nd Embodiment of this invention. It is the flowchart which showed the operation | movement of the image processing system which concerns on 2nd Embodiment. It is the figure which showed an example of the temporary ID information holding part which concerns on 2nd Embodiment. 10 is a flowchart illustrating an operation of the image processing system according to the third embodiment. It is the figure which showed an example of the mask information holding part which concerns on 3rd Embodiment. It is the figure which showed an example of the two-dimensional code for a display which concerns on 1st-3rd embodiment. It is the figure which showed the other example of the two-dimensional code for the display which concerns on 1st-3rd embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<First Embodiment>
[Configuration of image processing system]
First, the overall configuration of the image processing system according to the first embodiment of the present invention will be described with reference to FIG. The image processing system 10 includes a terminal device 100 and a server device 200.

  The terminal device 100 is provided at an overhead view point, and is used when an individual performs business in an office or the like. The terminal device 100 is connected to a monitor 190 for displaying a document or the like.

  The server device 200 is provided at the overhead view point, like the terminal device 100, and performs image mask processing. The server device 200 is connected to a camera 290 that captures an image displayed on the terminal device 100. Specifically, the camera 290 captures an overhead video of an office or the like and sends the captured video to the server device 200.

  The camera 290 is an example of an imaging unit that captures an image displayed on the terminal device 100. The imaging unit may capture an image using a unit other than the camera 290, such as acquiring an image using infrared rays. . The server apparatus 200 is an example of an image processing apparatus, and the image processing apparatus does not necessarily have to be provided at the overhead view point.

  The terminal device 100 and the server device 200 are connected by a network 400. The server apparatus 200 and the image receiving apparatus 300 located at a remote location are also connected by the network 400. The server device 200 performs a mask process on a desired position of the image captured by the camera 290, and sends the processed image to the remote image receiving device 300.

[Functional configuration of terminal device and server device]
Next, functional configurations of the terminal device 100 and the server device 200 will be described. The terminal device 100 includes a mask ID holding unit 110, a terminal processing unit 120, and a terminal communication unit 130. The mask ID holding unit 110 holds the mask ID sent from the server device 200. Hereinafter, the mask ID held in the mask ID holding unit 110 is referred to as a mask ID (TID). The terminal processing unit 120 executes various processes for each application in the terminal device 100. The terminal communication unit 130 communicates with the server device 200 via the network 400.

  The server device 200 includes an image recognition unit 210, an image processing unit 220, a mask information holding unit 230, a server processing unit 240, and a server communication unit 250. The image recognition unit 210 recognizes a two-dimensional code added to the video imaged by the camera 290. The two-dimensional code is included in the video displayed on the monitor 190, and is used to determine an area to be masked.

  The image processing unit 220 performs mask processing on the input video from the camera 290. More specifically, the image processing unit 220 determines the mask position of the video based on the two-dimensional code recognized by the image recognition unit 210 and puts the mask on the determined mask position.

  The mask information holding unit 230 holds information related to the area to be masked. The configuration of the mask information holding unit 230 is shown in FIG. The mask information holding unit 230 stores a mask ID 232, a mask area position 234, and a terminal address 236. The terminal address 236 is an address of a terminal device that has transmitted a mask processing request with a setting condition (two-dimensional code).

  The server processing unit 240 performs various calculations and processes. For example, the server processing unit 240 generates a mask ID and a two-dimensional code obtained by two-dimensionally encoding the mask ID in response to a request from the terminal device 100. The mask ID generated by the server processing unit 240 is stored in the mask ID 232 of the mask information holding unit 230.

  The server communication unit 250 transmits the mask ID and the two-dimensional code generated by the server processing unit 240 to the terminal device 100. The mask ID is a superordinate concept of the mask ID and the temporary ID.

[Operation of image processing system]
Next, operations of the terminal device 100 and the server device 200 according to the present embodiment will be described with reference to FIG. In addition, although communication between the terminal device 100 and the server device 200 is performed between the terminal communication unit 130 and the server communication unit 250, the description thereof is omitted here.

(Mask setting main process 1)
A user of the terminal device 100 who wants to apply a privacy mask to the monitor 190 of his / her terminal device 100 requests the terminal device 100 to apply a privacy mask by operating the terminal device 100. In response to the user's request, terminal processing unit 120 transmits a setting right request to server processing unit 240 (ST1).

  In response to this, the server processing unit 240 generates a mask ID and a display two-dimensional code obtained by two-dimensionally encoding the mask ID, and a server-side mask information holding unit as a server-side mask ID (SID). 230 (ST2, see FIG. 3).

  Next, the server processing unit 240 transmits the two-dimensional code display data and the mask ID to the terminal processing unit 120 (ST3). Furthermore, the server processing unit 240 requests the image recognition unit 210 to make a display data recognition request based on the mask ID (ST4).

  The terminal processing unit 120 holds the received mask ID as a terminal-side mask ID (TID) in the terminal-side mask ID holding unit 110 (ST5), and the terminal processing unit 120 monitors the display data of the two-dimensional code 190. (ST6). An example in which the display data of the two-dimensional code is displayed on the monitor 190 is shown in FIGS. FIG. 9 shows the display data of the two-dimensional code enlarged to the full monitor screen, and FIG. 10 shows the display data of the two-dimensional code one by one at the four corners of the monitor 190.

  With either method, it is possible to confirm on the video where the rectangular area of the monitor 190 is. The image of the recognized rectangular area is a mask area. In the case of FIG. 9, one two-dimensional code coincides with the edge (rectangle) of the monitor 190, and a mask region is defined by one two-dimensional code. In the case of FIG. 10, the area defined based on the two-dimensional code arranged at the four corners coincides with the edge (rectangular area) of the monitor 190, and the mask area is defined by four two-dimensional codes.

  As another example of expressing the mask area, in addition to the coordinates of the four corners, the original two-dimensional code (or a set thereof) is rotated / enlarged / reduced and translated in space and pasted into the image And a method of demarcating the mask area from the rotation angle, the enlargement / reduction ratio, and the position coordinates on the image of the center of the figure. In this case, it is necessary to have in advance information on how the two-dimensional code is displayed by rotating, reducing, enlarging, or translating in the three-dimensional space.

  Returning to FIG. 2 again, the image recognition unit 210 receives a recognition request based on the mask ID in ST4 and recognizes display data (two-dimensional code) from the camera video (ST7). When the display data is recognized as a two-dimensional code, the image recognition unit 210 outputs the coordinates of the area occupied by the recognized two-dimensional code on the screen and the mask ID included in the two-dimensional code. These pieces of information are held in the mask ID 232 and the mask area position 234 of the mask information holding unit 230.

  An area for finding a field corresponding to the recognized mask ID from the mask information holding unit 230 and identifying where the coordinates on the camera image are in the area of the image recognized at the mask area position in the mask ID field. Stores the position coordinates of.

  When the display data of the two-dimensional code is recognized by the image recognition unit 210, the image recognition unit 210 transmits a mask display request to the image processing unit 220 (ST8). The display area of the mask is a display area of recognized display data. Specifically, when one two-dimensional code is recognized by the image recognition unit 210 (FIG. 9), the image processing unit 220 determines the two-dimensional code based on the position of the two-dimensional code recognized by the image recognition unit 210. The area itself is determined to be the mask position of the image, and the mask is put on the mask position (ST9). At this time, the two-dimensional code is enlarged or reduced on the terminal side and displayed so that the mask area is enlarged / reduced depending on the size of the two-dimensional code even if the display position of the two-dimensional code is the same. It is also possible.

  When four two-dimensional codes are recognized by the image recognition unit 210 (FIG. 10), the image processing unit 220 is a region in which the two-dimensional codes are arranged at the four corners based on the arrangement of the two-dimensional codes recognized by the image recognition unit 210. Is determined as the mask position of the image, and the mask position is applied (ST9).

  After the mask display, the image processing unit 220 notifies the image recognition unit 210 of the mask display completion (S10). Receiving the mask display completion, the image recognition unit 210 sends a marker recognition completion notification and a mask ID (SID) to the server processing unit 240 (S11). The server processing unit 240 sends a marker recognition completion notification and a mask ID (SID) to the terminal processing unit 120 (S12).

  The terminal processing unit 120 compares the mask ID (SID) sent together with the marker recognition completion notification (mask completion notification) with the mask ID (TID) already held in the mask ID holding unit 110, and confirms that they match. Confirm (S13: authentication process). If the mask IDs do not match, there is a problem and an error is assumed.

  In the authentication process of S13, the terminal processing unit 120 executes a comparison matching process between the server-side mask ID (SID) and the terminal-side mask ID (TID) as follows. The mask ID (TID) is information on the currently displayed two-dimensional code. When a mask ID (SID) that is not included in the mask ID (TID) held in the mask ID holding unit 110 is transmitted from the server side, the terminal processing unit 120 determines that the authentication process has failed as misrecognition. Corresponding mask IDs (TIDs) need to be stored in the mask ID holding unit 110 for all transmitted mask IDs (SIDs).

  If there is a mask ID (SID) that has not been sent among the mask IDs (TID) stored in the mask ID holding unit 110, the stored mask ID (TID) is one. In this case, since the authentication could not be performed at all, the terminal processing unit 120 determines that the authentication process has failed.

  In the case of trying to recognize a rectangular area surrounded by a plurality of displayed two-dimensional codes, if two mask IDs (TIDs) are the same, four two-dimensional codes are enclosed on the server side. When a region can be specified, one mask ID (SID) is sent to the terminal side, so the authentication process is the same as when one two-dimensional code is used.

  In the case where there are a plurality of mask IDs (TIDs) and it is intended to recognize a rectangular area surrounded by a plurality of displayed two-dimensional codes, it is assumed that the two-dimensional code is surrounded by a rectangular area. For this reason, it is necessary to send a set of two-dimensional codes (two-dimensional codes on the diagonal) that can identify at least a rectangular area as a mask ID (SID). In addition to this, it is a necessary condition that a mask ID (SID) which is not held as a mask ID (TID) is not sent. For example, if there are four two-dimensional codes on the screen, and each mask ID (TID) is 1 for the upper left, 2 for the upper right, 3 for the lower left, and 4 for the lower right, When a mask ID (SID) is sent, it is a two-dimensional code on a diagonal line, so that the original shape can be understood, and it is determined that the authentication process is successful.

(Process after mask setting 1)
When the held mask ID (TID) matches the transmitted mask ID (SID) and authentication is established, the terminal processing unit 120 sends the mask processing recognized image request together with the mask ID to the image processing unit 220. (S14). In response to this, the image processing unit 220 transmits the mask processed image identified by the mask ID to the terminal processing unit 120 (S15). Thereafter, the terminal processing unit 120 displays the received mask processed image and performs a mask display confirmation process (S16).

  In the mask display confirmation process, the user is presented with an image-processed image and confirmed that the expected mask process has been performed. Although not described in this embodiment, recognition may be performed through a user interface.

  As described above, according to the present embodiment, the two-dimensional code is added to the video on the terminal device side. Therefore, since it is not necessary for the user to set the shape of the privacy mask area using a GUI (Graphical User Interface) or the like, the complexity of the mask processing can be reduced. Thereby, mask processing can be easily performed even in a remote place, and security can be improved. In addition, it is possible to apply a privacy mask to each user on a monitor having a shape common to many users.

  The reason why the mask ID is encoded into the two-dimensional code is to allow a plurality of terminal users to set a privacy mask on the monitor of their own terminal device at the same time. For example, it is assumed that users A and B use terminal apparatuses TA and TB, respectively. When the mask ID is encoded into a two-dimensional code, the operation is as follows.

  Terminal devices TA and TB are shown on the overhead camera. It is assumed that the user A performs an operation for setting a privacy mask, the terminal apparatus TA acquires TIDA as a mask ID (TID), and acquires CA as a two-dimensional code. The address of the terminal device TA is ADRA.

  At the same time, it is assumed that the user B performs an operation for setting a privacy mask, the terminal device TB acquires TIDB as a mask ID (TID), and acquires CB as a two-dimensional code. The address of the terminal device TB is ADRB.

  In this case, two two-dimensional codes CA and CB are displayed on the video of the overhead camera. From the recognition result of the two-dimensional code CA, position coordinates on the video of the two-dimensional code CA and TIDA as a mask ID are acquired. Similarly, the position coordinate on the image of the two-dimensional code CB and the TIDB as the mask ID are acquired from the recognition result of the two-dimensional code CB.

  The server apparatus 200 can detect which ID (SID) each of TIDA and TIDB is equal to based on the terminal address held in the mask information holding unit 230 (or the temporary ID information holding unit 260). . As a result, TIDA is equal to SIDA, and TIDB is equal to SIDB. In the mask information holding unit 230, SIDA is associated with the terminal address ADRA, and SIDB is associated with the terminal address ADRB. Thereby, the address corresponding to the two-dimensional code displayed on each of the terminal devices A and B can be obtained simultaneously. Therefore, a marker recognition completion notification can be transmitted at the same time.

  On the other hand, if the mask ID is not encoded into a two-dimensional code, the mask setting request from a plurality of terminal devices is not processed until a series of processing for the request from one terminal device is completed. Cannot answer requests from other terminal devices. The series of processing takes time because confirmation processing by the user of the terminal is necessary in addition to automatic processing by the computer. Therefore, in the case where the mask ID is not encoded into a two-dimensional code, the number of times from the mask setting request from a plurality of terminal devices to the completion of the series of processes is larger than in the case where the mask ID is encoded into a two-dimensional code. Of time. In this embodiment, since the mask ID is encoded into a two-dimensional code, such time loss does not occur.

  The mask ID is necessary for individually releasing the privacy mask for each privacy mask. That is, when the mask ID is generated on the server device side, the mask ID is stored on the terminal device side. When it is desired to cancel the privacy mask stored in the terminal device, the stored mask ID is transmitted to the server device. On the server device side, the privacy mask corresponding to the transmitted mask ID can be canceled. In addition to canceling, this mask ID can also be used when performing individual operations for each privacy mask. For example, an operation of individually changing the color of the mask can be realized.

  The mask ID according to the present embodiment is an example of a mask ID generated in response to a request from the terminal device.

<Second Embodiment>
As described above, in the first embodiment, the mask ID is encoded into a two-dimensional code. However, the amount of information that can be encoded into a two-dimensional code is limited. More specifically, in the first embodiment, the two-dimensional code is a two-dimensional encoding of the mask ID, and there is no particular limitation on the amount of information that can be encoded. However, the mask ID is given to the video in a state encoded in a two-dimensional code. For this reason, the number of mask IDs that can be managed in this embodiment (that is, the maximum number of mask IDs) that can be encoded into a two-dimensional code is the number of two-dimensional codes that can be simultaneously captured and processed in one video. It will be restricted.

  In the second embodiment, many mask IDs can be managed by properly using two IDs, a mask ID and a temporary ID described later, in spite of such restrictions. That is, even when the number of mask IDs is very large, the same processing as in the first embodiment is performed using a temporary ID for replacing the mask ID when the mask ID can be acquired after image recognition. Can be implemented. For example, even when the number of mask IDs is large and the mask ID cannot be encoded into a two-dimensional code, the same processing as in the first embodiment can be performed by introducing a temporary ID. Hereinafter, the second embodiment will be described in detail.

[Functional configuration of terminal device and server device]
Functional configurations of the terminal device 100 and the server device 200 according to the second embodiment will be described with reference to FIG. The second embodiment is characterized in that less information is included in the two-dimensional code than in the first embodiment. In the second embodiment, it is assumed that 2-bit information can be encoded.

  In the second embodiment, as illustrated in FIG. 4, the terminal device 100 includes a temporary ID holding unit 140 in addition to the functional configuration of the first embodiment, and the server device 200 includes the functional configuration of the first embodiment. In addition, a temporary ID holding unit 260 is included.

  FIG. 6 shows the internal configuration of the temporary ID information holding unit 260. Since the amount of information that can be encoded into a two-dimensional code is 2 bits, the temporary ID takes four values of 0, 1, 2, and 3. That is, it means that the number of mask IDs that can be set simultaneously on the server device 200 side is four.

[Operation of image processing system]
The operation of the image processing system 10 according to the present embodiment will be described with reference to FIG. In the mask setting main process 2 shown in FIG. 5, temporary IDs appearing in all steps of the mask setting main process 1 (ST1 to ST13) shown in FIG. 2 of the first embodiment are described in the first embodiment. It has the same function as the mask ID and has no change in operation. Further, the post-mask setting process 2 in FIG. 5 is a modification of the post-mask setting process 1 in FIG.

  The operation will be described below. As described above, the mask setting main processing 2 portion is obtained by replacing the mask ID of the mask setting main processing 1 of the first embodiment with a temporary ID, and the operation is not changed at all. To do. However, the temporary ID is held in the temporary ID information holding unit 260.

  The mask setting post-processing 2 will be described. In the mask setting main process 2, the masked image is sent from the terminal processing unit 120 to the image processing unit 220 by using the temporary ID for which the authentication confirmation that the temporary ID is matched in the temporary ID matching check (ST513). The request and temporary ID are transmitted (ST514). Receiving this, the image processing unit 220 transmits the masked image subjected to the mask processing to the server processing unit 240 (ST515). The server processing unit 240 generates a mask ID corresponding to the temporary ID and stores it in the mask information holding unit 230. At the same time, the temporary ID held in temporary ID information holding section 260 is deleted (ST516). At this time, temporary ID information holding the corresponding temporary ID is copied from the temporary ID information holding unit 260 and held in the mask information holding unit 230. Items to be copied are information regarding the mask area position 264 and the terminal address 266.

  Next, server processing section 240 transmits the masked image and mask ID to terminal processing section 120 (ST517). Terminal processing section 120 performs a mask display confirmation process, and after confirming that the mask is correctly displayed, deletes the temporary ID stored in temporary ID holding section 140 (ST518). The mask display confirmation process is the same as that performed in the mask setting post-process 1.

  If it is confirmed that the mask is correctly displayed, terminal processing section 120 notifies server processing section 240 of confirmation completion (ST519).

  As described above, according to the present embodiment, the shape of the privacy mask area does not need to be set by the user using a GUI (Graphical User Interface) or the like, as in the first embodiment. The complexity of processing can be reduced. In particular, according to the present embodiment, the mask setting main process 2 uses a temporary ID and does not use a mask ID. For this reason, it is not necessary to encode the mask ID into a two-dimensional code. Therefore, in the second embodiment, there is no restriction due to the amount of information that can be encoded into the mask ID. The information amount of the mask ID can be designed according to the capacity of the temporary ID information holding unit 260 and the communication speed and communication capacity between the server processing unit 240 and the terminal processing unit 120. For example, if the capacity of the temporary ID information holding unit 260 is 10 bits, 1024 IDs can be created. Since the capacity of information devices in recent years is a sufficiently small value, and the network capacity of a LAN (Local Area Network) is usually about 100 MBPS, 8 bits per ID is a sufficiently small value. No problem. As a result, in the second embodiment, privacy masks can be managed with a large number of mask IDs even when the amount of information that can be added to the two-dimensional code is small.

  The temporary ID according to the present embodiment is an ID that is temporarily used as a mask ID.

<Third Embodiment>
[Configuration of image processing system]
Next, an image processing system 10 according to a third embodiment of the present invention will be described. In the third embodiment, when a shielding object enters between the monitor 190 and the camera 290, privacy mask processing is not performed on the shielding object, and privacy mask processing is performed on a portion that is not shielded and is reflected on the camera 290. The point is to perform.

  In the third embodiment, as shown in FIG. 8, the mask information holding unit 230 stores the conversion coefficient 238 in addition to the items of the mask ID 232, the mask area position 234, and the terminal address 236. Different from form.

[Operation of image processing system]
The operation of the image processing system 10 according to the present embodiment will be described with reference to FIG. Since all steps ST1 to ST16 in FIG. 2 are the same as those in the first embodiment, the description thereof is omitted. When the processing up to ST16 is completed, the server apparatus 200 is in a state where a privacy mask is set in a necessary area, and the terminal apparatus 100 is in a state where a mask ID is held. As shown in FIG. 7, terminal processing section 120 sends a hidden processing request together with mask ID to server processing section 240 (ST700). Here, the hiding process means that when a shielding object enters between the monitor 190 and the camera 290, the shielding object is not subjected to the privacy mask process, and the part that is reflected on the camera 290 is not shielded. Privacy mask processing is performed.

  Server processing section 240 calculates a conversion coefficient (ST701). The server processing unit 240 calculates from the recognition result of the area corresponding to the mask ID how the original mask shape is transformed into an image photographed by the camera 290 when projective transformation is performed. The mask shape is a rectangle, and the image obtained by rotating / reducing / enlarging it in a three-dimensional space is reflected in the camera 290 and becomes a received image. In other words, the projection data of the display data sent to the terminal-side monitor 190 is a photographed image by the camera 290. For this reason, the conversion coefficient of this projective transformation is obtained. This is a known technique for recognizing an AR marker or the like.

  Terminal processing section 120 transmits the monitor screen image to image recognition section 220 together with the mask ID (ST702). The image recognition unit 220 performs projective transformation of the monitor screen image with the conversion coefficient corresponding to the mask ID obtained in ST701, and sets the image and the input image from the camera 290 corresponding to the original mask area for each pixel. Compare to. In comparison, a greatly different portion is determined to be a portion where a shield is placed between the camera 290 and the monitor 190, and this portion is calculated as a mask exclusion region (ST703). Image recognition section 210 sends the calculated mask exclusion area to image processing section 220 (ST704).

  The image processing unit 220 performs mask display according to the set mask area and mask exclusion area (ST705). That is, the image processing unit 220 does not perform privacy mask processing on the shielding object (mask exclusion region), and performs privacy mask processing on a portion that is not shielded and is reflected in the camera 290. The processing of ST702 to ST705 is repeated until a separate instruction to stop the procedure is received.

  As described above, according to the present embodiment, the shape of the privacy mask area does not need to be set by the user using a GUI (Graphical User Interface) or the like, as in the first embodiment. The complexity of processing can be reduced. In particular, according to the present embodiment, when a shielding object is partially or wholly placed between the monitor 190 and the camera 290, a part of the shielding object is not masked and is not applied to the monitor screen. Can only be masked. As a result, it is possible to avoid masking an unintended portion of the video.

  As described above, according to the first to third embodiments, a two-dimensional code on an image displayed on the terminal device side is taken in by a camera, and a mask position is determined according to the position of the two-dimensional code. Since the mask process is performed, the mask process can be easily executed even in a remote place.

  In the first to third embodiments, commands to the respective units are executed by a dedicated control device or a CPU (not shown) that executes a program. A program for executing input processing such as characters described below is stored in advance in a ROM or non-volatile memory (not shown), and the CPU reads out and executes each program from these memories, whereby the terminal processing unit 120 is executed. The functions of the server processing unit 240, the image recognition unit 210, and the image processing unit 220 are realized.

  In the first to third embodiments, the operations of the respective units are related to each other, and can be replaced as a series of operations and a series of processes in consideration of the mutual relationship. Thereby, the embodiment of the image processing apparatus can be the embodiment of the image processing method.

  Accordingly, a step of generating a two-dimensional code in response to a request from the terminal device, a step of transmitting the generated two-dimensional code to the terminal device, and a step of capturing an image displayed on the terminal device Recognizing the two-dimensional code added to the captured image, determining a mask position of the image based on the recognized two-dimensional code, and executing a mask process on the mask position; It is possible to provide an image processing method including:

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the above embodiment, one camera is illustrated, but the present invention is not limited to such an example. For example, in all embodiments, a plurality of cameras may be connected to the server device. In that case, an area storing information related to the mask area may be expanded to store which camera image is stored for each mask ID or each temporary ID.

  In all the embodiments, the amount of information can be increased by changing the two-dimensional code in the time direction. That is, a large amount of information is coded by sending a plurality of two-dimensional codes to be displayed and displaying them on a monitor at a constant time period. In this case, the overall recognition time becomes long.

  In all the embodiments, when determining the mask area, the rectangular area of the monitor 190 may not be used as the mask area. For example, two-dimensional codes are displayed at the two right corners of the monitor 190, respectively. For this, various methods such as recognizing the right half of the monitor 190 as a mask area can be used.

  In all the embodiments, information may be used instead of coding information with a two-dimensional code. By using the watermark technique, even if the two-dimensional code is not visually displayed, the code included in the video can be recognized by the camera.

  In all the embodiments, the image processing unit may determine whether or not to perform the masking process according to the attribute to which the video projected on the monitor belongs. For example, the mask process may be executed when the attribute to which the video belongs is a word file document, and the mask process may not be executed when the attribute is a mail document.

  In all the embodiments, the image processing unit may determine the presence or absence of a mask according to the type of application that activates the video.

DESCRIPTION OF SYMBOLS 10 Image processing system 100 Terminal device 110 Mask ID holding part 120 Terminal processing part 130 Terminal communication part 140 Temporary ID holding part 200 Server apparatus 210 Image recognition part 220 Image processing part 230 Mask information holding part 240 Server processing part 250 Server communication part 260 Temporary ID information holding unit 300 Image receiving device

Claims (10)

A server processing unit that generates a mask ID and a two-dimensional code obtained by two-dimensionally encoding the mask ID in response to a request from the terminal device;
A server communication unit that transmits the mask ID and the two-dimensional code generated by the server processing unit to the terminal device;
An imaging unit that captures an image displayed on the terminal device;
An image recognition unit for recognizing the two-dimensional code added to the video imaged by the imaging unit;
An image processing unit that determines a mask position of the video based on the two-dimensional code recognized by the image recognition unit, and executes a mask process corresponding to the mask ID at the mask position;
An image processing apparatus comprising:   When one of the two-dimensional codes is recognized by the image recognition unit, the image processing unit determines a mask position of the video based on the position and size of the two-dimensional code recognized by the image recognition unit, The image processing apparatus according to claim 1, wherein mask processing is executed at the mask position.   When a plurality of the two-dimensional codes are recognized by the image recognition unit, the image processing unit determines a mask position of the video based on the arrangement of the two-dimensional codes recognized by the image recognition unit, and sets the mask position. The image processing apparatus according to claim 1, wherein mask processing is executed. The server processing unit generates a temporary ID temporarily used as a mask ID and a two-dimensional code obtained by two-dimensionally encoding the temporary ID,
The server communication unit transmits the temporary ID and the two-dimensional code generated by the server processing unit to the terminal device,
The image recognition unit recognizes the two-dimensional code added to the video imaged by the imaging unit,
The server processing unit generates a mask ID corresponding to the temporary ID,
The image processing unit determines a mask position of the video based on the recognized two-dimensional code, and executes a mask process corresponding to the mask ID on the mask position. The image processing apparatus according to any one of the above.   When the terminal processing unit determines that there is a shielding object between the imaging unit and the video displayed on the terminal device, the terminal processing unit determines that the part containing the shielding object is a mask exclusion region. The image processing apparatus according to claim 1, wherein mask processing is not performed on the mask exclusion region.   The image processing apparatus according to claim 1, wherein the image processing unit determines whether to perform mask processing according to an attribute to which the video belongs.   The image processing apparatus according to claim 1, wherein the image processing unit determines presence or absence of a mask according to a type of an application that activates the video. The server communication unit causes the terminal device to hold the generated mask ID as a mask ID (TID),
The image processing unit
After the mask processing, by transmitting a mask completion notification and a mask ID as a mask ID (SID) to the terminal device, the held mask ID (TID) and the transmitted mask ID (SID) The terminal device to perform an authentication process based on
The image processing apparatus according to claim 1, wherein when the authentication process is successful, the image after the mask process is transmitted to the terminal apparatus. Generating a mask ID and a two-dimensional code obtained by two-dimensionally encoding the mask ID in response to a request from the terminal device;
Transmitting the generated mask ID and two-dimensional code to the terminal device;
Imaging a video displayed on the terminal device;
Recognizing the two-dimensional code added to the captured image;
Determining a mask position of the video based on the recognized two-dimensional code, and executing a mask process corresponding to the mask ID on the mask position;
An image processing method comprising: Causing the terminal device to hold the generated mask ID as a mask ID (TID);
After the mask processing, by transmitting a mask completion notification and a mask ID as a mask ID (SID) to the terminal device, the held mask ID (TID) and the transmitted mask ID (SID) And causing the terminal device to perform an authentication process based on:
The image processing method according to claim 9, further comprising a step of transmitting the image after mask processing to the terminal device when authenticated by the terminal device.

Images