JP4892071B2 - Image generating apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image generation apparatus, an image processing method, and a program that can generate an image that enables grasping of an expression while protecting a user's privacy.

In recent years, products including a camera (an imaging unit such as a CCD) are sold in notebook personal computers (personal computers) and the like. This camera is incorporated in, for example, an upper part of a frame that supports a display unit (display such as a liquid crystal), and can photograph a user (such as a face).
Such a camera is used, for example, when video chatting is performed. That is, since images captured by the camera are sent to each other in each communicably connected personal computer, each user can chat while looking at the other party's face.

  Recently, products in which such a camera is incorporated are also sold in portable game devices and the like. In addition, a game device is known in which a player can attach and use a camera of a peripheral device even if the camera is not incorporated from the beginning.

  In a game device or the like capable of such video chatting, there is also disclosed a technique for outputting a voice or the like uttered in synchronization with video at one user terminal in synchronization with the video at the other user terminal (for example, a patent) Reference 1).

JP 2007-222411 A (pages 9-19, FIG. 3)

In a game apparatus or the like that can perform video chat as described above, the user can generally set on / off of the camera function. Therefore, when the user does not want to show his / her face to the other party, the user can send only the voice to the other party by setting the camera function off.
However, if the user on the other side does not have an image only by voice, it is difficult to grasp the other party's state, and the advantages of video chat cannot be utilized.

In the first place, in video chat, the user turns off the camera function in an inconvenient current state (the current state of the face and skin, etc.) except in the extreme case where the user does not want to know his / her face. Most people do not want to show. For example, in the case of a female user, the camera function is turned off when makeup is being removed or when rough skin is a concern. Further, even a male user turns off the camera function when he / she is not ready because he / she is awake or when he / she has wrinkles under his / her eyes due to lack of sleep or the like.
In other words, in the past, since only the camera function can be turned on or off, there are many users who turn off the camera function in situations where it is inconvenient to show their faces.

  Therefore, in such a slightly inconvenient situation, development of a game device (image generation device) and the like that can grasp facial expressions necessary for communication while protecting the privacy of the user has been demanded.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image generation apparatus, an image processing method, and a program that can generate an image that enables grasping of facial expressions while protecting the privacy of the user. And

  An image generation apparatus according to a first aspect of the present invention includes a detection unit, an image extraction unit, an image processing unit, and an image generation unit.

  First, a detection part detects a person from the picked-up image image | photographed with the camera etc., for example. The image extraction unit extracts the detected person image of the person from the captured image. In addition, the image processing unit performs, for example, blur processing on the extracted person image. Then, the image generation unit generates a processed captured image obtained by synthesizing the processed person image on, for example, a background image on which no person is captured.

That is, in the present invention, a person image (user image) is extracted from the captured image and subjected to processing such as blurring. Then, the processed person image is synthesized on the background image to generate a processed photographed image. For this reason, even if the user does not want to show the details of the face and skin, it is possible to grasp the facial expression while hiding the details by the processed photographed image subjected to processing such as blurring. Also, since the person image is combined with the background image, for example, even if something that is inconvenient is seen behind the user at the time of shooting, other than the user's image is not extracted, the inconvenience Some things can be properly hidden.
As a result, it is possible to generate an image that enables grasping of facial expressions while protecting the privacy of the user.

  An image generation apparatus according to a second aspect of the present invention includes an image storage unit, a photographing unit, a detection unit, an image extraction unit, an image processing unit, an image generation unit, and an image transmission unit.

  First, the image storage unit stores, for example, a background image in which no person is shown. The photographing unit photographs a photographed image including a person. In addition, the detection unit detects a face image of the person from the captured image by face recognition. The image extraction unit extracts a person image of the person from the photographed image based on the detected face image and surrounding images. In addition, the image processing unit performs, for example, blur processing on the extracted person image. The image generation unit generates a processed photographed image obtained by combining (for example, translucent combining) the processed person image with the stored background image. Then, the image transmission unit transmits the generated processed photographed image to another device.

In other words, in the present invention, a person image (user image) is extracted based on a face image detected from a captured image, and processing such as blurring is performed. Then, the processed person image is synthesized on the background image to generate a processed photographed image, which is transmitted to another apparatus. For this reason, even if the user does not want to show details of the face or skin, the details can be hidden by the processed photographed image that has been subjected to processing such as blurring, on the other hand, for the other party (other device), From the processed image to the user's facial expression can be grasped. Also, since the person image is combined with the background image, for example, even if something that is inconvenient is seen behind the user at the time of shooting, other than the user's image is not extracted, the inconvenience Some things can be properly hidden.
As a result, it is possible to generate an image that enables grasping of facial expressions while protecting the privacy of the user.

The image processing unit blurs the extracted person image,
The image generation unit may generate a processed photographed image obtained by translucently combining the blurred person image with the background image.
In this case, the details of the user's face, skin, and the like are appropriately hidden by the blurred human image, and the processed human image can be appropriately blended (blended) with the background image.

The image generation apparatus further includes a measurement unit that measures the amount of movement of the person from the sequentially captured images.
The image processing unit may change the blurring degree based on the measured amount of motion and blur the person image. For example, the image processing unit reduces the blurring degree as the measured amount of motion increases, and conversely increases the blurring degree as the amount of motion decreases. That is, when the amount of movement of the user is large, the details of the user's face and skin are appropriately hidden by the movement, so the blurring degree is reduced. On the other hand, when the amount of movement of the user is small, the details of the user's face and skin become apparent as in the stationary state, so the degree of blur is increased and the details are hidden.
In this case, the user's image can be appropriately processed according to the user's movement.

The image generation apparatus further includes a measurement unit that measures the degree of personality of the person from the sequentially captured images.
The image processing unit may process the person image by changing the degree of blurring based on the measured degree of performance. For example, the image processing unit does not need to hide the details of the user as the degree of performance measured by the measurement unit increases, so the degree of blurring is reduced. At that time, the image processing unit may reduce the degree of blurring only for a portion where the quality is well-organized. For example, if the user puts on a jacket, the blurring of the clothes is reduced, and if the user puts on lipstick, the blurring of the mouth is reduced, and further, if the user prepares the hair, Processes to reduce blur.
In this case, the user's image can be appropriately processed according to the degree of the user's personality.

  An image processing method according to a third aspect of the present invention is an image processing method in an image generation apparatus that includes a detection unit, an image extraction unit, an image processing unit, and an image generation unit. It comprises a step, an image processing step, and an image generation step.

  First, in the detection step, for example, a person is detected from a photographed image photographed by a camera or the like. In the image extraction step, the detected person image of the person is extracted from the photographed image. In the image processing step, for example, the extracted person image is subjected to blur processing. In the image generation step, a processed photographed image is generated by combining the processed person image with, for example, a background image on which no person is shown.

That is, in the present invention, a person image (user image) is extracted from the captured image and subjected to processing such as blurring. Then, the processed person image is synthesized on the background image to generate a processed photographed image. For this reason, even if the user does not want to show the details of the face and skin, it is possible to grasp the facial expression while hiding the details by the processed photographed image subjected to processing such as blurring. Also, since the person image is combined with the background image, for example, even if something that is inconvenient is seen behind the user at the time of shooting, other than the user's image is not extracted, the inconvenience Some things can be properly hidden.
As a result, it is possible to generate an image that enables grasping of facial expressions while protecting the privacy of the user.

  A program according to a fourth aspect of the present invention is configured to cause a computer (including an electronic device) to function as the above-described image generation device.

  This program can be recorded on a computer-readable information recording medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information recording medium can be distributed and sold independently of the computer.

  According to the present invention, it is possible to generate an image that enables grasping of an expression while protecting the privacy of the user.

It is a mimetic diagram showing appearance of a game device concerning this embodiment. It is a block diagram which shows the schematic structure of the game device which concerns on this embodiment. It is a block diagram which shows the structure of the image generation apparatus which concerns on this embodiment. (A), (b) is a schematic diagram which shows an example of a background image. (A), (b) is a schematic diagram for demonstrating a mode that a person (user) is detected. (A)-(c) is all the schematic diagrams for demonstrating a mode that a person image is extracted. (A) is a schematic diagram which shows an example of the person image which carried out the blurring process, (b) is a schematic diagram which shows an example of the person image which carried out the painterly process. (A), (b) is a schematic diagram which shows an example of the produced | generated processed image. (A), (b) is a schematic diagram which shows an example of the video chat screen displayed on a display part. It is a flowchart for demonstrating the image processing process which concerns on this embodiment. It is a block diagram which shows the structure of the image generation apparatus which concerns on other embodiment. (A)-(e) is all the schematic diagrams for demonstrating a mode that the amount of motions of a face is calculated | required. It is a schematic diagram which shows an example of the process picked-up image with which the characteristic site | part was synthesize | combined.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is applied to a portable game machine (game device) will be described. However, the present invention is similarly applied to information processing devices such as various computers, PDAs, and mobile phones. The invention can be applied. In other words, the embodiments described below are for illustrative purposes and do not limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

(Embodiment 1)
FIG. 1 is a schematic diagram showing an appearance of a typical game device in which the image generation device according to the embodiment of the present invention is realized. As an example, the game apparatus 1 is a portable game device, and includes a first display unit 18 and a second display unit 19, and a camera 21 is disposed between them. Yes. A touch panel 20 is disposed on the front surface of the second display unit 19.

  FIG. 2 is a schematic diagram showing a schematic configuration of the game apparatus 1. Hereinafter, the game apparatus 1 will be described with reference to FIG.

  The game apparatus 1 includes a processing control unit 10, a connector 11, a cartridge 12, a wireless communication unit 13, a communication controller 14, a sound amplifier 15, a speaker 16, an operation key 17, and a first display unit 18. A second display unit 19, a touch panel 20, and a camera 21.

  The processing control unit 10 includes a CPU (Central Processing Unit) core 10a, an image processing unit 10b, a DMA (Direct Memory Access) controller 10c, a VRAM (Video Random Access Memory) 10d, a WRAM (Work RAM) 10e, An LCD (Liquid Crystal Display) controller 10f and a touch panel controller 10g are provided.

  The CPU core 10a controls the overall operation of the game apparatus 1 and is connected to each component to exchange control signals and data. Specifically, a program and data stored in a ROM (Read Only Memory) 12a in the cartridge 12 are read out with the cartridge 12 mounted in the connector 11, and predetermined processing is executed.

  The image processing unit 10b processes the data read from the ROM 12a in the cartridge 12 and the data processed by the CPU core 10a, and stores the processed data in the VRAM 10d. For example, the image stored in the WRAM 10e is processed and stored in the VRAM 10d.

  The DMA controller 10c transfers the captured image captured by the camera 21 to the WRAM 10e or the like.

The VRAM 10d is a memory for storing information for display, and stores image information processed by the image processing unit 10b and the like.
The WRAM 10e stores work data and the like necessary when the CPU core 10a executes various processes according to the program.
The WRAM 10e temporarily stores the image transferred from the DMA controller 10c.

  The LCD controller 10f controls the first display unit 18 and the second display unit 19 to display a predetermined display image. For example, the LCD controller 10 f converts the image information stored in the VRAM 10 d into a display signal at a predetermined synchronization timing, and outputs it to the first display unit 18. Similarly, the LCD controller 10 f displays a predetermined display image on the second display unit 19.

  When the touch panel 20 is pressed with a touch pen or a player's finger, the touch panel controller 10g acquires the coordinates (input coordinates).

  The connector 11 is a terminal that can be detachably connected to the cartridge 12, and transmits / receives predetermined data to / from the cartridge 12 when the cartridge 12 is connected.

The cartridge 12 includes a ROM 12a and a RAM (Random Access Memory) 12b.
The ROM 12a stores a program for realizing the game and image data, sound data, and the like accompanying the game.
The RAM 12b stores various data indicating the progress of the game.

The wireless communication unit 13 is a unit that performs wireless communication with the wireless communication unit 13 of another game apparatus 1 and transmits / receives predetermined data via an antenna (not shown) (built-in antenna or the like).
The wireless communication unit 13 can also perform wireless communication with a predetermined wireless access point. The wireless communication unit 13 is assigned a unique MAC (Media Access Control) address.

The communication controller 14 controls the wireless communication unit 13 and mediates wireless communication performed between the process control unit 10 and the process control unit 10 of another game device 1 in accordance with a predetermined protocol.
Further, when the game apparatus 1 connects to the Internet via a nearby wireless access point or the like, wireless communication performed between the processing control unit 10 and the wireless access point or the like is performed according to a protocol compliant with the wireless LAN. Intermediary communication.

The sound amplifier 15 amplifies the audio signal generated by the processing control unit 10 and supplies it to the speaker 16.
The speaker 16 is composed of, for example, a stereo speaker, and outputs predetermined music sound, sound effect, and the like according to the audio signal amplified by the sound amplifier 15.

  The operation key 17 includes a plurality of key switches and the like appropriately arranged on the game apparatus 1 and receives a predetermined instruction input according to a user operation.

  The 1st display part 18 and the 2nd display part 19 consist of LCD etc., are controlled by LCD controller 10f, and display a game image etc. suitably.

The touch panel 20 is arranged so as to be superimposed on the front surface of the second display unit 19 and detects an input by a touch pen or a player's finger.
For example, the touch panel 20 includes a resistive film type touch sensor panel and the like, detects a press (pressing) with a touch pen or the like, and outputs information (signal or the like) according to the coordinates.

  As an example, the camera 21 includes a single-focus lens and an image sensor (CMOS, CCD, etc.) having a predetermined number of pixels. Then, the camera 21 captures an image in a predetermined imaging range with the front direction of the first display unit 18 as an optical axis. For example, a subject such as a user who performs a video chat is photographed.

In such a configuration, when the cartridge 12 in which the program and data for video chat are recorded in the ROM 12a is mounted and the game apparatus 1 is turned on, the program is executed, and the image generation according to the present embodiment is performed. A device is realized.
Hereinafter, an image generation apparatus realized by such a game apparatus 1 will be described.

(Outline of image generation device)
FIG. 3 is a block diagram showing a schematic configuration of the image generation apparatus 100 according to the present embodiment. As an example, the image generation apparatus 100 is an apparatus used for video chatting with another image generation apparatus 100 connected via a predetermined network, and the first display unit 18 described above. For example, an image of the user (user himself) captured by the camera 21 can be transmitted while displaying an image of the other user or the like. Hereinafter, the image generation apparatus 100 will be described with reference to the drawing, taking as an example the case where such a video chat is performed.

  As illustrated in FIG. 3, the image generation apparatus 100 includes a photographing unit 110, an image storage unit 120, a detection unit 130, an image extraction unit 140, an image processing unit 150, an image generation unit 160, and a communication unit 170. And a display control unit 180 and a display unit 190.

First, the photographing unit 110 photographs a photographed image obtained by photographing a predetermined subject. For example, a captured image including a user (person) who performs video chat is captured.
Specifically, when performing video chat, the photographing unit 110 sequentially captures images including the user, and supplies the captured images (for example, moving images) to the detection unit 130 and the like.
In addition, before the video chat, the photographing unit 110 photographs a background image (a still image as an example) in which the user is not captured, and supplies the photographed background image to the image storage unit 120.
Note that the above-described camera 21 can function as such a photographing unit 110.

The image storage unit 120 stores an image previously captured by the camera 21. For example, a background image HG1 that is photographed by the camera 21 before the video chat and in which the user is not shown as shown in FIG. 4A is stored.
Note that the image storage unit 120 may store not only the background image captured by the camera 21 but also other background images. For example, a background image photographed by a predetermined photographing device (digital camera or the like), or a fictitious background image HG2 as shown in FIG. 4B generated by 3D graphics or the like may be stored.
The RAM 12b and the like of the cartridge 12 described above can function as such an image storage unit 120.

Returning to FIG. 3, the detection unit 130 detects the user (person) from the captured image captured by the imaging unit 110.
For example, the detection unit 130 detects a user's face image shown in the captured image by face recognition. Specifically, the detection unit 130 searches the skin color areas A1 to A4 from the photographed image SG as shown in FIG. 5A, and determines whether or not a human facial feature part can be extracted from these areas. To do. In this case, as shown in FIG. 5B, since at least a characteristic part including both eyes and mouth can be extracted from the area A1 and recognized as a human face, the detection unit 130 can detect the face image G1 of the user. Is detected.
Note that such a method of detecting a user (person) is an example, and other methods may be used. For example, a user may be detected by discriminating a region having a change (movement or the like) from the preceding and following photographed images and determining whether or not a human characteristic part can be extracted from the region.
The CPU core 10a and the image processing unit 10b of the processing control unit 10 described above can function as such a detection unit 130.

Returning to FIG. 3, the image extraction unit 140 extracts the human image of the user detected by the detection unit 120 from the captured image.
For example, the image extraction unit 140 extracts a user's person image from the captured image based on the detected face image and surrounding images.
Specifically, the image extraction unit 140 specifies a predetermined number of pixels at a position corresponding to the head (hair) from the face image G1 as shown in FIG. 6A and the surrounding image, and approximates these pixels. Edge detection based on the color to be performed. Then, the person image G2 including the head as shown in FIG. 6B is specified.
Furthermore, the image extraction unit 140 specifies a predetermined number of pixels at positions corresponding to the body portion from the person image G2 and the surrounding images, and performs edge detection based on colors that approximate those pixels. Then, the person image G3 including the trunk as shown in FIG. 6C is specified.
The image extraction unit 140 extracts the person image G3 specified in this way from the captured image.
Note that such a method for extracting a person image of a user is an example, and other methods may be used. For example, when the detection unit 130 detects the entire person from the change (movement or the like) of the previous and subsequent captured images, the entire person may be extracted from the captured image as it is.
The CPU core 10a, the image processing unit 10b, and the like of the processing control unit 10 described above can function as such an image extraction unit 140.

Returning to FIG. 3, the image processing unit 150 processes the person image extracted by the image extraction unit 140.
For example, the image processing unit 150 blurs the extracted person image. Specifically, the image processing unit 150 replaces the color of each pixel of the person image with the average value of the colors of adjacent pixels (eight pixels that are diagonally adjacent in the vertical and horizontal directions as an example), as shown in FIG. It is processed into a human image BG that is blurred as shown.
Note that such blur processing is an example, and other processing may be performed on the person image. For example, the image processing unit 150 may convert a person image into a painting style. For example, the image processing unit 150 processes the image into a person image KG converted into a painting tone as shown in FIG. 7B by a filter operation for conversion into a painting tone (for example, a watercolor painting tone).
In addition, such blurring processing and painting-style conversion processing techniques are examples, and other techniques may be used to perform blurring processing and painting-style conversion.
The CPU core 10a, the image processing unit 10b, and the like of the processing control unit 10 described above can function as such an image processing unit 150.

Returning to FIG. 3, the image generation unit 160 combines the person image processed by the image processing unit 150 on the background image read from the image storage unit 120 to generate a processed photographed image.
For example, the image generation unit 160 synthesizes the above-described blurred person image BG of FIG. 7A on the background image of FIG. 4A described above, as shown in FIG. A processed photographed image is generated.
In addition, the image generation unit 160 may synthesize a person image by making it translucent. For example, the image generation unit 160 appropriately lowers the alpha value (alpha value of each pixel) of the entire human image BG in FIG. 7A, and then translucently combines the background image in FIG. By performing alpha blending, a processed photographed image as shown in FIG. 8B is generated.
Then, the image generation unit 160 supplies the generated processed image to the communication unit 170 and the display control unit 180.
Note that the synthesis methods such as these are merely examples, and the processed photographed image may be generated by other methods. For example, the image generation unit 160 generates a processed photograph image in which the boundary between the person image and the background image is inconspicuous by lowering only the alpha value of the outer edge portion of the person image and then translucently combining it on the background image. May be.
The CPU core 10a, the image processing unit 10b, and the like of the processing control unit 10 described above can function as such an image generation unit 160.

Returning to FIG. 3, the communication unit 170 controls communication with another image generation apparatus 100 connected via a predetermined network.
Specifically, the communication unit 170 includes an image transmission unit 171 and an image reception unit 172.
The image transmission unit 171 transmits the processed captured image generated by the image generation unit 160 to another image generation apparatus 100. That is, the processed captured image obtained by processing the user's person image and combining the background image is transmitted to the image generation apparatus 100 of the video chat partner.
On the other hand, the image reception unit 172 receives an image transmitted from another image generation apparatus 100. That is, if the processed image is similarly generated in the image generation apparatus 100 of the video chat partner, the processed image is received.
Note that the above-described wireless communication unit 13, communication controller 14, and the like can function as such a communication unit 170.

The display control unit 180 displays on the display unit 190 a video chat screen in which the image received by the image receiving unit 172 and the processed captured image generated by the image generation unit 160 are combined.
For example, the display control unit 180 uses the image received by the image reception unit 172 as a main, and combines the processed photographed image generated by the image generation unit 160 with a part of the image as appropriate. Specifically, as shown in FIG. 9A, the display control unit 180 generates a video chat screen in which the reduced processed image KSG1 is synthesized on the upper right of the image ATG1 from the video chat partner.
Note that the combining method of the display control unit 180 is an example, and both images may be combined by another method. For example, as shown in FIG. 9B, an image ATG2 from a video chatting partner and a processed photographed image KSG2 trimmed at the center may be combined and combined.
The CPU core 10a, the image processing unit 10b, and the like of the processing control unit 10 described above can function as such a display control unit 180.

The display unit 190 displays the video chat screen synthesized by the display control unit 180. That is, a video chat screen as shown in FIGS. 9A and 9B is displayed.
The first display unit 18 and the second display unit 19 described above can function as such a display unit 190.

(Outline of operation of image generation device)
Hereinafter, the operation of the image generating apparatus 100 having such a configuration will be described with reference to the drawings. As an example, the processing operation during video chat will be described with reference to FIG. FIG. 10 is a flowchart showing a flow of image processing performed by the image generation apparatus 100. It is assumed that the image storage unit 120 stores a background image.

First, the image generation apparatus 100 captures a captured image including the user (step S201).
That is, the imaging unit 110 sequentially captures images including a user who performs a video chat, and supplies the captured images (for example, moving images) to the detection unit 130.

The image generation device 100 detects a user (person) from the captured image (step S202).
That is, the detection unit 130 detects the face image of the user shown in the captured image by face recognition. That is, as shown in FIGS. 5A and 5B, the user's face image is detected.

The image generation apparatus 100 extracts the detected person image of the user from the captured image (step S203).
That is, the image extraction unit 140 extracts a user's person image from the captured image based on the detected face image and surrounding images. That is, as shown in FIGS. 6A to 6C described above, a person image is specified, and the specified person image is extracted from the captured image.

The image generation device 100 processes the extracted person image (step S204).
That is, the image processing unit 150 processes the person image BG as shown in FIG. In addition to this, the image processing unit 150 may process the person image KG as shown in FIG.

The image generating apparatus 100 combines the processed person image with the background image to generate a processed photographed image (step S205).
That is, the image generation unit 160 combines the blurred human image with the background image read from the image storage unit 120 to generate a processed photographed image as shown in FIG. At that time, the image generation unit 160 may generate a processed photographed image as shown in FIG.

The image generation device 100 transmits the generated processed image to the other image generation device 100, and receives an image sent from the other image generation device 100 (step S206).
That is, the image transmission unit 171 transmits the processed captured image generated by the image generation unit 160 to the image generation apparatus 100 of the video chat partner. On the other hand, the image receiving unit 172 receives an image transmitted from the image generation apparatus 100 of the video chat partner.

The image generating apparatus 100 appropriately synthesizes and displays the received image and the generated processed image (step S207).
That is, the display control unit 180 synthesizes the image received by the image reception unit 172 and the processed captured image generated by the image generation unit 160, as shown in FIGS. 9A and 9B described above. A video chat screen is displayed on the display unit 190.

The image generating apparatus 100 determines whether the video chat has ended (step S208).
When determining that the video chat has not ended (step S208; No), the image generating apparatus 100 returns the process to step S201 and repeatedly executes the processes of steps S201 to S208 described above.
On the other hand, when it is determined that the video chat has ended (step S208; Yes), the image generation device 100 ends the image processing.

In such an image processing process, the image generation apparatus 100 extracts a person image (user image) based on a face image detected from a captured image and performs a process such as blurring. Then, the processed person image is synthesized on the background image to generate a processed photographed image, which is transmitted to another apparatus. For this reason, even if the user does not want to show the details of the face and skin, the details can be hidden by the processed photographed image subjected to the processing such as blurring. On the other hand, since the other party (the user of the other image generation apparatus 100) can grasp the processed photograph image sent to the user's facial expression, the user's state of the transmission source can be known, and the advantages of video chatting Is not disturbed.
Also, since the extracted person image is combined with the background image, for example, even if something that is inconvenient is seen behind the user during video chatting, other than the user image is not extracted, Such inconveniences can also be appropriately hidden.
As a result, it is possible to generate an image that enables grasping of facial expressions while protecting the privacy of the user.

(Other embodiments)
In the above embodiment, the case where the human image extracted from the captured image is processed uniformly by the image processing unit 150 has been described. Details such as the user's face and skin are difficult to see due to movement. Therefore, the degree (degree) of processing may be changed as appropriate according to the amount of movement of the user.
Hereinafter, another embodiment in which the degree of processing is appropriately changed according to the amount of movement of the user will be described with reference to the drawings.

FIG. 11 is a block diagram showing a schematic configuration of an image generation apparatus 300 according to another embodiment of the present invention. The image generation apparatus 300 has a configuration in which a measurement unit 310 is added to the image generation apparatus 100 of FIG. 3 described above.
That is, the configuration of the photographing unit 110 to the display unit 190 is the same as that of the image generation device 100 in FIG.

The measurement unit 310 measures the amount of movement of the user's face from the sequentially captured images. For example, the measurement unit 310 measures the amount of movement of the user's face from the change in the position of the face image and the orientation of the face image sequentially detected by the detection unit 130.
Specifically, as shown in FIG. 12A, the measurement unit 310 obtains the positions (P1, P2) of the eyes and the position (P3) of the mouth as the feature points of the detected face image KG. The amount of movement of the face is measured from the difference from the obtained position. That is, a predetermined number of the positions of the feature points of the face image are stored in time series, and the amount of motion of the user's face is measured from the difference from the past feature point positions. Note that the positions of both eyes and the mouth are obtained as feature points, but the positions may be obtained by using other parts (such as the nose and ears) as feature points to measure the amount of movement of the face.

Further, as shown in FIG. 12B, the measurement unit 310 obtains the face orientation from a plane (specifically, a triangle) F1 connecting both eyes and the mouth, which are feature points of the detected face image KG. Then, the amount of movement (twist, etc.) of the face is measured from the difference from the previously obtained face orientation. That is, in the case of the plane F1, as shown in FIG. 12C, the orientation of the user's face is required to be the front direction GL1 from the arrangement (balance) of both eyes and the mouth. On the other hand, when a distorted plane F2 as shown in FIG. 12D is obtained, as shown in FIG. 12E, the user's face orientation is required to be an oblique direction GL2. Therefore, the measurement unit 310 stores a predetermined number of such face orientations in time series, and measures the amount of motion of the user's face from the difference from the past face orientation. In this case as well, the face orientation is obtained from the plane connecting both eyes and the mouth, but the face orientation is obtained from the plane with other parts (such as the nose and ears) as feature points, and the amount of movement of the face May be measured.
The CPU core 10a, the image processing unit 10b, and the like of the processing control unit 10 described above can function as such a measurement unit 310.

Returning to FIG. 11, when the movement amount of the user's face is measured by such a measurement unit 310, the image processing unit 150 changes the degree of processing based on the measured movement amount, and Process the image.
For example, the image processing unit 150 processes the person image by increasing the degree of blurring as the measured amount of motion increases and conversely increasing the degree of blurring as the amount of motion decreases. That is, when the amount of movement of the user is large, the details of the user's face and skin are appropriately hidden by the movement, so the blurring degree is reduced. On the other hand, when the amount of movement of the user is small, the details of the user's face and skin become apparent as in the stationary state, so the degree of blur is increased and the details are hidden.
The image processing unit 150 may similarly change the degree of conversion based on the measured amount of motion when performing processing for converting a person image into a painting style.
In these cases, the user's image can be appropriately processed according to the user's movement.

In the other embodiments described above, the case where the degree of processing is appropriately changed according to the amount of motion of the user has been described, but the degree of processing may be changed as appropriate in addition to the amount of motion of the user.
For example, when it is no longer necessary to hide details such as the user's face and skin, the degree of processing may be reduced (loosened).
Specifically, the measurement unit 310 accumulates the user's images as appropriate, and compares the previous image with the previous image to measure the degree of quality of the current image. For example, as you get dressed up compared to previous images, such as changing clothes (wearing a jacket), putting on makeup (a woman puts on lipstick), or straightening hair, Measure higher numbers.
Then, the image processing unit 150 reduces the degree of blurring because it is not necessary to hide the user's details as the degree of performance measured by the measurement unit 310 increases.
At that time, the image processing unit 150 may reduce the blurring degree only for a portion where the quality is well-organized. For example, if the user puts on a jacket, the blurring of the clothes is reduced, and if the user puts on lipstick, the blurring of the mouth is reduced, and further, if the user prepares the hair, Processes to reduce blur.
In this case, the user's image can be appropriately processed according to the degree of the user's personality.

In the above embodiment, the case where the entire processed human image is combined on the background image has been described. However, the combination is not limited to the entire human image, and only a part of the human image is combined on the background image. May be.
For example, a part of the user's face may be extracted and synthesized on the background image. That is, only a part of the face where the user's facial expression can be grasped is extracted and synthesized on the background image.
Specifically, as illustrated in FIG. 5B described above, the detection unit 130 extracts feature parts of the user's face (for example, eyebrows, eyes, nose, ears, and mouth).
Then, the image generation unit 160 combines the feature image of the user's face extracted by the detection unit 130 on the background image read from the image storage unit 120 to generate a processed photographed image. That is, as shown in FIG. 13, the image generation unit 160 generates a processed photographed image obtained by combining the feature part image TB on the background image.
In this case, only the facial expression appears to be floating from the feature part image TB, and the user's facial expression can be appropriately grasped from the feature part image TB.

  In the above embodiment, the case where the image generation devices 100 and 300 of the present application are applied to the portable game device 1 has been described. However, the present invention is not limited to such a portable game device 1, and for example, a notebook type A personal computer etc. may be sufficient.

  As described above, according to the present invention, it is possible to provide an image generation apparatus, an image processing method, and a program that can generate an image that enables grasping of an expression while protecting the privacy of the user.

DESCRIPTION OF SYMBOLS 1 Game device 10 Processing control part 11 Connector 12 Cartridge 13 Wireless communication part 14 Communication controller 15 Sound amplifier 16 Speaker 17 Operation key 18 1st display part 19 2nd display part 20 Touch panel 21 Camera 100,300 Image generation apparatus 110 Photographing Unit 120 image storage unit 130 detection unit 140 image extraction unit 150 image processing unit 160 image generation unit 170 communication unit 180 display control unit 190 display unit 310 measurement unit

Claims (5)

A detection unit for detecting a person from the captured image;
An image extraction unit for extracting the detected person image of the person from the captured image;
A measurement unit that measures the amount of movement of the person from the sequentially captured images;
An image processing unit for processing the extracted person image;
An image generation unit that generates a processed captured image obtained by combining the processed person image on a predetermined background image ,
The image processing unit reduces the blurring degree as the amount of motion measured by the measuring unit is larger, and blurs the person image by increasing the degree of blurring as the amount of motion is smaller.
An image generation apparatus characterized by that. An image storage unit for storing a predetermined background image;
A shooting section for shooting a shot image including a person;
A detection unit that detects a face image of the person by face recognition from the captured image;
An image extraction unit that extracts a person image of the person from the captured image based on the detected face image and surrounding image;
A measurement unit that measures the amount of movement of the person from the sequentially captured images;
An image processing unit for processing the extracted person image;
An image generation unit that generates a processed photographed image obtained by combining the processed person image on the stored background image;
An image transmission unit that transmits the generated processed image to the other device ;
The image processing unit reduces the blurring degree as the amount of motion measured by the measuring unit is larger, and blurs the person image by increasing the degree of blurring as the amount of motion is smaller.
An image generation apparatus characterized by that. The image generation apparatus according to claim 1 or 2,
The image processing unit blurs the extracted person image,
The image generation unit generates a processed photographed image obtained by translucently synthesizing the blurred person image on the background image.
An image generation apparatus characterized by that. An image processing method in an image generation apparatus having a detection unit, an image extraction unit, a measurement unit, an image processing unit, and an image generation unit,
A detection step in which the detection unit detects a person from the captured image;
An image extraction step in which the image extraction unit extracts the detected person image of the person from the captured image;
The measurement unit measures the amount of movement of the person from the sequentially captured images,
An image processing step in which the image processing unit processes the extracted person image;
The image generation unit includes an image generation step of generating a processed captured image obtained by combining the processed person image on a predetermined background image,
In the image processing step, the degree of blur is decreased as the amount of motion measured in the measurement step is larger, and the degree of blur is increased as the amount of motion is smaller, and the person image is blurred.
An image processing method characterized by the above. Computer
A detection unit for detecting a person from a captured image;
An image extraction unit for extracting the detected person image of the person from the captured image;
A measurement unit that measures the amount of movement of the person from the captured images taken sequentially,
An image processing unit for processing the extracted person image;
Function as an image generation unit that generates a processed captured image obtained by combining the processed person image on a predetermined background image;
The image processing unit functions to blur the person image by decreasing the degree of blurring as the amount of motion measured by the measuring unit increases, and increasing the degree of blurring as the amount of motion decreases. A program characterized by letting

Images