JP7575657B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, IMAGE PROCESSING PROGRAM, REGION SPECIFICATION MODEL, AND MODEL GENERATION METHOD - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies (1) The exhibition held at Girls Mignon Shinsaibashi GATE store on June 6 and 7, 2020

The present invention relates to an image processing device, an image processing method, and an image processing program for processing specific parts of a face that appear in a captured image, as well as an area identification model and a model generation method used in this image processing.

Conventionally, photo sticker creation devices are known that perform predetermined image processing on a captured image and provide the result to a user. Specifically, there are devices that generate multiple mask images with different shapes for a predetermined part and identify it as a target for image processing. (See, for example, Patent Document 1.)

Patent document 1 describes, for example, how an area of a specific color in the lower area of a face image is recognized as lips, and a mask image corresponding to the recognized lip area is generated to generate a lip image.

JP 2020-53831 A

In Patent Document 1, there is a risk that the desired area cannot be accurately identified. In such a case, there is a risk that the image processing of the target area cannot be performed correctly.

The present invention aims to solve the above problems and provide an image processing device, image processing method, and image processing program that identify a specific area of a user to be processed and perform image processing that is preferable for the user, as well as an area identification model and model generation method used in this image processing.

A first aspect of the image processing device according to the present invention includes a reception unit that receives image data, an identification unit that has learned the relationship between image data including a face and each of the pixels of a plurality of predetermined areas where at least a portion of the image data overlaps, and that, when new image data including a face is input, identifies pixels of the plurality of predetermined areas from the new image data including the face received by the reception unit using an area identification model that identifies pixels of the plurality of predetermined areas from the new image data, and an image processing unit that processes each of the plurality of areas identified from the image data in a predetermined manner.

This makes it possible to provide highly complete images for specific areas where there is overlap between users being processed.

The depth relationship in the image data for the multiple regions may be predetermined, and the image processing unit may overlay image data processed for the region located in the foreground on image data processed for the region located in the background.

This allows image data for specific areas to be overlaid with high precision to provide a highly complete image.

The identification unit may use semantic segmentation to identify each of the multiple regions.

This makes it easier to identify areas and provides a more complete image.

The first aspect of the image processing method according to the present invention includes a receiving step of receiving image data, a specifying step of specifying pixels of the multiple regions from the new image data including the face using a region specifying model that has already learned the relationship between the image data including a face and each of the pixels of multiple predetermined regions that at least a portion of the image data overlaps, and that, when new image data including a face is input, specifies the pixels of the multiple predetermined regions from the new image data, and a processing step of processing each of the multiple regions specified from the image data in the specifying step in a predetermined manner.

This makes it possible to provide highly complete images for specific areas where there is overlap between users being processed.

A first aspect of the image processing program of the present invention causes a computer to function as a reception unit that receives image data, an identification unit that has learned the relationship between image data including a face and each of the pixels of multiple predetermined areas where at least a portion of the image data overlaps, and that, when new image data including a face is input, identifies pixels of the multiple areas from the new image data including a face received by the reception unit using an area identification model that identifies pixels of the multiple predetermined areas from the new image data, and an image processing unit that processes each of the multiple areas identified from the image data in a predetermined manner.

This makes it possible to provide highly complete images for specific areas where there is overlap between users being processed.

The first aspect of the region identification model of the present invention has learned the relationship between image data including a face and each of the pixels in multiple predetermined regions with which the image data overlaps at least in part, and when new image data including a face is input, the model identifies the pixels in the multiple regions with which the image data overlaps at least in part from the new image data.

This makes it possible to provide highly complete images for specific areas where there is overlap between users being processed.

A first aspect of the model generation method according to the present invention includes a receiving step of receiving image data including a plurality of sets of faces and a plurality of data identifying pixels in a predetermined area where at least a portion of the image data overlaps, and a learning step of learning the relationship between each image data and each of the pixels in the plurality of predetermined areas of the image data, and when image data including new faces is input, a region identification model is generated from the new image data that identifies pixels in the plurality of areas where at least a portion of the image data overlaps.

This makes it possible to provide highly complete images for specific areas where there is overlap between users being processed.

The present invention can provide highly complete images even for specific areas where there is overlap between users being processed.

1 is a perspective view of a photo sticker creating device according to an embodiment of the present invention; FIG. FIG. 13 is a diagram for explaining the spatial movement of a user during a photo sticker creation game. FIG. 2 is a front view of the pre-selection unit. FIG. FIG. 2 is a front view of the background portion from the shooting space side. FIG. 2 is a front view of the editing unit from the editing space side. FIG. 2 is a left side view of the editing unit. 2 is a block diagram showing an internal configuration of the photo sticker creating device; FIG. 2 is a block diagram showing functional blocks of the photo sticker creating device. 11 is an example showing an area specified from image data in the photo sticker creating device. 13 is an example of an image of an eye in which an area is specified in the photo sticker creation device. 4 is a flowchart showing a series of operations relating to a photo sticker creating game by the photo sticker creating device. 13 is a flowchart showing details of a synthesis process. 13 is an example of an editing screen used in the editing process. FIG. 1 is a schematic diagram illustrating the generation of a region specific model. 1 is an example of input and output of a domain specific model.

Specific embodiments of the present invention will be described below with reference to the attached drawings. Components having the same functions are given the same reference numerals, and their description will be omitted as appropriate. Furthermore, the shapes of the components shown in each drawing, as well as the dimensions such as length, depth and width, do not reflect the actual shapes and dimensions, but have been changed as appropriate to clarify and simplify the drawings.

The image processing device according to the present invention will be described as an example of a photo sticker creation device. The photo sticker creation device to which the present invention is applied is a game device that allows a user to take and edit images as a game, and provides the taken and edited images to the user as photo stickers or data. Photo sticker creation device 1 is installed, for example, in game centers, shopping malls, and stores in tourist spots.

In a game provided by the photo sticker creation device, a user takes a photo of themselves or the like using a camera provided in the photo sticker creation device. The user edits the captured image by compositing a foreground image and/or background image, or by using editing functions such as pen image input or stamp image input (graffiti editing function) as an image to be composited for editing, to design the captured image into a colorful one. After the game is over, the user receives a photo sticker or the like with the edited image printed on it as a finished product. Alternatively, the photo sticker creation device can provide the edited image to the user's mobile device, and the user can receive the finished product via the mobile device.

(Configuration of the photo sticker creation device)
1 and 2 are perspective views showing an example of the external configuration of the photo sticker creating device 1. The photo sticker creating device 1 is a game machine that provides captured images and edited images. The photo sticker creating device 1 provides images to users by printing the images on sticker paper or transmitting the images to a server so that the images can be viewed on the user's mobile terminal. Users of the photo sticker creating device 1 are mainly young women such as female junior high school students and female high school students. In the photo sticker creating device 1, multiple users, mainly two or three people per group, can enjoy the game. Of course, in the photo sticker creating device 1, one user can also enjoy the game.

In the photo sticker creation device 1, the user takes a photo of themselves as the subject. Through editing, the user selects an image from the images obtained by shooting and combines it with a composite image such as handwritten text or a stamp image. In this way, the captured image is edited into a gorgeous image. The user receives a sticker paper on which the edited image, which is the edited image, is printed, and the series of games is completed.

As shown in FIG. 1, the photo sticker creation device 1 is basically configured by installing a photographing unit 11 and an editing unit 12 in contact with each other.

The photographing unit 11 is composed of a pre-selection operation section 20, a photographing operation section 21, and a background section 22. The pre-selection operation section 20 is installed on the side of the photographing operation section 21. The space in front of the pre-selection operation section 20 becomes the pre-selection space where the pre-selection processing is carried out. In addition, the photographing operation section 21 and the background section 22 are installed a predetermined distance apart. The space formed between the photographing operation section 21 and the background section 22 becomes the photographing space where the photographing processing is carried out.

As a pre-selection process, the pre-selection operation unit 20 performs guidance to introduce the games provided by the photo sticker creation device 1 and processes to allow the user to select various settings for the photography process carried out in the photography space. The pre-selection operation unit 20 is provided with a coin slot through which the user inserts payment, a touch panel monitor used for various operations, and the like. The pre-selection operation unit 20 appropriately guides users in the pre-selected space to the photography space depending on the availability of the photography space.

The shooting operation unit 21 is a device for shooting a user as a subject. The shooting operation unit 21 is located in front of the user who has entered the shooting space. On the front side of the shooting operation unit 21 facing the shooting space, a camera and a touch panel monitor used for various operations are provided. If the left side as seen from a user facing forward in the shooting space is the left side and the right side is the right side, the left side of the shooting operation unit 21 is composed of the side panel 41A and the right side is composed of the side panel 41B. Furthermore, the front side of the shooting operation unit 21 is composed of the front panel 42. The above-mentioned pre-selection operation unit 20 is installed on the side panel 41A. The pre-selection operation unit 20 may be installed on the side panel 41B, or on both the side panels 41A and 41B.

The background section 22 is composed of a rear panel 51, a side panel 52A, and a side panel 52B. The rear panel 51 is a plate-like member located behind a user facing forward. The side panel 52A is attached to the left end of the rear panel 51 and is a plate-like member narrower in width than the side panel 41A. The side panel 52B is attached to the right end of the rear panel 51 and is a plate-like member narrower in width than the side panel 41B.

Side panel 41A and side panel 52A are provided on approximately the same plane. The upper parts of side panel 41A and side panel 52A are connected by connecting part 23A, which is a plate-shaped member. The lower parts of side panel 41A and side panel 52A are connected by connecting part 23A', which is, for example, a metal member provided on the floor surface. Side panel 41B and side panel 52B are also provided on approximately the same plane. The upper parts of side panel 41B and side panel 52B are connected by connecting part 23B. The lower parts of side panel 41B and side panel 52B are connected by connecting part 23B'.

A green chromakey sheet, for example, is attached to the surface of the rear panel 51 facing the shooting space. The photo sticker creation device 1 performs chromakey compositing during the shooting process and editing process by taking a photo with the chromakey sheet as the background. This allows the background image desired by the user to be composited onto the sheet portion.

The opening formed by being surrounded by side panel 41A, connecting portion 23A, and side panel 52A serves as an entrance/exit to the photography space. The opening formed by being surrounded by side panel 41B, connecting portion 23B, and side panel 52B also serves as an entrance/exit to the photography space.

Above the shooting space, a ceiling is formed, surrounded by the front of the shooting operation unit 21, the connecting part 23A, and the connecting part 23B. A ceiling strobe unit 24 is provided in part of the ceiling. One end of the ceiling strobe unit 24 is fixed to the connecting part 23A, and the other end is fixed to the connecting part 23B. The ceiling strobe unit 24 has a built-in strobe that emits light into the shooting space in accordance with shooting. In addition to the strobe, a fluorescent lamp is provided inside the ceiling strobe unit 24. As a result, the ceiling strobe unit 24 also functions as lighting for the shooting space.

The editing unit 12 is a device for editing captured images. The editing unit 12 is connected to the capture unit 11 with one side in contact with the front panel 42 of the capture operation section 21.

If the configuration of the editing unit 12 shown in FIG. 1 is the configuration on the front side, components used for editing work are provided on both the front side and the back side of the editing unit 12. With this configuration, two groups of users can perform editing work at the same time.

The front side of the editing unit 12 is composed of a surface 61 and an inclined surface 62 formed above the surface 61. The surface 61 is perpendicular to the floor surface and is approximately parallel to the side panel 41A of the shooting operation unit 21. A tablet built-in monitor and a touch pen are provided on the inclined surface 62 as components used for editing work. A columnar support part 63A that supports one end of the lighting device 64 is provided on the left side of the inclined surface 62. A columnar support part 63B that supports the other end of the lighting device 64 is provided on the right side of the inclined surface 62. A support part 65 that supports the curtain rail 26 is provided on the upper surface of the support part 63A.

A curtain rail 26 is attached above the editing unit 12. The curtain rail 26 is composed of three rails 26A, 26B, and 26C that are combined together. The three rails 26A, 26B, and 26C are combined so that they form a U-shape when viewed from above. One end of rails 26A and 26B, which are arranged in parallel, is fixed to connecting portion 23A and connecting portion 23B, respectively, and the other ends of rails 26A and 26B are joined to both ends of rail 26C, respectively.

A curtain is attached to the curtain rail 26 so that the space in front of the front and the space in front of the rear of the editing unit 12 cannot be seen from the outside. The space in front of the front and the space behind the rear of the editing unit 12 enclosed by the curtain become the editing space where the user performs editing work.

As will be described later, an outlet through which printed sticker paper is discharged is provided on the left side of the editing unit 12. The space in front of the left side of the editing unit 12 becomes a print waiting space where the user waits for the printed sticker paper to be discharged.

(Movement of users)
Here, the flow of the image creation game and the associated movement of the user will be described. Fig. 3 is a diagram for explaining the spatial movement of the user during the photo sticker creation game.

First, the user inserts coins into the coin slot in the pre-selection space A0, which is the space in front of the pre-selection operation unit 20. Next, the user performs various settings according to the screen displayed on the touch panel monitor. For example, the user performs a pre-selection task such as selecting a course for photography to be performed in the photography space.

After completing the pre-selection task, the user enters the shooting space A1 formed between the shooting operation unit 21 and the background unit 22 from the entrance G1 between the side panel 41A and the side panel 52A, as indicated by the outlined arrow #1. The user then performs shooting work using the camera and touch panel monitor provided in the shooting operation unit 21.

When a user has finished taking photos, they can either leave the photo shoot space A1 through the entrance G1 and move to the editing space A2-1, as shown by the outlined arrow #2, or leave the photo shoot space A1 through the entrance G2 and move to the editing space A2-2, as shown by the outlined arrow #3.

Editing space A2-1 is the editing space on the front side of the editing unit 12. On the other hand, editing space A2-2 is the editing space on the rear side of the editing unit 12. The user is guided as to whether to move to editing space A2-1 or editing space A2-2 by a screen display on the touch panel monitor of the shooting operation unit 21. For example, the user is guided to the one of the two editing spaces that is vacant as the destination. The user who moves to editing space A2-1 or editing space A2-2 begins editing work. The user of editing space A2-1 and the user of editing space A2-2 can perform editing work at the same time.

After the editing work is completed, printing of the edited image begins. When printing begins, a user who has finished editing work in editing space A2-1 moves from editing space A2-1 to print waiting space A3, as indicated by outlined arrow #4. Also, a user who has finished editing work in editing space A2-2 moves from editing space A2-2 to print waiting space A3, as indicated by outlined arrow #5.

The user moves to the print waiting area A3 and waits for the image printing to finish. When printing is finished, the user receives the sticker paper that has been discharged from the discharge port on the right side of the editing unit 12, and the image creation game is completed.

(Configuration of pre-selection unit)
Next, the configuration of each device will be described.

A touch panel monitor 71 is provided above the pre-selection operation unit 20. The touch panel monitor 71 is composed of a monitor such as an LCD (Liquid Crystal Display) and a touch panel laminated on it. The touch panel monitor 71 has the function of displaying various GUIs (Graphical User Interfaces) and accepting selection operations by the user. The touch panel monitor 71 displays a screen used for pre-selection processing, which allows the user to select a course for photography, select an image to be used as the background of the image to be edited, select the layout of the image to be created, select at least one of BGM (Back Ground Music), sound and voice to be played during the image creation game, and input the user's name.

The photography courses available are a two-person course for two users to take photos together, and a large group course for three or more people to take photos together. There may also be a couple course for couples, where a man and a woman can take photos together.

A speaker 72 is provided below the touch panel monitor 71. The speaker 72 outputs voice guidance for the pre-selection process, background music, sound effects, and the like. In addition, a coin insertion/return slot 73 into which the user inserts coins is provided adjacent to the speaker 72.

(Configuration of the shooting unit)
5 is a front view of the photographing operation unit 21 serving as a photographing unit. The photographing operation unit 21 is configured to be surrounded by a side panel 41A, a side panel 41B, and a front panel 42.

A camera unit 81 is provided in the center of the front panel 42. The camera unit 81 is composed of a camera 91 and a touch panel monitor 92 as a display unit.

The camera 91 is, for example, a single-lens reflex camera, and is attached inside the camera unit 81 so that the lens is exposed. The camera 91 has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and captures images of users in the shooting space A1. The moving images captured by the camera 91 (hereinafter also referred to as live view display images) are displayed in real time on the touch panel monitor 92. A still image captured by the camera 91 at a specified timing, such as when shooting is instructed, is saved as a captured image.

The touch panel monitor 92 is provided below the camera 91. The touch panel monitor 92 is composed of a monitor such as an LCD and a touch panel laminated on it. The touch panel monitor 92 has a function as a live view monitor that displays moving images captured by the camera 91, and a function that displays various GUIs and accepts selection operations by the user. Specific examples of the selection operations include selection of a shooting course, instructions to start and end shooting (shooting control instructions), selection of eye deformation and color and skin color correction, selection of an image to be used as a background image in the created image, and selection of background music (sound/audio) during the image creation game. Moving images (live view images) and still images (taken images) captured by the camera 91 are displayed on the touch panel monitor 92.

An upper strobe 82 is installed above the camera unit 81, with its curved light-emitting surface facing the user. The upper strobe 82 shines light onto the user's face and upper body from above and in front.

Furthermore, a foot strobe 85 that shines light on the lower body and feet of the user is provided below the camera unit 81. The upper strobe 82 and foot strobe 85 each include a strobe and a fluorescent lamp.

Although not shown in Figs. 1 and 5, a speaker 93 is provided on the front panel 42, for example, near the ceiling. The speaker 93 outputs voice guidance for the shooting process, background music, sound effects, etc., in response to audio signals output from the control unit 201.

(Configuration of background part)
FIG. 6 is a front view of the background portion 22 on the side of the photographing space A1.

A rear upper strobe 101 is installed above the rear panel 51. The rear upper strobe 101 irradiates light to the user from above the rear.

In the figure, a left rear strobe 102 is installed to the left of the rear panel 51. The left rear strobe 102 illuminates the user from the right rear side. In the figure, a right rear strobe 103 is installed to the right of the rear panel 51. The right rear strobe 103 illuminates the user from the left rear side.

A chromakey sheet 121 may be attached to the surface of the rear panel 51 facing the shooting space A1 (the front side in the figure). The color of the chromakey sheet 121 is, for example, green.

Although not shown, a chromakey sheet may also be attached to the surface of the side panels 52A and 52B facing the shooting space A1, similar to the chromakey sheet 121.

(Editing unit composition)
FIG. 7 is a front view of the editing unit 12 on the editing space A2-1 side.

A tablet-built-in monitor 131 is provided at approximately the center of the inclined surface 62. A touch pen 132A is provided on the left side of the tablet-built-in monitor 131. A touch pen 132B is provided on the right side of the tablet-built-in monitor 131. The tablet-built-in monitor 131 is configured by providing a tablet with an exposed display. The tablet allows operation input using the touch pen 132A or the touch pen 132B. For example, an editing screen used for editing work is displayed on the tablet-built-in monitor 131. For example, when two users are performing editing work at the same time, the touch pen 132A is used by the user on the left side of the tablet-built-in monitor 131, and the touch pen 132B is used by the user on the right side of the tablet-built-in monitor 131.

Figure 8 is a left side view of the editing unit 12.

A sticker paper outlet 161 is provided on the lower left side of the editing unit 12. A printer is provided inside the editing unit 12 as an output section. The printer prints an image of a user of editing space A2-1 or an image of a user of editing space A2-2 in a specified layout on sticker paper, which is then discharged from the sticker paper outlet 161.

(Internal configuration of the photo sticker creation device)
Fig. 9 is a block diagram showing an example of the internal configuration of the photograph sticker creating device 1. In Fig. 9, the same components as those described above are given the same reference numerals, and duplicated explanations will be omitted as appropriate.

The control unit 201 is composed of a CPU (Central Processing Unit) and the like. The control unit 201 executes programs stored in a ROM (Read Only Memory) 206 and a storage unit 202, and controls the overall operation of the photo sticker creation device 1. The storage unit 202, communication unit 203, drive 204, ROM 206, and RAM (Random Access Memory) 207 are connected to the control unit 201. The components of the pre-selection operation unit 20, the photography operation unit 21, the background unit 22, the editing operation units 27A and 27B, and the printing operation unit 28 are also connected to the control unit 201.

The storage unit 202 is a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 202 stores various setting information and the like supplied from the control unit 201. The information stored in the storage unit 202 is read out by the control unit 201 as appropriate.

The communication unit 203 is an interface for a network such as the Internet. The communication unit 203 communicates with external devices according to the control of the control unit 201. For example, the communication unit 203 transmits captured images and edited images selected by a user to a server. The images transmitted from the communication unit 203 are assigned a predetermined memory area in the server and stored there, and are displayed or downloaded on a mobile terminal that has accessed the server.

A removable medium 205, such as an optical disk or semiconductor memory, is appropriately attached to the drive 204. Programs and data read from the removable medium 205 by the drive 204 are supplied to the control unit 201, and are stored in or installed in the memory unit 202.

The ROM 206 stores the programs and data executed by the control unit 201. The RAM 207 temporarily stores the data and programs processed by the control unit 201.

The pre-selection operation unit 20 performs pre-selection processing for users in the pre-selection space A0. The pre-selection operation unit 20 is composed of a touch panel monitor 71, a speaker 72, and a coin processing unit 74.

The touch panel monitor 71 displays various selection screens according to the control of the control unit 201 and accepts operations by the user on the selection screens. An input signal representing the content of the user's operation is supplied to the control unit 201, and various settings are made.

The coin processing unit 74 detects the insertion of coins into the coin insertion/return slot 73. When the coin processing unit 74 detects that a predetermined amount of coins has been inserted, it outputs a start signal to the control unit 201 to instruct the start of the game.

The shooting operation unit 21 realizes shooting processing for a user in the shooting space A1. The shooting unit 220 is composed of an upper strobe 82, a left strobe 83, a right strobe 84, a foot strobe 85, a camera 91, a touch panel monitor 92, and a speaker 93.

The upper strobe 82, left strobe 83, right strobe 84, and foot strobe 85 are arranged within the shooting space A1 and emit light according to the lighting control signal supplied from the control unit 201.

The camera 91 captures images according to shutter control by the control unit 201, and outputs the captured image (image data) obtained by the capture to the control unit 201.

Editing operation unit 27A realizes editing processing targeted at a user in editing space A2-1. Editing operation unit 27A is composed of a tablet built-in monitor 131, touch pens 132A and 132B, and a speaker 133. Editing operation unit 27B realizes editing processing targeted at a user in editing space A2-2, and has the same configuration as editing operation unit 27A. In the following, when there is no particular distinction between editing operation units 27A and 27B, they will simply be referred to as editing operation unit 27.

The tablet built-in monitor 131 displays an editing screen according to the control of the control unit 201 and accepts operations on the editing screen by the user. An input signal representing the content of the user's operation is supplied to the control unit 201, and the captured image to be edited is edited.

The print operation unit 28 performs a print process that provides sticker paper with a printed image to a user in the print waiting space A3. The print operation unit 28 is configured to include a printer 140. A sticker paper unit 141 is attached to the printer 140.

Based on the print data supplied from the control unit 201, the printer 140 prints the edited image onto sticker paper 142 stored in the sticker paper unit 141 and ejects it from the sticker paper ejection port 161.

(Function block of the photo sticker creation device)
10 is a block diagram showing the functional blocks of the photo sticker creating device 1. The photo sticker creating device 1 functions as a pre-selection unit 210, a photographing unit 220, an editing unit 230, and a printing unit 240. The control unit 201 also functions as a pre-selection processing unit 301, a photographing processing unit 302, a receiving unit 303, an extraction unit 304, a specification unit 305, an image processing unit 306, a synthesis unit 307, an editing processing unit 308, and a printing processing unit 309 by executing the photo sticker creating program of the present invention.

The pre-selection unit 210 includes the pre-selection operation unit 20 and the pre-selection processing unit 301. The pre-selection processing unit 301 performs pre-selection processing by controlling the touch panel monitor 71, the speaker 72, and the coin processing unit 74 in the pre-selection operation unit 20. The pre-selection processing unit 301 displays a selection screen for selecting a course related to photography performed in the photography space A1 on the touch panel monitor 71. The pre-selection processing unit 301 also accepts user operation input to the touch panel monitor 71. Specifically, the pre-selection processing unit 301 accepts operation input of selection on the selection screen displayed on the touch panel monitor 71, input of the user's name, etc. The pre-selection processing unit 301 also controls output of guidance explaining various selection operations. The pre-selection processing unit 301 displays a screen explaining various selection operations on the touch panel monitor 71, and outputs audio explaining various selection operations from the speaker 72.

The photographing unit 220 includes the photographing operation unit 21 and the photographing processing unit 302. The photographing processing unit 302 controls the camera 91, the touch panel monitor 92, and the speaker 93 in the photographing operation unit 21 to perform photographing processing.

The image capture processing unit 302 accepts operation input by the user on the touch panel monitor 92. For example, the image capture processing in the image capture space A1 is started when the image capture processing unit 302 accepts as input a touch operation by the user on the touch panel monitor 92.

The image capture processing unit 302 controls the camera 91 to capture an image of the user as a subject. The captured image can be a moving image or a still image. The image capture processing unit 302 controls the display of the touch panel monitor 92 to display a live view of the moving image captured by the camera 91 on the touch panel monitor 92, or to display a still image that is the result of the capture as a captured image.

The photography processing unit 302 also causes the touch panel monitor 92 to display instruction screens that explain the number of shots to be taken, the user's standing position, sample poses, messages about line of sight, and timing for taking pictures. Furthermore, the narration audio and background music corresponding to each instruction screen are output from the speaker 93.

The reception unit 303 receives image data of the user captured by the image capture unit 220.

The extraction unit 304 extracts the user's facial region from the image data accepted by the acceptance unit 303, and also extracts image data of the region to be processed. For example, in the example shown in FIG. 11, facial regions F1 and F2 are first extracted. Then, multiple processing regions E11, E12, E21, and E22 are extracted from each of the facial regions F1 and F2. These processing regions are the regions to be subjected to image processing by the image processing unit 306.

The identification unit 305 uses the area identification model M, which will be described later with reference to FIG. 17, to output a plurality of data items that identify pixels of a plurality of predetermined areas where at least a portion of the face overlaps, from the image data received by the reception unit 303. Specifically, the identification unit 305 outputs data items that identify pixels of a plurality of predetermined areas from the image data of each processing area extracted by the extraction unit 304. The plurality of predetermined areas include specific parts, such as the eyes and mouth, and at least a portion of the areas overlap. For example, when the area identification model M identifies two predetermined areas, the identification unit 305 outputs first data items related to information on pixels of the first area and second data items related to information on pixels of the second area from new image data received by the reception unit 303. Here, the "area" is a range identified from an image based on a predetermined criterion. In the example of FIG. 12, the "range identified according to eye parts" is described as the first area, and the "range identified according to eyelashes" is described as the second area. Therefore, the identification unit 305 outputs, from the image data of the processing area E11 shown in FIG. 12(a), first data D1 including pixel position information for the first area representing the eye parts shown in FIG. 12(b), and second data D2 including pixel position information for the second area representing the eyelashes shown in FIG. 12(c).

The first region identified by the first data D1 shown in FIG. 12(b) includes seven regions of the eye parts: double eyelid B01, pupil B02, iris B03, caruncle B04, white of the eye B05, mucous membrane B06, and other B07. Therefore, the first data D1 includes pixel position information for each of the parts B01 to B07 that make up this first region.

Here, the double eyelid B01 is the area from the eyeball to the first crease of the upper eyelid. In other words, the double eyelid B01 has a different meaning from the double eyelid commonly used, such as double eyelid, deep-set eyelid, single eyelid, etc. The pupil B02 is the area of the hole inside the iris B03 in the black of the eye. The iris B03 is a circular area formed by an outer circle that is the boundary between the white of the eye and the black of the eye, and an inner circle that is the boundary with the pupil B02. Note that the iris B03 is not formed in an exact circle, and is often not circular depending on how the eyes are opened. In the example shown in FIG. 12(b), the upper side of the iris B03 is partially hidden by the upper eyelid, and the lower side is also partially hidden by the lower eyelid, so it is a ring that is not a perfect circle. The caruncle B04 is the area inside the inner corner of the eye that secretes tear components. The white of the eye B05 is also called the sclera, and is a white area. Mucous membrane B06 is the wet area on the edge of the skin around the eyeball. Other B07 is an area other than the above-mentioned parts B01 to B06.

The second region specified by the second data D2 shown in FIG. 12(c) includes three regions: upper eyelashes B11, lower eyelashes B12, and others B13. Therefore, data D2 includes pixel position information for each of the portions B11 to B13 that make up this second region. Here, upper eyelashes B11 is the upper region. Lower eyelashes B12 is the lower region. Others is the region other than upper eyelashes B11 and lower eyelashes B12.

Therefore, in this description, each area identified by the identification unit 305 is assumed to include further subdivided subareas, as shown in Figures 12(b) and 12(c).

The first region related to the eye features in the first data D1 shown in FIG. 12(b) partially overlaps with the second region related to the eyelashes in the data D2 shown in FIG. 12(c). Specifically, the double eyelids B01 partially overlap with the upper eyelashes B11. The upper eyelashes B11 and the lower eyelashes B12 partially overlap with the rest B07. Here, the anteroposterior relationship of the multiple regions identified by the identification unit 305 is determined in advance. For example, the relationship between the first region related to the eye features shown in FIG. 12(b) and the second region related to the eyelashes shown in FIG. 12(c) is that the first region is behind and the second region is in front, based on the paper surface of the image. Specifically, the eyelashes B11 and B12 are in front of the eye parts B01 to B06. Such a anteroposterior relationship is determined in advance.

The identification unit 305 can identify each of the multiple regions, for example, by using semantic segmentation. By extracting each region using this semantic segmentation technology, it becomes possible to extract fine regions on a pixel-by-pixel basis for each region. This allows image processing to be performed using the set regions, making it possible to realize image processing that is set in detail according to each region.

The image processing unit 306 processes each of the multiple regions of the image data using a predetermined method. For example, the image processing unit 306 processes a first region of the image data using a predetermined first method, and processes a second region of the image data using a predetermined second method.

In the example shown in FIG. 12, the image processing unit 306 may perform a process to enlarge the shape of the iris B03 or a process to whiten the color of the white of the eye B05 for the first region of the image data specified by the first data D1 shown in FIG. 12(b). By performing image processing on the iris B03 and the white of the eye B05 in this way, it is possible to make the eyes look larger and clearer, thereby improving the aesthetic appearance. The image processing unit 306 may also perform a process to change the color of the iris B03 for the first region of the image data specified by the first data, as if wearing color contact lenses. Furthermore, the image processing unit 306 may perform image processing on the first region of the image data specified by the first data D1 as if makeup was applied to the parts of the eye. Specifically, the image processing unit 306 may perform a process to apply eyeshadow to the double eyelid B01 and the other B07 region within a predetermined range from the double eyelid B01, or a process to draw an eyeliner around the mucous membrane B06. The image processing unit 306 may perform a combination of the above-mentioned processes.

For example, for the second region of the image data identified by the second data D2 shown in FIG. 12(c), the image processing unit 306 may perform image processing such as lengthening or thickening the upper eyelashes B11 and the lower eyelashes B12 to give the appearance of false eyelashes or mascara.

As described above, the multiple regions identified by the identification unit 305 can be treated as different layers, and the depth relationship in the image data is predetermined. Therefore, the image processing unit 306 can ultimately overlay a layer of image data processed for a region located on the foreground side on a layer of image data processed for a region located on the background side. In the above example, the image processing unit 306 can overlay image data processed for a second region located on the front side on image data processed for a first region located on the rear side. Even if there are three regions, the processed image data can be overlaid according to the depth. This makes it possible to obtain natural image data, for example, with image-processed eyelashes overlaid on image-processed eye parts. In other words, there is no risk of unnatural image data in which eye parts are overlaid on eyelashes.

The compositing unit 307 uses the processed image as a compositing image, combines the compositing image with the moving image captured by the camera 91, and displays the combined image as a live view display image on the touch panel monitor 92. Therefore, the user can take a picture while checking the final image in real time.

The editing unit 230 includes the above-mentioned editing operation units 27A and 27B, and an editing processing unit 308. The editing processing unit 308 performs editing processing by controlling the tablet built-in monitor 131 and the speaker 133 in the editing operation units 27A and 27B. The editing processing unit 308 accepts operation input by the user using the touch pens 132A and 132B on the tablet built-in monitor 131.

In addition, the editing processing unit 308 performs a predetermined image processing on the captured image as the image to be edited in response to a selection operation on the selection screen displayed on the tablet built-in monitor 131, and displays the image on the tablet built-in monitor 131. Alternatively, the editing processing unit 308 performs a predetermined image processing on the image to be combined in response to an input operation on the editing screen displayed on the tablet built-in monitor 131, or generates a new image to be combined in response to the input operation, combines it with the captured image, and displays it on the tablet built-in monitor 131.

The editing processing unit 308 also controls the output of guidance that explains how to proceed with editing. For example, the editing processing unit 308 causes a guidance screen that explains how to proceed with editing to be displayed on the tablet built-in monitor 131, and causes audio guidance that explains how to proceed with editing to be output from the speaker 133. The editing processing unit 308 also controls the communication unit 203 to perform processing related to communication via a network such as the Internet. The editing processing unit 308 may also perform printing processing by controlling the printer 140 of the print operation unit 28.

The printing unit 240 includes the printing operation unit 28 and the printing processing unit 309 described above. The printing processing unit 309 receives print data from the editing processing unit 308 and performs printing processing by controlling the printer 140 of the printing operation unit 28. Note that, although the printing unit 240 that outputs photo stickers, which are print data, has been described here as an example of an output unit that outputs image data, the method of outputting image data is not limited to this. For example, the output unit may be a transmission means that transmits the image data itself to an external communication terminal using a network or the like.

In this way, in exchange for a fee, the photo sticker creation device 1 provides a photo sticker creation game with various mechanisms (selection of poses for taking photos, background music, narration, etc.) to excite the user. Therefore, the images created by the photo sticker creation device 1 are images that bring out the joyful expressions of the user and are elaborate and gorgeous images.

In addition, the photo sticker creation device 1 is fully equipped with lighting and other facilities, and can perform image transformation processing (for example, the size of the subject's eyes or the length of the legs) and color correction (such as whitening the subject's skin) using advanced technology, so the images created with the photo sticker creation device 1 will look good on the user.

In addition, editing (drawing on an image) is easier than image processing (for example, processing with a photo editing application) done outside of the photo sticker creation device 1, and there are many variations in the editing options. From this point of view, it can be said that images created with the photo sticker creation device 1 have a relatively gorgeous finish compared to images taken and processed outside of the photo sticker creation device 1.

(Photo sticker creation game flow)
Next, a process flow in which a user plays a photo sticker creating game in the photo sticker creating device 1 will be described with reference to Fig. 13. Fig. 13 is a flow chart showing a process flow from the start of the game in the photo sticker creating device 1 to creating a photo sticker in the game.

Before the game starts, the control unit 201, functioning as the pre-selection processing unit 301 of the photo sticker creation device 1, displays a message encouraging the user to insert coins on the touch panel monitor 71 of the pre-selection operation unit 20. As shown in FIG. 13, the control unit 201 also determines whether or not a coin has been inserted into the coin insertion/return slot 73 based on the presence or absence of a start-up signal from the coin processing unit 74 (S1). If the control unit 201 determines that no coins have been inserted into the coin insertion/return slot 73 (S1: NO), it continues the process of determining whether or not a coin has been inserted.

A user who wishes to start a game inserts a coin into the coin insertion/return slot 73 in the pre-selection space A0, which is the space in front of the pre-selection operation unit 20. When a coin is inserted into the coin insertion/return slot 73, a start signal is output from the coin processing unit 74 to instruct the start of the game. When the control unit 201 inputs the start signal from the coin processing unit 74, it determines that a coin has been inserted into the coin insertion/return slot 73 (S1: YES) and executes a pre-service process for the user (S2).

In the pre-customer service process, the control unit 201 causes the touch panel monitor 71 to display messages and the like encouraging the user to select a course, input a name, select a design, and the like. When the user makes various selections or inputs in accordance with the messages and the like displayed on the touch panel monitor 71, the control unit 201 sets the photography course, name, design, print layout, and the like. The control unit 201 causes the touch panel monitor 92 to display multiple selectable background images for compositing, and allows the user to select a background image for compositing.

When the pre-service processing is completed, the control unit 201 causes the touch panel monitor 71 to display a message or the like encouraging the user to move to the shooting space A1 to take a photo. The control unit 201, functioning as the shooting processing unit 302, causes the touch panel monitor 92 of the shooting operation unit 21 to display a message encouraging the user to touch the screen. A start button may be displayed together with or instead of this message. The control unit 201 also causes the speaker 93 to output a narration encouraging the user to touch the screen along with background music. When the user who has moved to the shooting space A1 touches the touch panel monitor 92, the control unit 201 reads that the touch panel monitor 92 has been touched and starts the shooting processing (S3).

In the shooting process, the control unit 201 displays, for example, guidance regarding lighting on the touch panel monitor 92, and prompts the user to select a lighting level. When the user selects a lighting level, the control unit 201 sets the lighting level to the selected level.

The control unit 201 also causes an instruction screen to be displayed on the touch panel monitor 92 to explain the number of shots to be taken, and causes the corresponding narration to be output from the speaker 93. In this embodiment, as an example, the number of shots to be taken is set to seven.

Next, the control unit 201 causes an instruction screen to be displayed on the touch panel monitor 92 to guide the user to a proper standing position, and causes the corresponding narration to be output from the speaker 93.

After displaying the instruction screen as described above, the control unit 201 causes the touch panel monitor 92 to display a live view display image, which is a combination of the background image for synthesis and the moving image captured by the camera 91, according to the shooting course selected by the user (S4). This allows the user to pose while checking the final image.

The control unit 201 displays the live view until just before the end of the countdown for shooting, during which the control unit 201 displays sample poses on the touch panel monitor 92 together with the live view display or in place of the live view display. The control unit 201 outputs narration that matches the sample poses from the speaker 93.

The control unit 201 manages the time from the start to the end of the live view display, and when a preset time has elapsed, a countdown is performed by displaying on the touch panel monitor 92 and playing audio from the speaker 93.

When the countdown ends, the control unit 201 sends a lighting control signal to the upper strobe 82, the left strobe 83, the right strobe 84, and the foot strobe 85, and also sends a shutter signal to the camera 91.

As a result, flashes are emitted from the upper strobe 82, left strobe 83, right strobe 84, and foot strobe 85, and a captured image showing the illuminated user together with the background is captured by the camera 91. In this embodiment, as an example, the process from step S3 to step S4 is repeated multiple times to capture seven captured images. The control unit 201 also stores in the memory unit 202 the image to be edited, which is the captured image combined with the background image to be combined.

Note that, although an example has been described here in which a photographed image is acquired at every preset time, the timing of acquiring the photographed image is not limited to this. For example, if the photographing operation unit 21 has an operation button for photographing, the photographed image may be acquired at the timing when this operation button is operated.

After shooting is completed, the control unit 201 displays a guidance screen on the touch panel monitor 92 encouraging the user to move to either editing space A2-1 or editing space A2-2, and outputs audio guidance to the speaker 93.

Then, the control unit 201 executes an editing process that allows the user to edit the image to be edited (S5). In more detail, the control unit 201 displays the image to be edited on the tablet built-in monitor 131, and allows the user to draw stamp images, pen images, etc. on the image to be edited with the touch pens 132A and 132B to create an edited image.

After that, the control unit 201 displays a guidance screen on the tablet's built-in monitor 131 to encourage the user to move to the print waiting space A3 where the sticker paper outlet 161 is located, and outputs audio guidance to the speaker 133.

Furthermore, the control unit 201 executes a printing process to create an image for printing by placing the edited image in the print layout selected in the pre-selection operation unit 20, and prints this image for printing on sticker paper 142 (S6).

When the printing process is completed, the control unit 201 executes the process of discharging the sticker paper 142 (S7), and discharges the printed sticker paper 142 from the sticker paper outlet 161, providing it to the user as a photo sticker, and ending the game. In this way, the photographed image of the user created by the photo sticker creation device 1 can be output as a photo sticker. Although a detailed explanation is omitted, the photo sticker creation device 1 of this embodiment can also output the photographed image to a mobile terminal or the like via communication.

(Details of the composition process)
Next, the compositing process in step S4 in the flowchart of Fig. 13 will be described with reference to the flowchart shown in Fig. 14. As shown in Fig. 14, in the compositing process in step S4, the control unit 201 receives an image captured by the imaging unit 220 (S41).

The control unit 201 extracts the facial area of the user included in the captured image accepted in step S41 (S42). If the captured image includes multiple users, the control unit 201 extracts the facial areas of all users. In the example of FIG. 11, the control unit 201 extracts facial areas F1 and F2.

The control unit 201 further identifies each area to be processed for the user whose face area was extracted in step S42 (S43). In the example of FIG. 1, areas E11 and E12 are identified from face area F1, and areas E21 and E22 are identified from face area F2.

The control unit 201 performs a predetermined process on the image data of each area extracted in step S43 (S44).

Furthermore, the control unit 201 may perform other image processing on the image data of the user's face area (S45). For example, the control unit 201 may process the image data to make the face smaller or to make the skin whiter according to a predetermined method. At this time, the control unit 201 may process the image data automatically or according to the user's operation.

The control unit 201 synthesizes an image using the images processed in steps S44 and S45 (S46). The image synthesized here is acquired while being confirmed by the user through a live view display.

(Display example)
15 is an example of an editing screen 400 displayed on the tablet built-in monitor 131. As shown in FIG.

The first editing section 401 includes a first thumbnail section 403, a first image to be edited display section 405, a first operation button display section 407, and a first palette 409. Similarly, the second editing section 402 includes a second thumbnail section 404, a second image to be edited display section 406, a second operation button display section 408, and a second palette 410.

Thumbnails of multiple captured images to be edited Im1 to Im5 are displayed in the first thumbnail section 403 and the second thumbnail section 404. The thumbnails displayed in the first thumbnail section 403 and the second thumbnail section 404 can be selected with the touch pen 132A or 132B. In the example shown in FIG. 15, images to be edited Im1 and Im3 are selected, respectively. Although not shown, the word "selected" may be displayed on the thumbnail of the selected image to be edited. Furthermore, if seven images are captured in step S3 of the flowchart in FIG. 13, seven images to be edited are displayed in the thumbnail sections 403 and 404, respectively, but for convenience, FIG. 15 will be described using an example in which five images to be edited are displayed.

The first image to be edited display section 405 displays the image to be edited selected in the first thumbnail section 403. The second image to be edited display section 406 displays the image to be edited selected in the second thumbnail section 404. When the image to be edited selected in the image to be edited displayed in the thumbnail section 403 or 404 is changed, the image to be edited displayed in the image to be edited display section 405 or 406 is also changed.

The first palette 409 is used to select the type of content, etc., when decorating the image to be edited displayed in the first image to be edited display section 405 with content such as text, patterns, colors, etc. The second palette 410 is used to select the type of content, etc., when decorating the image to be edited displayed in the second image to be edited display section 406 with content. As will be described in detail later, in the example shown in FIG. 15, each palette 409 and 410 includes tabs for "role play," "face doodle," "item," and "makeup."

As described above, in this embodiment, by extracting each area, it is possible to perform the processing expected for each area, thereby achieving image processing that is preferable for the user.

(area specific model)
The area identification model M outputs a plurality of data items that identify pixels in a plurality of predetermined areas from input image data.

To generate this region identification model M, as shown in FIG. 16, a set of multiple image data and data related to pixels in multiple specified regions of each image data are input to the learning device as a learning dataset. The learning device also learns the relationship between each image data and each of the pixels in the specified region. Specifically, the example shown in FIG. 16 is one in which two specified regions are learned, and the learning device learns the relationship between the image data, which is the input data, and the first data related to the first specified region, which is the correct answer data, and also learns the relationship between the image data, which is the input data, and the second data related to the second specified region, which is another correct answer data. Note that the specified regions, which are the correct answer data, overlap at least partially.

The region identification model M thus generated outputs multiple data identifying pixels of multiple predetermined regions from the new image data input. For example, as shown in FIG. 17, the region identification model M outputs first data of pixels identifying a first predetermined region and outputs second data of pixels identifying a second predetermined region for the image data.

Specifically, the image data, which is the input data, is data that includes a human face. For example, the first data D1 is data related to the eye parts described above with reference to FIG. 12(b), and the second data D2 is data related to the eyelashes described above with reference to FIG. 12(c). Therefore, in the example of FIG. 12, when image data including a face is input, the region identification model M identifies a first region that is a part of the eye, and identifies a second region that is a part of the eyelashes.

<Modification>
The above embodiment is merely an example, and various modifications are possible without departing from the scope of the present invention. For example, the example has been described using layers divided into eye parts and eyelashes, but the eye parts can be further divided into multiple layers. For example, the pupil B02, iris B03, and white of the eye B05 can be treated as one layer on the background side, and other parts, specifically the double eyelid B01, caruncle B04, and mucous membrane B06, can be treated as another layer on the front side.

The identification unit 305 may also use multiple consecutive frames of image data to obtain one piece of data as output. Specifically, for example, to obtain output for certain image data, the identification unit 305 may use the target image data and image data up to four frames prior to obtain multiple predetermined regions for the target image data. This makes it possible to identify a predetermined region even if the target image data hides the predetermined region, and provides a natural image. In other words, even if the eye is hidden by a finger or half-hidden by half-closing the eyelids, natural image processing is possible if the normal eye shape can be identified as the predetermined region for the invisible part.

It is also possible to manually switch the shooting interval. For example, a self-shutter button and a slide bar or the like are provided, and the self-shutter button is configured to determine the timing of shooting, rather than switching the shooting interval. Then, it is possible to change the shooting interval after operating the self-shutter button by operating the slide bar or the like.

Furthermore, in the above example, a photo sticker creation device was used as an example of an image processing device, but the same applies to any terminal that can acquire and process images, such as a smartphone or tablet.

(Example of software implementation)
The functional blocks of the photo sticker creating device 1 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit).

In the latter case, the photo sticker creation device 1 includes a CPU that executes the commands of a program, which is software that realizes each function, a ROM (Read Only Memory) or storage device (these are referred to as "recording media") in which the program and various data are recorded so as to be readable by the computer (or CPU), and a RAM (Random Access Memory) in which the program is expanded. The object of the present invention is achieved by the computer (or CPU) reading and executing the program from the recording medium. The recording medium may be a "non-transient tangible medium," such as a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The program may also be supplied to the computer via any transmission medium (such as a communication network or broadcast waves) capable of transmitting the program. The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments.

1. Photo sticker creation device (image processing device)
301 Pre-selection processing unit 302 Photography processing unit 303 Reception unit 304 Extraction unit 305 Identification unit 306 Image processing unit 307 Combination unit 308 Edit processing unit 309 Print processing unit (output unit)

Claims (5)

A reception unit that receives image data captured by a user ;
an identification unit that has learned a relationship between image data including a face and each of pixels in a plurality of predetermined regions where at least a portion of the image data overlaps and whose context is previously determined , and that, when new image data including the face of the user is input, identifies pixels of the plurality of predetermined regions from the new image data including the face received by the reception unit using an area identification model that identifies pixels of the plurality of predetermined regions from the new image data;
an image processing unit that processes each of the plurality of regions identified from the image data in a predetermined manner;
Equipped with
The image processing unit superimposes image data processed for an area located on the foreground side on image data processed for an area located on the background side.
Image processing device. The image processing device according to claim 1 , wherein the identification unit is configured to identify each of the plurality of regions by using semantic segmentation. A receiving step in which a user receives image data captured by the camera ;
a step of learning a relationship between image data including a face and each of pixels in a plurality of predetermined regions where at least a portion of the image data overlaps and whose context is predefined , and when new image data including the face of the user is input, identifying pixels of the plurality of predetermined regions from the new image data by using an area identification model that identifies pixels of the plurality of predetermined regions from the new image data;
a processing step of processing the plurality of regions identified from the image data in the identifying step by a predetermined method;
Including,
The processing step includes overlaying image data processed for an area located on the foreground side on image data processed for an area located on the background side.
Image processing methods. Computer,
A reception unit that receives image data captured by a user ;
an identification unit that has learned a relationship between image data including a face and each of pixels in a plurality of predetermined regions where at least a portion of the image data overlaps and whose context is previously determined , and that, when new image data including the face of the user is input, identifies pixels of the plurality of predetermined regions from the new image data including the face received by the reception unit using an area identification model that identifies pixels of the plurality of predetermined regions from the new image data;
an image processing unit that processes each of the plurality of regions identified from the image data in a predetermined manner;
and make it work .
An image processing program that causes the image processing unit to superimpose image data processed for an area located on the foreground side on image data processed for an area located on the background side . receiving image data including a plurality of sets of faces and a plurality of data identifying pixels in a predetermined area in which at least a portion of the image data overlaps and in which a context is previously determined ;
learning a relationship between each of the image data and each of the pixels in the plurality of predetermined regions of the image data;
A model generation method for generating, when new image data of a user is input, an area specifying model for specifying pixels of a plurality of areas that at least partially overlap the predetermined area from the new image data.