JP7590640B2 - Image processing device, image processing method, and program - Google Patents

Description

In particular, this technology relates to an image processing device, an image processing method, and a program that enable partial adjustment of the composition of multiple pieces of content.

Photo sticker creation devices are known in the art. Photo sticker creation devices photograph a user, allow the user to edit the photographed image, and print the edited image on sticker paper for provision. Photo sticker creation devices are installed in amusement facilities, etc.

The flow of one game provided by the photo sticker creation device typically involves taking a photo of the user in the shooting space as the subject, then moving the user to the editing space, editing the image according to the operations performed in the editing space, and printing the edited image on sticker paper.

After moving from the shooting space to the editing space, the images obtained by shooting are edited using an editing screen displayed on a display provided in the editing space. The editing screen displays the image to be edited, as well as buttons used to select various editing tools.

For example, Patent Document 1 discloses a free doodle function that can be used when editing a photographic image. The free doodle function is a function that allows a material image placed in a predetermined position to be combined with a material image placed in a position based on a person whose image is recognized, and placed with a single operation.

JP 2017-32780 A

The technology described in Patent Document 1 makes it easy to place multiple stock images on an image to be edited, but if the stock images are not placed in the appropriate positions, they cannot be adjusted.

This technology was developed in light of these circumstances, and allows partial adjustment of the composition of multiple pieces of content.

An image processing device according to one aspect of the present technology includes an image processing unit that, when an editing operation using content is performed, composites stamp images of different types constituting the content at each of a plurality of composite positions determined based on an analysis result of the characteristics of a person appearing in a captured image, adjusts the composite mode of an arbitrary stamp image selected from the composited stamp images in response to a user operation, and, when an operation to change the type of the content is performed, composites the stamp images constituting the changed content, whose composite positions are determined based on the same position as the arbitrary stamp image, by inheriting the composite mode after the adjustment .

In one aspect of the present technology, when an editing operation using a content is performed, stamp images of different types constituting the content are composited at each of a plurality of composite positions determined based on an analysis result of the characteristics of a person appearing in a photographed image, and a composite mode of an arbitrary stamp image selected from the composited stamp images is adjusted according to a user's operation. Also, when an operation to change the type of the content is performed, the stamp image constituting the changed content, the composite position of which is determined based on the same position as the arbitrary stamp image, is composited while inheriting the composite mode after adjustment.

This technology makes it possible to partially adjust the composition of multiple pieces of content.

FIG. 1 is a perspective view showing an example of the external configuration of a photo sticker creating device. FIG. 2 is a perspective view of the photo sticker creating device as seen from another angle. FIG. 2 is a perspective view showing an example of the configuration of the front of the imaging space. FIG. 2 is a perspective view showing an example of the configuration of a ceiling surface of a photography space. FIG. 13 is a diagram showing an example of a captured image. 1 is a block diagram showing an example of the configuration of a photo sticker creating device; 2 is a block diagram showing an example of a functional configuration of the photo sticker creating device; 2 is a block diagram showing an example of the configuration of an edit processing unit; 10 is a flowchart illustrating an operation of the photo sticker creating device. 10 is a flowchart illustrating the editing process performed in step S4 of FIG. 9. 11 is a flowchart illustrating a sticker design selection process performed in step S12 of FIG. 10. FIG. 13 is a diagram showing an example of a sticker design selection screen. FIG. 1 is an enlarged view of a sample image; FIG. 13 is a diagram showing examples of content that are components of a sticker design. FIG. 13 is a diagram illustrating an example of a template. FIG. 13 is a diagram showing an example of generation of an image for a sticker. FIG. 13 is an enlarged view of another sample image. FIG. 13 is a diagram showing examples of content that are components of a sticker design. FIG. 13 is a diagram showing an example of generation of an image for a sticker. FIG. 13 is a diagram showing an example of an image selection screen. FIG. 13 is a diagram showing an example of an image for a sticker. FIG. 13 is a diagram showing an example of a printed sticker sheet. FIG. 13 is a diagram showing a display example of a mobile-only graffiti screen. FIG. 13 is a diagram showing an editing process using a role-playing stamp. FIG. 25 is an enlarged view of the graffiti input area of FIG. 24 . 13 is an enlarged view of a stamp image constituting the role-playing stamp; FIG. 13A to 13C are diagrams illustrating an example of an operation when adjusting a stamp image. FIG. 13 is a diagram showing an example of an enlarged stamp image. FIG. 11 is another diagram showing an example of enlarging the stamp image. FIG. 11 is a diagram showing types of adjustment. FIG. 13 is a diagram showing an example of movement of a stamp image. 13A and 13B are diagrams illustrating an example of taking over adjustment of a stamp image. FIG. 13 is a diagram showing the difference in synthesis position.

<<Configuration of the photo sticker creation device>>
<External configuration>
1 and 2 are perspective views showing an example of the external configuration of a photo sticker creating device 1. FIG.

The photo sticker creation device 1 is a game machine that provides images taken of the user as a subject. The photo sticker creation device 1 provides images to the user by printing the captured image on sticker paper or by making the captured image viewable on the user's mobile device. The photo sticker creation device 1 is installed in amusement facilities such as game arcades and in stores.

The users of the photo sticker creation device 1 are mainly high school girls and young women. With the photo sticker creation device 1, multiple users, usually two or three people per group, can enjoy the game.

The user of the photo sticker creation device 1 takes a photo of themselves as the subject. The user selects an image from the images obtained by shooting and edits it to combine it with a composite image such as handwritten text or a stamp image. In this way, the captured image is edited into a colorful image. The user receives a sticker paper with an image of themselves printed on it, completing the series of games.

The photo sticker creation device 1 can be said to be a photo creation game machine that provides a photo creation game.

As shown in FIG. 1, the photo sticker creation device 1 is basically configured by installing a photographing unit 11 and an editing unit 12 in contact with each other. The photographing unit 11 and the editing unit 12 are electrically connected.

The photographing unit 11 has a box-shaped housing with a space large enough for a person to enter, and is composed of a pre-selection section 20, a photographing section 21, and a background section 22. The pre-selection section 20 is installed on the side of the photographing section 21. The space in front of the pre-selection section 20 becomes the pre-selection space where the pre-selection processing is carried out. In addition, the photographing section 21 and the background section 22 are installed a predetermined distance apart. The space formed between the photographing section 21 and the background section 22 becomes the photographing space where the photographing processing is carried out.

As part of the pre-selection process, the pre-selection unit 20 provides guidance to introduce the games provided by the photo sticker creation device 1 and performs various settings for photography in the photography space. The pre-selection unit 20 is provided with a coin slot through which the user inserts money, a touch panel monitor 71, and the like.

The touch panel monitor 71 is composed of a monitor such as an LCD (Liquid Crystal Display) and a touch panel layered on it. The touch panel monitor 71 has the function of displaying various GUIs (Graphical User Interfaces) and accepting selection operations by the user. The touch panel monitor 71 displays a screen used to select the number of users, etc.

The photographing unit 21 is a device for photographing the user as a subject. The photographing unit 21 is located in front of the user who has entered the photographing space. A camera and a touch panel monitor used for various operations are provided in front of the photographing unit 21 facing the photographing space.

When viewed from a user facing forward in the shooting space, the left side is the left side and the right side is the right side. The left side of the shooting unit 21 is made up of side panel 41A, and the right side is made up of side panel 41B. Furthermore, the front of the shooting unit 21 is made up of front panel 42. The above-mentioned pre-selection unit 20 is installed on side panel 41A.

The background section 22 is composed of a rear panel 51, a side panel 52A, and a side panel 52B. The rear panel 51 is a plate-shaped member that is located behind a user facing forward. The side panel 52A is attached to the left end of the rear panel 51. The side panel 52B is attached to the right end of the rear panel 51.

The opening between side panel 41A and side panel 52A serves as an entrance/exit to the photography space. The opening between side panel 41B and side panel 52B also serves as an entrance/exit to the photography space.

At least a portion of the ceiling surface formed above the shooting space is covered with a ceiling sheet 23. The ceiling sheet 23 may be in the form of a sheet or a plate.

The editing unit 12 is a device for editing captured images. As shown in FIG. 2, the editing unit 12 is connected to the photographing unit 11 with one side in contact with the front panel 42 of the photographing section 21.

If the configuration of the editing unit 12 shown in FIG. 1 is the configuration on the front side, components used for editing work are provided on both the front side and the back side of the editing unit 12. With this configuration, two groups of users can perform editing work at the same time.

The front side of the editing unit 12 is composed of a vertical surface 61 and an inclined surface 62 formed above the vertical surface 61. A tablet built-in monitor 141 and touch pens 142A and 142B are provided on the inclined surface 62 as components used for editing work. A lighting device 63 is provided above the inclined surface 62.

The tablet-built-in monitor 141 is constructed by stacking a tablet and a display. For example, an editing screen used for editing is displayed on the tablet-built-in monitor 141. When two users are editing simultaneously, the touch pen 142A is used by the user standing on the left side as viewed from the tablet-built-in monitor 141, and the touch pen 142B is used by the user standing on the right side as viewed from the tablet-built-in monitor 141. Hereinafter, when there is no need to distinguish between the touch pens 142A and 142B, they will be collectively referred to as the touch pens 142.

A curtain rail 24 is attached above the editing unit 12. The curtain rail 24 is made up of three rails 24A to 24C.

A curtain is attached to the curtain rail 24 so that the space in front of the front and the space in front of the rear of the editing unit 12 cannot be seen from the outside. The space in front of the front and the space behind the rear of the editing unit 12 enclosed by the curtain become the editing space where the user performs editing work.

A sticker paper outlet 161 is provided on the side of the editing unit 12. A printer is provided inside the editing unit 12. Sticker paper with an image of the user printed on it is discharged from the sticker paper outlet 161. The space in front of the side of the editing unit 12 (sticker paper outlet 161) becomes a print waiting space where the user waits for the printed sticker paper to be discharged.

<Composition of the front of the shooting space>
FIG. 3 is a perspective view showing an example of the configuration of the front of the imaging space.

At the front of the shooting space, the components of the shooting unit 21 are arranged so as to be surrounded by side panel 41A, side panel 41B, and front panel 42. Note that in FIG. 3, the pre-selection unit 20 installed on side panel 41A is omitted from the illustration.

A front camera unit 81 is provided slightly above the center of the front panel 42. The front camera unit 81 is provided with a front camera 91 and a front monitor 92.

The front camera 91 is, for example, a single-lens reflex camera, and is built into the front camera unit 81 so that the lens is exposed. The front camera 91 has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and captures images of users in the shooting space. The front camera 91 functions as a shooting unit that captures images of the user as a subject.

The moving image (hereinafter also referred to as a live view image) captured by the front camera 91 is displayed in real time on the front monitor 92. A still image captured by the front camera 91 at a specified timing, such as when a shooting command is issued, is saved as a captured image.

The front monitor 92 is provided below the front camera 91. The front monitor 92 is composed of a monitor such as an LCD and a touch panel stacked on it. The front monitor 92 has a function as a live view monitor that displays moving images captured by the front camera 91, and a function to display various GUIs and accept selection operations by the user.

A front strobe unit 82 is installed above the front camera unit 81, which irradiates the face and upper body of the user from above and in front of the user. The front strobe unit 82 contains a light bulb for illumination as well as a strobe light tube that emits light in accordance with the timing of shooting.

The front strobe unit 82 has three panels 82a, 82b, and 82c arranged in a horizontal row as light-emitting surfaces, each panel being made of a transparent or semi-transparent acrylic plate or the like. The light-emitting surfaces (panels 82a, 82b, and 82c) of the front strobe unit 82 are installed facing diagonally downward.

Below the front camera unit 81 is a lower box strobe unit 83 that shines light onto the lower body and feet of the user. The lower box strobe unit 83 has a built-in light bulb for illumination as well as a strobe light tube that emits light in accordance with the timing of shooting.

The light-emitting surface of the lower box strobe unit 83 is formed so as to completely cover the front between the side panels 41A and 41B. The light-emitting surface of the lower box strobe unit 83 is also formed from a transparent or semi-transparent acrylic plate or the like. The top surface of the lower box strobe unit 83 is used as a luggage storage area for users performing photography work to place their baggage, etc.

<Ceiling surface configuration of the shooting space>
FIG. 4 is a perspective view showing an example of the configuration of the ceiling surface of the imaging space.

A ceiling camera unit 101 and a ceiling strobe unit 102 are provided at approximately the center of the ceiling surface of the shooting space so as to protrude from the ceiling surface.

The ceiling camera unit 101 is provided with a ceiling camera 111 and a ceiling monitor 112.

Like the front camera 91, the ceiling camera 111 is, for example, a single-lens reflex camera and is built into the ceiling camera unit 101 with its lens exposed. The ceiling camera 111 captures images of users in the shooting space.

The live view image captured by the ceiling camera 111 is displayed in real time on the ceiling monitor 112. A still image captured by the ceiling camera 111 at a specified timing, such as when a shooting command is issued, is saved as a captured image.

The ceiling monitor 112 is provided below the ceiling camera 111. The ceiling monitor 112 is composed of a monitor such as an LCD. The ceiling monitor 112 has a function as a live view monitor that displays moving images captured by the ceiling camera 111.

The surface of the ceiling camera unit 101 on which the ceiling camera 111 and ceiling monitor 112 are mounted is formed to face diagonally downward.

A ceiling strobe unit 102 is installed adjacent to the ceiling camera unit 101 on the ceiling surface of the shooting space, in front of the ceiling camera unit 101. The ceiling strobe unit 102 irradiates the user with strobe light from above and diagonally in front of the user's head. The ceiling strobe unit 102 has a built-in light bulb for lighting in addition to a strobe light tube that emits light in accordance with the shooting timing.

The ceiling strobe unit 102 has a panel 102a formed of a transparent or semi-transparent acrylic plate or the like as a light-emitting surface. In addition, speakers are provided on the left and right sides of the light-emitting surface (panel 102a) of the ceiling strobe unit 102 for outputting voice guidance for the shooting process, background music, sound effects, etc.

The light-emitting surface (panel 102a) of the ceiling strobe unit 102 is also formed to face diagonally downward.

In this way, the ceiling camera unit 101 and the ceiling strobe unit 102 are arranged on the ceiling surface of the shooting space so as to have the same positional relationship as the front camera unit 81 and the front strobe unit 82 at the front of the shooting space.

Figure 5 shows an example of a captured image.

Photographing in the photographing space is performed using both the front camera 91 and the ceiling camera 111. The image captured using the front camera 91 shows a large image of the face and upper body of the user facing forward, as shown on the left side of Figure 5.

On the other hand, the image captured by the ceiling camera 111 is an image that captures the entire body of a user sitting on a bench provided in the shooting space, as shown on the right side of Figure 5. In the shooting space, a bench is provided close to the rear panel 51, on which the user can sit and face the ceiling camera 111 to take the image.

Hereinafter, the image captured using the front camera 91 will be referred to as a close-up image, where appropriate. Also, the image captured using the ceiling camera 111 will be referred to as a ceiling camera image. In this way, by providing the ceiling camera 111, it becomes possible to capture multiple types of images with different angles of view.

<Block configuration of the photo sticker creation device>
Fig. 6 is a block diagram showing an example of the configuration of the photo sticker creating device 1. In Fig. 6, the same components as those described above are given the same reference numerals, and duplicated explanations will be omitted as appropriate.

As shown in FIG. 6, the photo sticker creation device 1 is configured by connecting a pre-selection unit 302, a photographing unit 304, editing units 305A and 305B, and a printing unit 306 to a PC (Personal Computer) unit 301 via an external input/output interface 303. The PC unit 301, which controls the overall operation of the photo sticker creation device 1, is housed, for example, inside the editing unit 12.

The CPU (Central Processing Unit) 311, ROM (Read Only Memory) 312, and RAM (Random Access Memory) 313 that make up the PC section 301 are interconnected by a bus 314.

The CPU 311 executes a predetermined program and controls the overall operation of the photo sticker creation device 1. The ROM 312 stores the programs and data executed by the CPU 311. The RAM 313 temporarily stores the data and programs processed by the CPU 311.

An input/output interface 315 is further connected to the bus 314. A memory unit 316, a communication unit 317, and a drive 318 are connected to the input/output interface 315.

The storage unit 316 is a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 316 stores various setting information and the like supplied from the CPU 311. The information stored in the storage unit 316 is read out by the CPU 311 as appropriate.

The communication unit 317 is an interface for a network such as the Internet. The communication unit 317 communicates with external devices under the control of the CPU 311. The communication unit 317 transmits an image selected by a user to, for example, an image management server managed by the manufacturer of the photo sticker creation device 1. The image transmitted from the communication unit 317 is allocated a specified memory area in the image management server and stored there, and is displayed on or downloaded to a mobile terminal that has accessed the image management server.

Removable media 319, such as an optical disk or semiconductor memory, is appropriately attached to drive 318. Programs and data read from removable media 319 by drive 318 are supplied to CPU 311, where they are stored in storage unit 316 or installed.

The external input/output interface 303 is connected to the input/output interface 315. The PC unit 301 controls each unit via the external input/output interface 303.

The pre-selection unit 302 performs pre-selection processing for users in the pre-selection space. The pre-selection unit 302 is composed of a touch panel monitor 71, a speaker 321, and a coin processing unit 322.

The touch panel monitor 71 displays various selection screens under the control of the PC unit 301 and accepts operations by the user on the selection screens. An input signal representing the content of the user's operation is supplied to the PC unit 301, and various settings are made.

The coin processing unit 322 detects the insertion of coins into the coin slot. When the coin processing unit 322 detects that a predetermined amount of coins has been inserted, it outputs a start signal to the PC unit 301 to instruct the start of the game.

The image capturing unit 304 performs image capturing processing on the user in the image capturing space. The image capturing unit 304 is composed of a front camera 91, a front monitor 92, a ceiling camera 111, a ceiling monitor 112, a lighting device 331, and a speaker 332.

The front camera 91 and ceiling camera 111 take images according to the control of the PC unit 301, and output the still image or video image data obtained by the image capture to the PC unit 301.

The front monitor 92, under the control of the PC unit 301, displays in real time a live view image captured by the front camera 91, accepts user operations on various screens, and outputs signals indicating the contents of the accepted operations to the PC unit 301.

The ceiling monitor 112 displays the live view image captured by the ceiling camera 111 in real time under the control of the PC unit 301.

The lighting devices 331 are strobe units within the shooting space, and emit light according to control signals supplied from the PC section 301. The lighting devices 331 are composed of a front strobe unit 82, a lower box strobe unit 83, and a ceiling strobe unit 102.

Speaker 332 is a speaker installed in the ceiling strobe unit 102 within the shooting space.

Editing unit 305A performs editing processing for a user in one editing space. Editing unit 305A is composed of tablet built-in monitor 141, touch pens 142A and 142B, and speaker 341. Editing unit 305B performs editing processing for a user in the other editing space, and has the same configuration as editing unit 305A. Hereinafter, editing units 305A and 305B will be collectively referred to as editing unit 305 as appropriate.

The tablet built-in monitor 141 displays an editing screen according to the control of the PC unit 301 and accepts operations on the editing screen by the user. An input signal representing the content of the user's operation is supplied to the PC unit 301, and the captured image to be edited is edited.

The printing section 306 is configured to include a printer 351. A sticker paper unit 352 is attached to the printer 351.

The printer 351 drives the built-in head 361 and prints an image on a roll of sticker paper 363 stored in the sticker paper unit 352 based on the print data supplied from the PC section 301. The printer 351 also cuts the sticker paper 363 with the image printed thereon at a predetermined cutting position using a cutter 362, and discharges it to the sticker paper outlet 161.

<Example of functional configuration of control unit>
Fig. 7 is a block diagram showing an example of a functional configuration of the photo sticker creating device 1. At least a part of the functional units shown in Fig. 7 is realized by the CPU 311 executing a predetermined program.

The control unit 401 is realized in the photo sticker creation device 1. The control unit 401 is composed of a pre-selection processing unit 411, a photography processing unit 412, an editing processing unit 413, a printing processing unit 414, and a transmission processing unit 415.

The pre-selection processing unit 411 executes the pre-selection process by controlling each part of the pre-selection unit 302. The pre-selection process selects the number of users, the game theme, and the like. A number of themes are available for the game of the photo sticker creation device 1.

During the game, effects according to the game theme are performed, for example, by the screen display and background music. In addition, the editing tools that can be used during editing change according to the game theme. The game theme determines the progress of the game. Information indicating the selection made by the user is supplied to the shooting processing unit 412.

The shooting processing unit 412 performs shooting processing by controlling each unit of the shooting unit 304. In the shooting processing, multiple shot images, for example, seven shot images, are taken. The shooting processing unit 412 outputs data of the multiple shot images obtained by performing the shooting processing to the editing processing unit 413.

The editing processing unit 413 performs editing processing by controlling each part of the editing unit 305. The editing processing includes editing the captured image, printing the image on sticker paper, and sending the image to an external device, an image management server.

As described above, the images sent to the image management server can be viewed or downloaded by the user on a mobile device such as a smartphone after the user has finished playing the game on the photo sticker creation device 1.

Editing the captured image includes a process of generating an image to be printed on sticker paper based on the captured image, and a process of generating an image to be sent to an image management server based on the captured image.

As will be described in more detail later, the image to be printed on the sticker paper is generated by having the user select a sticker design and then performing image processing according to the sticker design selected by the user.

In addition, the mobile-only image, which is the image to be sent to the image management server, is generated by applying image processing to the captured image to be edited in accordance with the operation performed using a tool selected by the user from among pre-prepared editing tools, such as a pen tool or stamp tool. By using the pen tool or stamp tool, a pen image or stamp image as an image to be combined is combined with the captured image to be edited. The user selects the editing tool of their choice and selects the position on the captured image to be edited where the image processing is to be applied, and proceeds with editing the captured image.

In this way, the user edits the captured image to create a mobile phone-only image by using the touch pen 142 as if doodling on the captured image. Hereinafter, editing will be referred to as doodling where appropriate.

The editing processing unit 413 outputs print data generated based on the image to be printed on the sticker paper to the print processing unit 414, and outputs the data of the mobile-only image to the transmission processing unit 415.

The print processing unit 414 outputs the print data to the printer 351 and prints the image on the sticker paper.

The transmission processing unit 415 controls the communication unit 317 to communicate with the image management server and transmit (upload) the mobile-only images to the image management server.

Figure 8 is a block diagram showing an example configuration of the editing processing unit 413.

The editing processing unit 413 is composed of an image analysis unit 421, a sticker image generation unit 422, a sticker design data storage unit 423, a mobile-only image generation unit 424, and a display control unit 425. Details of the functions of each processing unit will be described later with reference to a flowchart. The captured image data output from the capture processing unit 412 is input to the image analysis unit 421 and the display control unit 425.

The image analysis unit 421 analyzes the features of the user that appear in each captured image, and recognizes the positions of specific body parts of the user that appear in the captured image. For example, the positions of the head, eyes, nose, and mouth are recognized as the positions of specific body parts. Information indicating the positions recognized by the image analysis unit 421 is supplied to the sticker image generation unit 422 and the mobile-only image generation unit 424 together with the captured image data.

The sticker image generation unit 422 reads information about the sticker design selected by the user from the sticker design data storage unit 423, and applies image processing to the captured image based on the read information.

The sticker image generating unit 422 also combines the sticker image obtained by applying image processing to the captured image based on information about the sticker design with a sticker edge image corresponding to the sticker design, to generate a layout image that represents the print image of the sticker paper. The layout image is an image that combines a sticker image in which the user appears as the subject with a sticker edge image of a specified design. The sticker image generating unit 422 accepts operations performed by each user and generates a layout image for each user.

The layout image data generated by the sticker image generation unit 422 is supplied to the print processing unit 414 as print data. Information representing the content of the image processing by the sticker image generation unit 422 is appropriately supplied to the display control unit 425 and used for screen display.

The image processing by the sticker image generation unit 422 can be said to be processing for creating an image to be printed on sticker paper. The layout image generated by the image processing by the sticker image generation unit 422 is an image for printing that is used for printing on sticker paper.

The mobile phone-only image generation unit 424 applies image processing to each captured image in accordance with the mobile phone-only graffiti made by the user, and generates a mobile phone-only image. The mobile phone-only graffiti is graffiti used to create an image to be sent to the image management server. The image with the mobile phone-only graffiti is an image for transmission, which is used to send to the image management server.

Note that "for printing" indicates that the image is intended for printing, which is one of the two possible uses, printing and sending. "for sending" indicates that the image is intended for sending, which is one of the two possible uses, printing and sending. Images for printing are not used to send to an image server, and images for sending are not used to print on sticker paper. In other words, images for printing and images for sending may be used for purposes other than printing and sending, such as for display on the screen of the tablet's built-in monitor 141.

The mobile-only image generated by the mobile-only image generation unit 424 is supplied to the transmission processing unit 415. Information representing the content of the image processing by the mobile-only image generation unit 424 is appropriately supplied to the display control unit 425 and used for screen display.

The display control unit 425 displays various screens, including a screen used to generate images for stickers and a mobile-only doodling screen, on the tablet built-in monitor 141. The display control unit 425 controls the display of the screen in response to the user's operation.

<<Operation of the photo sticker creation device>>
<Photo sticker creation game>
Here, the operation of the photograph sticker creating device 1 that provides the photograph sticker creating game will be described with reference to the flow chart of FIG.

In step S1, the pre-selection processing unit 411 determines whether or not a predetermined amount of coins has been inserted, and waits until it determines that coins have been inserted. The fee to be paid at the start of one game is, for example, 400 yen.

If it is determined in step S1 that a predetermined amount of coins has been inserted, then in step S2, the pre-selection processing unit 411 performs a pre-selection process. In the pre-selection process, the number of users and the game theme are selected.

In step S3, the image capture processing unit 412 controls the image capture unit 304 to perform image capture processing. Specifically, the image capture processing unit 412 displays the moving image captured by the front camera 91 on the front monitor 92 in a live view. When the image capture timing arrives, the image capture processing unit 412 captures the user in the image capture space as a subject and generates a captured image that is a close-up image.

The image capturing processing unit 412 also displays the moving image captured by the ceiling camera 111 on the ceiling monitor 112 in a live view. When the time comes to capture an image, the image capturing processing unit 412 captures the user in the capture space as a subject, and generates a captured image that becomes a ceiling camera image.

When multiple images have been captured, the capture process ends. For example, six close-up images and one ceiling camera image are captured.

In step S4, the editing processing unit 413 performs editing processing by controlling the editing unit 305 corresponding to the editing space to which the user has moved after completing the shooting process.

<Editing process details>
Next, the editing process performed in step S4 in FIG. 9 will be described with reference to the flowchart in FIG.

In step S11, the display control unit 425 of the editing processing unit 413 displays the address input screen on the tablet built-in monitor 141.

The address input screen is a screen used to input the user's email address. The address input screen displays a software keyboard and is used to input the email address.

The email address entered using the address input screen is uploaded to the image management server together with the mobile-only image, and is used to notify the user of the URL where the mobile-only image is saved by email from the image management server. For example, the user can display the mobile-only image created in the game on the photo sticker creation device 1 on the screen of the mobile device by tapping the URL notified by email.

In step S12, a sticker design selection process is performed. In the sticker design selection process, image processing according to the sticker design selected by the user is performed on the captured image to generate an image for a sticker. In addition, a layout image, which is an image for printing, is generated using the image for the sticker. Details of the sticker design selection process will be described later with reference to the flowchart in FIG. 11.

In step S13, the print processing unit 414 starts printing the layout image based on the print data representing the layout image. The timing for starting printing the layout image can be any timing after the sticker design selection process has been performed.

In step S14, the display control unit 425 displays the mobile phone-specific graffiti screen on the tablet built-in monitor 141. The user uses the mobile phone-specific graffiti screen to doodle on the captured image. The mobile phone-specific graffiti is performed on all seven captured images obtained by the capture process as graffiti targets. Details of doodling using the mobile phone-specific graffiti screen will be described later.

In step S15, the mobile phone-only image generation unit 424 performs image processing on the captured image of the graffiti target in response to the user's operation to generate a mobile phone-only image.

In step S16, the mobile phone-only image generation unit 424 determines whether the mobile phone-only graffiti has ended. If it is determined in step S16 that the mobile phone-only graffiti has not ended, the process returns to step S14, and the mobile phone-only graffiti continues. For example, if a preset time limit has elapsed, the mobile phone-only graffiti ends.

If it is determined in step S16 that the mobile phone-only graffiti has ended, in step S17, the display control unit 425 causes an image selection screen to be displayed on the tablet built-in monitor 141. The image selection screen displayed here is a screen used to select one mobile phone-only image from the seven mobile phone-only images created by the mobile phone-only graffiti.

Of the seven mobile-only images uploaded to the image management server, one mobile-only image selected using the image selection screen is provided to the user who is a free member. The image provision service provided by the image management server is a membership-based service that allows users to view images on their mobile terminals and download images to their mobile terminals.

A free member can view one mobile-only image. On the other hand, a paid member can view all mobile-only images.

When the selection of the mobile-only image is completed, in step S18, the transmission processing unit 415 transmits the mobile-only image to the image management server. When the transmission of the mobile-only image is completed, the process returns to FIG. 9, and the series of processes for the photo sticker creation game is completed.

<Selecting the sticker design>
The sticker design selection process performed in step S12 of FIG. 10 will be described with reference to the flowchart of FIG.

In step S31, the image analysis unit 421 analyzes the features of the user shown in each captured image and recognizes the position of each part of the user's body.

In step S32, the sticker image generation unit 422 generates a sample image that serves as a model of a sticker design according to the game theme. The game theme is selected in the pre-selection process.

In step S33, the display control unit 425 displays the sticker design selection screen using the sample image generated by the sticker image generation unit 422.

In step S34, the sticker image generating unit 422 determines whether or not a specific sticker design has been selected from among a plurality of sticker designs. If it is determined in step S34 that a sticker design has not been selected, the process returns to step S33, and processing continues according to the user's operation on the sticker design selection screen.

Figure 12 shows an example of a sticker design selection screen.

Area 501L is formed on the left side of the sticker design selection screen, and area 501R is formed on the right side. Area 501L is the area used for operation by a user standing on the left side facing the tablet built-in monitor 141. Area 501R is the area used for operation by a user standing on the right side facing the tablet built-in monitor 141.

In this way, the selection of the sticker design is performed by each user using the display in areas 501L and 501R. The operations by each user on areas 501L and 501R are accepted, for example, by the sticker image generation unit 422. The display in area 501L will be mainly described, but the display in area 501R will be similar to the display in area 501L.

A message is displayed in the upper center of the sticker design selection screen, encouraging the user to select the number of divisions, which is the layout of the image. Below the message are the "Pocket Print" and "Other Layouts" tabs, with the "Pocket Print" tab being selected in the example of Figure 12. Sticker design selection is performed when the "Pocket Print" tab is selected. When the "Other Layouts" tab is selected, sample layouts of various patterns are displayed.

A sample image 511L is displayed in the center of area 501L. The sample image is an example of the sticker design.

To the left of the sample image 511L, a color selection button 512L is displayed, which can be used to select a color.

Sticker design selection buttons 521-1 to 521-7 are displayed in button area 513L formed below sample image 511L. Sticker design selection buttons 521-1 to 521-7 are buttons used to select a sticker design. When a sticker design selection button is pressed, the display of sample image 511L changes. In the example of FIG. 12, sticker design selection button 521-1 is selected.

The user can select a sticker design by pressing the confirm button while a specific sample image is displayed.

Figure 13 shows an enlarged sample image.

Sample image 511-1 shown in FIG. 13 is an image that is displayed in area 501L as sample image 511L when, for example, sticker design selection button 521-3 in FIG. 12 is pressed.

Sample image 511-1 is constructed by arranging sticker image P1 and sticker image P2 vertically and combining them with sticker edge image P11. Both sticker image P1 and sticker image P2 are close-up images. Sticker designs in which the sticker images are arranged in different positions may be prepared, such as sticker image P1 and sticker image P2 arranged horizontally.

As described above, the sticker image is an image generated based on a captured image of the user as the subject, to be printed on sticker paper.

The sticker edge image is an image that represents the design of the edge portion of the sticker paper (the area surrounding the image in which the user is the subject). In the example of FIG. 13, an image of a specific color, including a logo representing the model name of the photo sticker creation device 1, is used as the sticker edge image. For example, the color of the sticker edge image changes each time the color is changed using the color selection button 512L.

In this way, an image using an image actually obtained by photography is displayed as a sample image that serves as an example of the sticker design. The user can select a sticker design while imagining the layout image that will actually be printed. A sample image using an image of a person other than the user, such as a model person, may also be displayed.

Sample image 511-1 is generated, for example, by combining a seal edge image P11, a pattern image P12, and a sticky stamp image P13, as shown in Figures 14A to 14C.

The pattern image is an image that is composited with the background of the user appearing in the captured image. Pattern image P12 shown in FIG. 14B is an image of a floral pattern.

The sticky stamp image is a stamp image (image for compositing) that is composited at a compositing position determined based on the position of a specific part of the user that appears in the captured image. For example, a position on the user is determined as the compositing position so that the image is composited over the user. Sticky stamp image P13 shown in Figure 14C is composed of six flower images arranged with their positions shifted.

Multiple templates representing the components of each sticker design are provided for each game theme.

Figure 15 shows an example of a template.

In the example of FIG. 15, six templates, templates 1 to 6, are linked to the game theme "girly." As shown in the speech bubble, each template is composed of a combination of at least one of a sticker edge image, a pattern image (background image), a sticky stamp image, a frame image, and a filter image. Sticker designs that do not use the above-mentioned pattern images or sticky stamp images may also be provided.

A frame image is an image that is composited into the foreground of a captured image. Unlike a sticky stamp image, the image of a specific design that makes up a frame image is composited into a fixed position on the captured image.

A filter image is an image that represents the content of the filter processing to be applied to a captured image. By applying filter processing using a filter image to a captured image, the brightness, color, etc. of the captured image are changed partially or entirely.

The information on templates representing sticker designs corresponding to each of these game themes is stored in the sticker design data storage unit 423 of FIG. 8. For example, when selecting a sticker design, content that will be the components of the sticker design, such as a sticker edge image, a pattern image, and a sticky stamp image, is identified based on the template information linked to the game theme selected by the user, and multiple sample images are generated by the sticker image generation unit 422. In addition, the images generated by the sticker image generation unit 422 are used to display the sticker design selection screen.

Sticker designs using other types of content, such as stamp images that are composited across the photographed image and parts other than the photographed image, may also be prepared. In addition, images for makeup, such as images of eyeshadow, eyeliner, and eyelashes that are composited at the position of the eyes on the face, a blush image that is composited at the position of the cheeks, and a lip image that is composited at the position of the lips, may be used as the sticky stamp images.

Figure 16 shows an example of generating an image for a sticker.

As indicated by arrow #1, a pattern image P12 is composited with the background of photographed image P21 to generate a sticker image P1 as indicated by the tip of arrow #2.

The pattern image P12 is composited by compositing it onto the background of the person area (the layer below the layer of the person area) extracted from the photographed image P21 by chromakey processing.

As shown by arrow #3, a pattern image P12 is composited into the background of the captured image P22, and as shown by arrow #4, a stamp image P13 is composited at the head positions of each of the two users appearing in the captured image P22, generating a sticker image P2 as shown by arrow #5.

The sticky stamp image P13 is synthesized by synthesizing it at positions p1 and p2, which are synthesis positions determined based on the head positions of the two users recognized by analyzing the captured image P22. The sticky stamp image P13 is synthesized on a layer above the layer of the person area. The sticky stamp image P13 is rotated according to the direction of the user's head and scaled according to the size of the user's head, as appropriate.

The sticker image P1 and sticker image P2 generated in this way are combined with the sticker edge image P11 to generate sample image 511-1.

In this way, a sticker design is prepared that uses content to be composited at a composite position determined based on the position of a specific part of the user that appears in the captured image, and content to be used to edit parts other than the image for the sticker. In the above example, the content to be composited at a composite position determined based on the position of a specific part of the user that appears in the captured image is the sticky stamp image P13, and the content to be used to edit parts other than the image for the sticker is the sticker edge image P11.

The information of the template representing the sticker design includes information indicating which of the seven captured images is to be used to generate the sample image. The captured images P21 and P22 used to generate sample image 511-1 are, for example, the first captured image obtained by the first capture and the second captured image obtained by the second capture, respectively. The default captured images of the sticker design of sample image 511-1 are two close-up images, the first captured image and the second captured image.

Some of the sticker designs will not only feature close-up images, but will also feature a design that uses a seventh image taken with a different angle of view, a ceiling camera image, as the default image.

Figure 17 shows an enlarged view of another sample image.

Sample image 511-2 shown in FIG. 17 is the image that is displayed in area 501L as sample image 511L when a specific sticker design selection button is pressed.

Sample image 511-2 is constructed by arranging sticker image P51 and sticker image P52 vertically and combining them with sticker edge image P71. Sticker image P51 is a close-up image, and sticker image P52 is an image taken with a ceiling camera.

In this way, sticker designs will also be available that combine ceiling camera images with close-up images.

Sample image 511-2 is generated, for example, by combining seal edge image P71, filter image P72, and sticky stamp images P73 and P74, as shown in Figures 18A to 18C.

The filter image P72 shown in FIG. 18B is an image used to generate an image that gives the impression that the entire image is colored in a specific color. Filter processing using the filter image P72 changes the pixel values of the pixels in the entire captured image.

The sticky stamp image P73 shown in FIG. 18C is an image representing an angel halo. The sticky stamp image P74 is an image representing angel wings. The sticky stamp image of an angel halo and the sticky stamp image of angel wings are used as a set.

Figure 19 shows an example of generating an image for a sticker.

As shown by arrow #11, a filter process using the filter image P72 is applied to the captured image P81 to generate a sticker image P51 as shown by the tip of arrow #12.

Furthermore, as shown by arrow #13, a filter process using filter image P72 is applied to captured image P82, and as shown by arrows #14 and #15, a sticky stamp image P73 is composited onto the heads of each of the two users appearing in captured image P82, and a sticky stamp image P74 is composited near the shoulders of each of the two users, thereby generating a sticker image P52 as shown by arrow #16.

The sticky stamp image P73 is synthesized at positions p11 and p12, which are synthesis positions determined based on the head positions of the two users recognized by analyzing the captured image P82. The sticky stamp image P73 is synthesized on a layer above the layer of the person area. The sticky stamp image P73 is rotated according to the direction of the user's head and enlarged or reduced according to the size of the user's head, as appropriate.

The sticky stamp image P74 is synthesized at positions p21 and p22, which are synthesis positions determined based on the head positions of the two users recognized by analyzing the captured image P82. The sticky stamp image P74 is synthesized on the same layer as the sticky stamp image P73. The sticky stamp image P74 is rotated according to the orientation of the user's head, enlarged or reduced according to the size of the user's head, etc., as appropriate.

The sticker image P51 and sticker image P52 generated in this way are composited with the sticker edge image P71 to generate sample image 511-2.

The sticker design represented by sample image 511-2 is also a design that uses content that is composited at a composite position determined based on the position of a specific part of the user's body that appears in the captured image, and content that is used to edit parts other than the sticker image.

Using the sticker design selection screen that displays sample images like those above, multiple users can each select a sticker design.

For example, when color selection button 512L is pressed, at least one of the colors of the seal edge image, the background image, and the filter image is changed according to the selected color. Each user selects a seal design while looking at the content that constitutes the seal design, as well as the color of the content.

It may also be possible to turn filter processing using a filter image on/off, adjust color density, etc., when selecting a sticker design.

If it is determined in step S34 of FIG. 11 that the specified sticker design has been selected by two users, processing proceeds to step S35.

In step S35, the display control unit 425 causes the tablet built-in monitor 141 to display an image selection screen instead of the sticker design selection screen. The image selection screen is a screen used to select a captured image to be used as an image for a sticker.

The sample image displayed on the sticker design selection screen is an image that uses a default photographed image that is automatically selected by the photo sticker creation device 1. After selecting a sticker design, the user can change (replace) the default photographed image with another photographed image.

Figure 20 shows an example of an image selection screen.

In areas 501L and 501R, images 551L and 551R showing the seal edge image of the selected seal design are displayed, respectively. The captured image is selected by each user using the display in areas 501L and 501R.

In the example of Figure 20, the sticker design selected by the user on the left is a sticker design using two close-up images, as described with reference to Figure 13, etc. The sticker design selected by the user on the right is a sticker design using a close-up image and a ceiling camera image, as described with reference to Figure 17, etc.

Image arrangement areas 561L and 562L are formed vertically in image 551L in area 501L. The user on the left side uses touch pen 142A to select two captured images, for example by moving thumbnail images displayed in display area 552L to the right of image 551L to image arrangement areas 561L and 562L.

Thumbnail images representing six captured images P101 to P106 are displayed in display area 552L. Captured images P101 to P106 are the first to sixth captured images, respectively. The thumbnail images displayed in display area 552L are thumbnail images of close-up images. Because the sticker design selected by the user on the left is a sticker design that uses two close-up images, the thumbnail image of the ceiling camera image is not displayed, and only the thumbnail image of the close-up image is displayed.

The user on the left will be asked to select two close-up images to be used for the sticker design they have selected.

On the other hand, image arrangement areas 561R and 562R are formed vertically aligned in image 551R of area 501R. A ceiling camera image has already been arranged in image arrangement area 562R, and image arrangement area 562R is displayed in its entirety in a grayed-out state. It is not possible to replace the captured image to be arranged in image arrangement area 562R.

In this example, the sticker design selected by the user on the right is a sticker design that uses a close-up image and a ceiling camera image, and since the ceiling camera image has only been captured once, only the close-up image to be placed in the image placement area 561R can be selected. Whether or not to accept the selection of the captured image is switched by the sticker image generation unit 422, etc., depending on the angle of view of the default captured image of the sticker design.

The user on the right side uses the touch pen 142B to select one captured image, for example by moving the thumbnail image displayed in the display area 552R to the right of the image 551R to the image arrangement area 561R. The same thumbnail image as the thumbnail image displayed in the display area 552L is displayed in the display area 552R.

Using this display, two users can each use their preferred photographed image for the sticker design they selected, regardless of the photographed image that was placed in the sample image on the sticker design selection screen. Since the photographed image to be used for the sticker design can be selected after the sticker design is selected, the users can replace the default photographed image with a photographed image that matches the sticker design.

For example, when the user on the left side selects a captured image to be placed in the image placement areas 561L and 562L, the sticker image generation unit 422 performs image processing using a template representing the sticker design described with reference to FIG. 13, etc.

The captured image placed at the top of the sticker design is subjected to image processing using the pattern image P12 to generate an image for the sticker.

In addition, the captured image placed in the lower row of the sticker design is subjected to image processing using the pattern image P12 and the sticky stamp image P13. For example, if the user on the left places the captured image P104 in the lower image placement area 562L, as shown in FIG. 21, image processing is performed on the captured image P104 using the pattern image P12 and the sticky stamp image P13. In the example of FIG. 21, the pattern image P12 is composited into the background of the two users, and the sticky stamp image P13 is composited at a composite position determined based on the position of each of the two users' heads.

Similarly, when the user on the right side selects a captured image to be placed in the image placement area 561R, the sticker image generation unit 422 performs image processing using a template representing the sticker design as described with reference to FIG. 17, etc.

The process returns to step S35 and continues to select captured images until it is determined in step S36 of FIG. 11 that the selection of captured images using the image selection screen described above has ended. If it is determined in step S36 that a captured image has been selected, the process proceeds to step S37.

In step S37, the sticker image generating unit 422 applies the sticker design selected by the user to the captured image selected by the user to generate a sticker image. The sticker image generating unit 422 also generates a layout image by combining the sticker image with a sticker edge image that constitutes the sticker design selected by the user, and outputs the layout image data as print data.

Then, the process returns to step S12 in FIG. 10, and subsequent processes are carried out. When printing of the layout image is completed, the printed sticker paper is removed from the sticker paper outlet 161.

Figure 22 shows an example of a printed sticker sheet.

For example, when the sticker designs described with reference to Figures 13 and 17 are selected, sticker sheets s1 and s2 on which layout images of the respective sticker designs are printed are provided to the user, as shown in Figure 22.

In this way, in the game using the photo sticker creation device 1, users do not doodle on the images that are provided printed on sticker paper. Users can decide what to print on the sticker paper simply by selecting a sticker design and a photographed image.

Modifications In the above embodiment, the selection of the photographed image is performed after the selection of the sticker design, but the selection of the sticker design may be performed after the selection of the photographed image. A single screen may be used to select both the sticker design and the photographed image.

It may be possible to select either the sticker design or the captured image first, and then select the other, after which the user can return to the previous selection.

You may also be asked to select the game theme again when selecting a sticker design.

It may be possible to allow each user to select a game theme. For example, by allowing two users to each select a different game theme, it is possible to improve user satisfaction.

<About mobile phone-only graffiti>
Screen Configuration of the Mobile Phone-Only Graffiti Screen FIG. 23 is a diagram showing a display example of the mobile phone-only graffiti screen.

As shown in FIG. 23, the mobile-only graffiti screen is basically configured with each display being symmetrically arranged on either side of the central thumbnail display area 603. The area on the left is used by users on the left side. The area on the right is used by users on the right side.

Note that the dashed rectangular lines surrounding each area on the mobile-only graffiti screen shown in Figure 23 are not actually displayed on the screen.

The thumbnail display area 603 located in the center of the mobile-only graffiti screen is a display area for thumbnail images that represent captured images. The user uses the thumbnail images displayed in the thumbnail display area 603 to select the captured image on which to doodle.

In the thumbnail display area 603, six close-up images and one thumbnail image showing a ceiling camera image are displayed side by side. Mobile-only graffiti is performed on all captured images.

In the example of FIG. 23, the thumbnail image at the top left is selected by the user on the left, and the thumbnail image at the top right is selected by the user on the right. The thumbnail image at the top left represents the first captured image P101, and the thumbnail image at the top right represents the fourth captured image P104.

A doodle input area 601L is provided approximately in the center of the left-hand area. The doodle input area 601L is a display area for the captured image that is selected using the thumbnail image and is to be doodled on. The user can select an editing tool using the touch pen 142A and edit the captured image P101 displayed in the doodle input area 601L.

To the right of the doodle input area 601L is a tool area 602L containing an array of buttons, such as a button to be pressed to select the eraser tool, a button to be pressed to reset the doodle, and a button to be pressed to return to the previous operation.

A palette area 604L is provided below the graffiti input area 601L. The palette area 604L is an area that displays buttons and the like used to select various editing tools to be combined with the captured image displayed in the graffiti input area 601L.

Above the palette area 604L, tabs are provided for selecting categories of editing tools. The user can use the tabs to switch between editing tools displayed in the palette area 604L. In the example of FIG. 23, the tabs for Pretend Stamp, MAKE, STAMP, and PEN are provided, and the Pretend Stamp tab is selected. The MAKE, STAMP, and PEN tabs are respectively for a makeup tool, which is an editing tool used for applying makeup to the face, a stamp tool used to synthesize stamp images, and a pen tool used to synthesize pen images.

In the palette area 604L, selection buttons 611-1 to 611-5 are displayed, which are buttons used to select the type of impersonation stamp. Selection buttons 611-1 to 611-5 are used to select impersonation stamps with the names "Rabbit", "Moon", "Cherry", "Cat", and "Wirelight", respectively.

In addition, in the right-hand area used by the right-hand user, the same configuration as the left-hand area described above is arranged in a symmetrical position.

About the role-playing stamp The role-playing stamp is an editing tool used to combine a stamp image according to the game theme with a photographed image of the graffiti target. The type of role-playing stamp that can be used for graffiti changes depending on the game theme.

A single input using the roleplay stamp will combine multiple stamp images with designs that correspond to the game theme. A roleplay stamp is made up of multiple stamp images.

Each stamp image, which is the content that makes up the role-playing stamp, is composited at a composite position that is determined based on the position of a specific part of the user's body that is recognized by analyzing the captured image, just like the sticky stamp image described above.

For example, when the user on the left presses the selection button 611-3, a "Cherry" role-playing stamp is superimposed on the photographed image P101 that is the graffiti target, as shown in FIG. 24. In the example of FIG. 24, multiple stamp images are superimposed on the faces of users U1 and U2 that appear in the photographed image P101.

Figure 25 is an enlarged view of the graffiti input area 601L in Figure 24.

A bubble-shaped stamp image P201 containing the word "cherry" is synthesized at the position of the head of each of users U1 and U2 appearing in the captured image P101. The stamp image P201 is shown enlarged in FIG. 26A. The stamp image P201 is a stamp image synthesized at a synthesis position determined, for example, based on the position of the head of the user appearing in the captured image.

In addition, a stamp image P202 consisting of a set of two cherry images is synthesized under the eyes of each of users U1 and U2 who appear in the captured image P101. Stamp image P202 is shown enlarged in FIG. 26B. Stamp image P202 is a stamp image synthesized at a synthesis position determined, for example, based on the center of the face of the user who appears in the captured image. For example, the position of the nose is used as the center of the face.

In this way, when one operation is performed using the role-playing stamp, multiple stamp images are combined together at a combination position determined based on the position of a specific part of the user's body.

In addition, multiple stamp images are synthesized for each user who appears in the captured image. If a set of two cherry images is considered as one stamp image P202, then in the example of FIG. 25, four stamp images are synthesized into one captured image P101. The two cherry images that make up the stamp image P202 may each be synthesized as a separate stamp image.

Adjusting the composite aspect of the role-playing stamp The multiple stamp images composited with the role-playing stamp are images that can be adjusted in various ways as separate content.

Figure 27 shows an example of operations when adjusting the stamp image that constitutes the role-playing stamp.

As shown on the left side of Fig. 27, for example, when a user selects a stamp image P201 superimposed at the position of the head of user U1 shown on the left side with the touch pen 142, a rectangular frame is displayed surrounding the stamp image P201, as shown on the right side of Fig. 27. A circular zoom button 621 and a delete button 622 are displayed at the bottom right and top left of the rectangular frame, respectively.

The zoom button 621 is a button used to zoom in or out on the stamp image. As described below, the zoom button 621 is also used to rotate the stamp image. The delete button 622 is a button used to delete the stamp image.

Figure 28 shows an example of enlarging the stamp image P201.

As shown in FIG. 28, the stamp image is enlarged by moving the touch pen 142 in a lower right direction while the zoom button 621 is pressed. For example, the size of the stamp image P201 is determined by releasing the touch pen 142 from the zoom button 621.

Conversely, by moving the touch pen 142 in the upper left direction while pressing the zoom button 621 with the touch pen 142, the stamp image P201 will shrink in accordance with the movement of the touch pen 142.

Figure 29 is another diagram showing an example of enlargement of stamp image P202.

As shown on the left side of FIG. 29, for example, when a user selects a stamp image P202, which is superimposed on the center of the face of user U1 shown on the left side, with the touch pen 142, a rectangular frame is displayed surrounding the stamp image P202. A zoom button 621 and a delete button 622 are displayed at the bottom right and top left of the rectangular frame, respectively.

By moving the touch pen 142 in the lower right direction while the zoom button 621 is pressed, the stamp image P202 is enlarged in accordance with the movement of the touch pen 142, as shown on the right side of FIG. 29. For example, the size of the stamp image P202 is determined by releasing the touch pen 142 from the zoom button 621.

Conversely, by moving the touch pen 142 in the upper left direction while the zoom button 621 is pressed, the stamp image P202 will shrink in accordance with the movement of the touch pen 142.

The stamp image P201 and stamp image P202, which are synthesized for user U2, can also be enlarged or reduced.

Figure 30 is a diagram showing the types of adjustments. In Figure 30, the types of adjustments are shown using a stamp image P201.

As shown in FIG. 30A, the stamp image P201 can be enlarged or reduced by using the zoom button 621.

As shown in FIG. 30B, the stamp image P201 can be rotated by using the zoom button 621. For example, the stamp image can be rotated by moving the touch pen 142 upward or downward while the zoom button 621 is pressed. The stamp image P201 rotates in accordance with the movement of the touch pen 142, and the orientation of the stamp image is determined by releasing the touch pen 142 from the zoom button 621.

A button used to rotate the stamp image may be provided separately from the zoom button 621. The button used to rotate the stamp image may be displayed, for example, in one of the four corners of the frame surrounding the stamp image. This makes it easier to visually understand that it is a button used for rotation.

As shown in FIG. 30C, stamp image P201 can be deleted by using delete button 622. For example, if two stamp images are combined, it is possible to adjust the design to your liking by deleting only one of the stamp images or deleting both stamp images.

In this way, the user can adjust the composition of stamp images that are automatically composed by using the role-playing stamp for each stamp image, i.e., partially.

Adjusting the position (movement) is also possible as an adjustment to the composition aspect of the stamp image. Adjustments to the composition aspect include not only size adjustment, orientation adjustment, and deletion as shown in FIG. 30, but also position adjustment.

Figure 31 shows an example of moving a stamp image.

The stamp image is moved, for example, by pressing the stamp image with a rectangular frame or the like displayed with the touch pen 142 and then moving the touch pen 142 in that state. The stamp image moves in accordance with the movement of the touch pen 142, and the position of the stamp image is determined by releasing the touch pen 142. In the example of FIG. 31, stamp image P201, which was composited at position p31, which is the default position, has moved to position p32, which is a position to the left of position p31.

It is also possible to move the stamp image P202 synthesized for user U1, and the stamp images P201 and P202 synthesized for user U2. The user can adjust the position of each of the multiple stamp images synthesized using the role-playing stamp from the default position to a position of their choice.

In this way, the multiple stamp images that are combined using the role-playing stamp are each separate pieces of content that can be adjusted by moving them, as well as by enlarging/reducing them, rotating them, and deleting them.

When a stamp image has been composed using a specific role-playing stamp and an operation to select another role-playing stamp is performed using selection buttons 611-1 to 611-5, the stamp image of the newly selected role-playing stamp is composed in place of the previously composed stamp image. In the photographed image to be doodled, the previously composed stamp image disappears and the newly selected stamp image is composed. The role-playing stamp is an editing tool that can be used for each photographed image, with only one type selected from multiple types available.

For example, when the selection button 611-4 is pressed and the "cat" impersonation stamp is selected in the state shown in FIG. 24 in which the "Cherry" impersonation stamp has been composited, the stamp image of the "cat" impersonation stamp is composited in place of the already-composite stamp images P201 and P202. The "cat" impersonation stamp is also composed of multiple stamp images.

Here, if the above-mentioned adjustments have been made to the stamp images P201 and P202 that make up the already-combined "Cherry" impersonation stamp, some of the adjustments will be carried over to the composition of the "Cat" impersonation stamp.

For example, if the stamp image P201 constituting the "Cherry" impersonation stamp is moved, the stamp image constituting the "Cat" impersonation stamp, whose composite position is determined based on the position of the head, is not composited in the default position but in the same position as the stamp image P201 after it was moved. In this case, the content of the movement of the stamp image is carried over.

Figure 32 shows an example of transfer of stamp image adjustments.

The state shown on the left side of Figure 32 is the state in which stamp image P201 has moved from the default position p31 to position p32, as described with reference to Figure 31. Stamp image P202 is composited at position p41 for user U1, and stamp images P201 and P202 are composited at positions p42 and p43, respectively, for user U2. Position p41 is the default composite position determined based on the center of user U1's face. Positions p42 and p43 are default composite positions determined based on the position of user U2's head and the center of his face, respectively.

In this state, if the selection button 611-4 is pressed and the "cat" impersonation stamp is selected, the stamp image of the "cat" impersonation stamp will be synthesized as shown on the right side of Figure 32.

The "cat" role-play stamp is composed of stamp image P211, which is a set of images of left and right cat ears, and stamp image P212, which is a set of images of left and right whiskers and a nose (dots). Stamp image P211 is a stamp image that is composited at a composite position determined based on the position of the user's head in the captured image. Stamp image P212 is a stamp image that is composited at a composite position determined based on the center of the user's face in the captured image.

Some of the role-playing stamps, such as the "Cherry" role-playing stamp and the "Cat" role-playing stamp, have their composite position determined based on the same position.

In the example on the right side of FIG. 32, stamp image P211 intended for user U1 is composited at position p32, and stamp image P212 is composited at position p41. Additionally, stamp image P211 intended for user U2 is composited at position p42, and stamp image P212 is composited at position p43.

In other words, the movement of the stamp image P211 before the selection of the "cat" impersonation stamp is inherited, and it is superimposed from the beginning on position p32, which was the destination of the stamp image P201.

Figure 33 shows the difference in composition positions.

The state shown on the left side of Figure 33 is the state in which each stamp image that makes up the "cat" impersonation stamp is composited in a default position. If the stamp images are not moved before the "cat" impersonation stamp is selected, as shown on the left side of Figure 33, stamp images P211 and P212 are composited at a composite position determined based on the position of the user's head in the captured image and a composite position determined based on the center of the user's face, respectively.

The state shown on the right side of Figure 33 is the state in which the movement contents have been carried over, as described with reference to Figure 32.

In this way, the adjustments to the composite mode are carried over when switching between different types of impersonation stamps, allowing users to efficiently doodle using impersonation stamps. For example, if the stamp image of the "Cherry" impersonation stamp is moved, and then the stamp image of the "Cat" impersonation stamp is composited into the default position when the user switches to the "Cat" impersonation stamp, the user would need to move it again, but this can be avoided.

Of the adjustments made to the composite target of the roleplay stamp, we have described above as the movement adjustment being carried over, but it would also be acceptable to have the zoom in/out adjustment carried over, or the rotation adjustment carried over. It would also be acceptable to have the deletion adjustment carried over.

For example, if the contents of the deletion are carried over, when the stamp image P201 is deleted after selecting the "Cherry" impersonation stamp, when the stamp image P201 is switched to the "Cat" impersonation stamp, the stamp image P211 is not composited, and only the stamp image P212 is composited.

In this way, the adjustments that are carried over can be changed as desired. At least one of the adjustments for multiple types of composite aspects will be carried over.

This allows the user to individually adjust the composition of each automatically composed stamp image, allowing them to create a beautiful, mobile phone-friendly image with a well-balanced composition.

The user can also make the above-mentioned adjustments to the graffiti using the role-playing stamps and the stamp images combined with the role-playing stamps for each captured image. The user can create multiple, high-quality images to be used exclusively on a mobile phone, based on each captured image.

If the back button is pressed after the stamp image has been moved, the position of the stamp image returns to the position before it was moved. As described above, the tool area 602L is provided with a button that is pressed to return to the previous operation. As a behavior when the back button is pressed, rather than returning the position of the stamp image to the position before it was moved, the role-playing stamp operation itself may be canceled and all stamp images composited with the role-playing stamp may be deleted.

If the position of a stamp image combined with a role-playing stamp is changed and then deleted, for example, the position before deletion is stored. If graffiti is created again using a role-playing stamp, the stamp image is combined in the position before deletion, reducing the user's effort in adjusting the position again.

-Variation Example As stamp images to be synthesized using the role-playing stamp, images for makeup such as images of eye shadow, eyeliner, and eyelashes to be synthesized at the position of the eyes on the face, a blush image to be synthesized at the position of the cheeks, and a lip image to be synthesized at the position of the lips may be used.

This allows the user to input multiple makeup images at once. The user can also adjust the composition of the multiple makeup images.

In the above, the stamp image of the role-playing stamp is synthesized when a specific operation such as pressing the selection buttons 611-1 to 611-5 is performed, but it may be automatically synthesized when doodling begins without performing such an operation. For example, a specific stamp image is randomly selected from multiple types of stamp images prepared according to the game theme, and then synthesized together.

This allows the user to easily input multiple roleplay stamp images simply by selecting a game theme. The composition of the stamp images that are automatically composed in this way can also be adjusted as described above.

The composition aspect of the sticky stamp image that is composed by selecting a sticker design may be adjustable in the same way as the stamp image of the role-playing stamp. When multiple sticky stamp images are composed by selecting a sticker design, the user can adjust the composition aspect of each sticky stamp image.

In the above, operations such as doodling have been performed using the touch pen 142, but they may also be performed using a finger.

＜Other＞
The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

The program to be installed is provided by being recorded on a removable medium such as an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or semiconductor memory. It may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be pre-installed in the ROM or memory unit.

The program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.
It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

The embodiment of this technology is not limited to the above-mentioned embodiment, and various modifications are possible without departing from the spirit of this technology.

1 Photo sticker creation device, 141 Tablet built-in monitor, 142 Touch pen, 301 PC section, 302 Pre-selection section, 304 Photography section, 305 Editing section, 306 Printing section, 401 Control section, 411 Pre-selection processing section, 412 Photography processing section, 413 Editing processing section, 414 Printing processing section, 415 Transmission processing section, 421 Image analysis section, 422 Sticker image generation section, 423 Sticker design data storage section, 424 Mobile dedicated image generation section, 425 Display control section

Claims (11)

When an editing operation using the content is performed, stamp images of different types constituting the content are synthesized at a plurality of synthesis positions determined based on an analysis result of the characteristics of a person appearing in the photographed image,
adjusting a composite state of an arbitrary stamp image selected from the composite stamp images in response to a user's operation ;
When an operation for changing the type of the content is performed, the stamp image constituting the changed content, the synthesis position of which is determined based on the same position as the arbitrary stamp image, is synthesized by inheriting the synthesis mode after adjustment.
An image processing device comprising an image processing unit. The image processing device according to claim 1 , wherein the image processing unit composites each of the stamp images at a composite position determined based on a position of a specific part of a person appearing in the photographed image. The image processing device according to claim 1 , wherein the image processing unit adjusts a synthesis state of the stamp image for each of the photographed images. The image processing unit adjusts a synthesis position as a synthesis mode of the arbitrary stamp image in response to an operation by the user, and synthesizes the stamp image constituting the changed content at the adjusted synthesis position.
4. The image processing device according to claim 1. The image processing unit adjusts a size of the arbitrary stamp image as a composite mode in response to an operation by the user, and composites the stamp image constituting the changed content with the adjusted size.
5. The image processing device according to claim 1. The image processing unit adjusts the orientation of the arbitrary stamp image as a composite mode in response to an operation by the user, and composites the stamp images constituting the changed content in the adjusted orientation.
6. The image processing device according to claim 1 . a display control unit that displays an editing screen for the captured image on which information including a button used to select the type of the content is arranged,
7. The image processing device according to claim 1, wherein the display control unit, when the arbitrary stamp image is selected from the stamp images after synthesis, displays a button used for adjusting a synthesis state of the arbitrary stamp image . The image processing device according to claim 7 , wherein the display control unit displays a frame surrounding the arbitrary stamp image, and displays a button used for adjusting a combination state of the arbitrary stamp image so as to be superimposed on the frame . The display control unit displays buttons for adjusting the size and deleting the arbitrary stamp image as buttons for adjusting the combination state of the arbitrary stamp image.
The image processing device according to claim 8. The image processing device
When an editing operation using the content is performed, stamp images of different types constituting the content are synthesized at a plurality of synthesis positions determined based on an analysis result of the characteristics of a person appearing in the photographed image,
adjusting a composite state of an arbitrary stamp image selected from the composite stamp images in response to a user's operation ;
When an operation for changing the type of the content is performed, the stamp images constituting the changed content, the synthesis position of which is determined based on the same position as the arbitrary stamp image, are synthesized by inheriting the synthesis mode after adjustment.
Image processing methods. On the computer,
When an editing operation using the content is performed, stamp images of different types constituting the content are synthesized at a plurality of synthesis positions determined based on an analysis result of the characteristics of a person appearing in the photographed image,
adjusting a composite state of an arbitrary stamp image selected from the composite stamp images in response to a user's operation ;
When an operation for changing the type of the content is performed, the stamp image constituting the changed content, the synthesis position of which is determined based on the same position as the arbitrary stamp image, is synthesized by inheriting the synthesis mode after adjustment.
A program for executing a process.

Images