JP7549565B2 - Image processing device, image processing method, and program - Google Patents

Description

This technology relates to an image processing device, an image processing method, and a program, and in particular to an image processing device, an image processing method, and a program that enable efficient adjustment of the appearance of a subject in a captured image.

In photo taking devices such as passport photo machines and print sticker machines installed in game arcades, the user can adjust the appearance of the captured image. The user can adjust the appearance of the image that is printed on sticker paper or the like, one by one, to their liking.

JP 2015-118329 A

Adjusting the quality of a large number of images, such as 50 or 100, one by one is tedious.

This technology was developed in light of these circumstances, and makes it possible to efficiently adjust the appearance of subjects in captured images.

An image processing device according to one aspect of the present technology acquires selected images, which are some of a plurality of captured images selected in response to a user's operation on a first screen, adjusts the appearance of each of the selected images in response to the user 's operation on a second screen , and, after an operation to complete the adjustment of the appearance of the plurality of selected images has been performed , includes an image processing unit that reflects the adjustment contents of the appearance of the selected image that is closest in at least any of the size, position, orientation, brightness, and number of faces depicted in each of the unselected images, which are the captured images that were not selected for the adjustment of appearance .

In one aspect of the present technology, selected images, which are some of a plurality of captured images selected in response to a user's operation on a first screen, are acquired, and the appearance of each of the selected images is adjusted in response to the user 's operation on a second screen . After an operation to finish adjusting the appearance of the selected images is performed , the adjustment content of the appearance of the selected image that is closest in at least one of the size, position, orientation, brightness, and number of faces is reflected in each of the unselected images, which are the captured images that were not selected as targets for the adjustment of appearance .

This technology makes it possible to efficiently adjust the appearance of a subject in a captured image.

FIG. 1 is a diagram showing a workflow in a photography studio. FIG. 13 is a diagram showing an example of providing a captured image. FIG. 13 is a diagram showing an example of file management in a retouching PC. FIG. 13 is a diagram showing an example of an image selection screen. FIG. 13 is a diagram showing an example of a retouch screen. FIG. 13 is an enlarged view showing a display of a retouch selection area. FIG. 13 is a diagram illustrating an example of application of retouching parameters. FIG. 13 is a diagram illustrating another example of application of retouching parameters. FIG. 13 is a diagram illustrating an example of application of retouching parameters. FIG. 13 is a diagram illustrating an example of calculation of a face ratio. FIG. 2 is a block diagram showing an example of the configuration of a retouching PC. FIG. 2 is a block diagram showing an example of the functional configuration of a retouching PC. 11 is a flowchart illustrating a retouching process.

<Work flow in the photography studio>
FIG. 1 is a diagram showing the flow of work in a photography studio.

This technology is used, for example, in photography studios where commemorative photos are taken for coming-of-age ceremonies. This technology can also be applied to photography studios where commemorative photos for various events, such as entrance/graduation ceremonies, wedding photos, and Shichi-Go-San ceremonies, are taken.

As shown on the left side of Figure 1, photography in a photography studio is performed by a photographer. In the example of Figure 1, a user wearing a long-sleeved kimono is standing in front of a background screen and the subject of the photography is a user. Strobe devices that emit light in time with the photographer's photography are placed around the user.

The photographer takes pictures by changing the composition and taking many still images (photographs), such as 50 or 100. The composition of the still image is determined by the user's pose, the relative positions of the user and the photographer, and camera parameters such as focal length. The still images taken by the photographer are transferred to and saved in a PC (not shown) connected to the camera.

After the shooting is finished, the user moves to the retouching space prepared in the shooting studio and uses the retouching PC 1 to check the images taken by the photographer, as shown on the right side of Figure 1.

The user also selects the number of images to be printed, for example, and adjusts the appearance of each image by retouching them themselves. Retouching is performed on a portion of the images, such as five or ten images, selected from the images taken by the photographer. Retouching (processing) includes various types of image processing to change the appearance of the images.

In the retouching space, one or more retouching PCs 1 are prepared. The captured images stored in the storage PC are transferred to the retouching PC 1 and displayed on the display 11 of the retouching PC 1. A retouching application 1A, which is an application used to retouch images taken by a photographer, is installed on the retouching PC 1.

In this way, in a photography studio where a retouching PC 1 to which the present technology is applied is installed, the user himself/herself retouches the photographed image taken by the photographer. The user can use the functions of the retouching application 1A to retouch the photographed image to obtain the desired appearance. The retouching application 1A may be installed on a terminal other than the PC, such as a tablet terminal, and used for retouching.

Figure 2 shows an example of providing captured images.

The captured images retouched by retouching application 1A are printed by the photography studio's management company and later given to the user in the form of, for example, an album, as shown by arrow #1.

The retouched photographed images are uploaded from the retouching PC 1 to the image management server 2, as shown by arrow #2, and are managed by the image management server 2. The image management server 2 is, for example, a server on the Internet. The image management server 2 is managed by the management company of the photography studio, or by a business entity different from the management company of the photography studio.

By operating their own mobile terminal 3 such as a smartphone and operating the image management app 3A, the user can view and download images that have been taken of them as the subject, as shown by arrow #3. The image management app 3A is an application for viewing, etc., the captured images managed in the image management server 2. The image management app 3A is provided, for example, by the business entity that manages the image management server 2.

The images taken in the photography studio will be provided in the form of an album and, where appropriate, in the form of data. It may be possible to provide the images in the form of data for a fee separate from the fee for the album.

The provision of photographed images in the form of data is carried out so that not only photographed images that have been retouched by the user, but also photographed images that have not been selected by the user as subjects for retouching are provided. Photographed images that have not been selected by the user as subjects for retouching are automatically retouched by the retouching application 1A without any user operation, as described below.

The user can receive in the form of data even captured images that were not selected for retouching. Instead of using the image management app 3A, the user can use a browser to access a specific site and receive captured images managed in the image management server 2. For example, by registering as a member of an image providing service provided by the administrator of the image management server 2, the user can receive captured images managed in the image management server 2 using the image management app 3A or a browser.

Instead of the mobile terminal 3, it may be possible to receive captured images using other terminals such as a tablet terminal or a PC.

<About the retouching app>
File Management FIG. 3 is a diagram showing an example of file management in the retouching PC 1. As shown in FIG.

As shown in the speech bubble in Figure 3, two folders, the all folder and the album folder, are prepared and linked to the user ID, which is the user's identification information. The all folder is a folder with the folder name "all," and the album folder is a folder with the folder name "album."

The captured image files transferred from the storage PC are stored in the all folder. The captured image files that were selected for retouching and have been retouched are stored in the album folder. The files stored in the album folder are the files used for printing.

When a user who has finished taking a photograph starts retouching, the user ID is entered on the screen of retouching application 1A. Based on the user ID, the captured image is read by retouching application 1A and processing is performed using the captured image. For example, the files of all captured images stored in the all folder linked to the user ID are read by retouching application 1A. After the captured images are read, the captured images to be retouched are selected.

Screen Display FIG. 4 is a diagram showing an example of an image selection screen.

A message is displayed at the top of the image selection screen instructing the user to select five captured images. The number of selected captured images is set, for example, by a staff member at the photography studio before image selection. If the user has signed up for a plan to receive five captured images printed as an album, five is set as the number of selected images.

A thumbnail display area 21 is formed on the right side of the image selection screen. The thumbnail display area 21 is an area in which thumbnail images of the captured images are displayed. For example, if 50 captured images have been taken, thumbnail images of each of the 50 captured images are displayed. Various operations using the screen of the retouching application 1A are performed using a mouse, keyboard, etc.

A photographed image selected using a thumbnail image is enlarged and displayed in a display area 22 formed to the left of the thumbnail display area 21. In the example of FIG. 4, the thumbnail image of photographed image P1 displayed at the upper left corner of the thumbnail display area 21 is selected, and photographed image P1 is enlarged and displayed in the display area 22.

A confirm button 23 is displayed below the display area 22. The confirm button 23 is a button that is pressed when deciding on the photographed image displayed in the display area 22 as the one to be retouched. The user selects thumbnail images to sequentially enlarge and display each photographed image in the display area 22, and when the user finds a photographed image that the user likes, the user presses the confirm button 23 to select it as the one to be retouched.

The number of remaining selections is displayed above the thumbnail display area 21. Each time a photographed image to be retouched is selected, the display of the number of selections is updated.

When five photographed images have been selected and the end button 24 has been pressed, the selection of photographed images to be retouched is completed. After the selection of photographed images to be retouched is completed, retouching is performed on each of the photographed images selected as the retouched images.

Figure 5 shows an example of a retouch screen.

A display area 31 is formed at the upper center of the retouch screen. The display area 31 is an area in which the photographic image currently being retouched is displayed in an enlarged form. A photographic image selected in the thumbnail display area 32 using a thumbnail image is displayed in an enlarged form in the display area 31.

A thumbnail display area 32 is formed in the lower left of the retouch screen. The thumbnail display area 32 is an area in which thumbnail images of the photographed images selected on the image selection screen are displayed. In the example of FIG. 5, thumbnail images of five photographed images P1 to P5 are displayed in the thumbnail display area 32. The photographed image P1, which is selected using the thumbnail image, is enlarged and displayed in the display area 31.

A retouching selection area 33 is formed below the retouching screen. The buttons displayed in the retouching selection area 33 are used to select the retouching content. Each time a retouching content is selected using a button displayed in the retouching selection area 33, the selected content is reflected in the display in the display area 31. While viewing the display in the display area 31, the user can use the display in the retouching selection area 33 to select the retouching content to be applied to the captured image currently being retouched.

Figure 6 shows an enlarged view of the retouch selection area 33.

The retouch selection area 33 displays buttons used to select eye size, eye shape, makeup intensity, nose bridge on/off, skin type, face shape, face slimming, and style up on/off.

It is possible to select from five different eye sizes. Eye size "1" indicates no processing, i.e., the eye size is left unchanged at its original size. By selecting one of the sizes, the user can adjust the size of their own eyes as they appear in the captured image displayed in the display area 31.

The user can select one of the following eye shapes: "round eyes," "downturned eyes," or "cat eyes." By selecting one of these shapes, the user can adjust the shape of their own eyes that will appear in the captured image displayed in the display area 31. By selecting "no processing," it is also possible to leave the shape of the eyes unchanged.

The user can select from three levels of makeup intensity. The makeup intensity of "1" indicates that the makeup intensity is not changed and remains at the original intensity. By selecting one of the intensities, the user can adjust the intensity of their makeup that appears in the captured image displayed in the display area 31.

It is possible to select ON/OFF for the nose bridge. By selecting ON or OFF, the user can choose whether to emphasize the highlights of the bridge of their nose that appear in the captured image displayed in the display area 31, or to leave it in its original state.

It is possible to select from five levels of strength for skin texture. Skin texture "1" indicates that the skin texture is not changed and remains the original skin texture. By selecting one of the strengths, the user can adjust the degree of smoothness of their skin texture as it appears in the captured image displayed in the display area 31.

The user can select one of three face shapes: "round," "egg," or "slim." By selecting one of these shapes, the user can adjust the shape of their face that appears in the captured image displayed in the display area 31. By selecting "no processing," it is also possible to leave the shape of the face unchanged.

Five different sizes can be selected to achieve a smaller face. A smaller face size of "1" indicates that the size of the face is not changed and remains at its original size. By selecting one of the sizes "2" through "4," the user can adjust the size of their face in the captured image displayed in the display area 31.

The style enhancement can be selected to be on or off. By selecting on or off, the user can choose whether to increase the user's height in the captured image displayed in the display area 31 or to keep the original height.

Using these various functions, the user can select the retouching content to be applied to the captured image displayed in the display area 31. In this way, retouching that can be performed using the retouching application 1A includes at least adjusting the size and shape of the eyes and the size and shape of the face.

The captured image to be retouched is switched using the display in the thumbnail display area 32, and retouching is performed sequentially on each of the captured images P1 to P5. The retouched files for each of the captured images P1 to P5 are stored in the album folder.

Returning to the explanation of FIG. 5, a zoom in/out button 34 is provided on the right side of the display area 31. The zoom in/out button 34 is a button that is operated to zoom in on a portion of the captured image displayed in the display area 31 or to zoom out on the zoomed in portion.

Below the zoom in/out buttons 34 is a before-editing button 35. The before-editing button 35 is a button that is operated when switching the display in the display area 31 to the state before retouching. While the before-editing button 35 is pressed, the captured image in the state before retouching is displayed in the display area 31. By using the before-editing button 35, the user can compare the state after retouching with the state before retouching and check whether the degree of retouching is too strong, for example.

At the top right of the retouching screen, a back button 36 is displayed, which is operated to return to the image selection screen, and an end button 37 is displayed, which is operated to end retouching. When the retouching content for each of the captured images P1 to P5 is selected and the end button 37 is pressed, the retouching ends.

- Transfer of retouching parameters The retouching parameters are parameters that indicate the content of retouching selected using the display of the retouching selection area 33. When retouching is performed on a certain photographed image, the retouching parameters of the photographed image to be retouched are changed. The retouching parameters for a certain photographed image are applied as the retouching parameters for other photographed images.

Figures 7 and 8 show examples of applying retouching parameters.

As shown by arrow #11 in FIG. 7, when the captured image P1 is the first image to be retouched, the user operates to set the following retouching parameters for the captured image P1: "eye size" to "3," "eye shape" to "cat eyes," "makeup intensity" to "2," and so on. The appearance of the retouched captured image P1 shown in the lower part of FIG. 7 is different from the appearance of the captured image P1 before retouching shown in the upper part.

When retouching of photographed image P1 is completed and the retouching target is switched from photographed image P1 to photographed image P2, the retouching parameters of the retouched photographed image P1 are inherited as the default retouching parameters of photographed image P2, which is the second photographed image to be retouched, as shown at the end of arrow #12 in FIG. 8. In the display area 31, photographed image P2 is displayed in a state in which it has been retouched in the same manner as retouched photographed image P1. The display of retouching selection area 33 also changes according to the retouching content applied to photographed image P2.

The user can start retouching the photographed image P2 using the photographed image P2, which has been retouched in the same manner as the photographed image P1, as a reference. In the example of FIG. 8, the images have been adjusted to the contents represented by the retouching parameters: "eye size" is "4", "eye shape" is "cat eyes", "makeup intensity" is "1", etc.

Similarly, when retouching of photographed image P2 is completed and a third photographed image to be retouched is selected, the retouching parameters of photographed image P2 are carried over as the default retouching parameters for the third photographed image to be retouched, as shown by arrow #14.

In this way, when a new captured image that has not yet been retouched is selected as the captured image to be retouched, the retouching contents of the immediately preceding captured image to be retouched are carried over. This allows the user to efficiently retouch each captured image.

Each captured image to be retouched may be classified according to the composition of the subject, and retouching parameters of a captured image with the same composition may be applied. In this case, the composition of each captured image is analyzed by the retouching application 1A, and each captured image is classified as a captured image with one of multiple types of compositions, such as "close-up," "bust-up," and "full body."

A "close-up" is a composition in which the subject's face is shown prominently, and a "bust-up" is a composition in which the upper half of the subject's body is shown prominently. A "full-body" is a composition in which the subject's entire body is shown. For example, the size of the face is analyzed, and the composition is classified according to the size of the face. Compositions may also be classified according to not only the size of the face, but also the direction and position of the face.

For example, if a close-up image has been retouched and then the same close-up image is selected as the image to be retouched, the retouching parameters of the retouched image are set as the retouching parameters of the new close-up image selected as the image to be retouched.

Appropriate retouching parameters may be prepared in advance for each composition. When a photographed image is selected as the subject of retouching, the retouching parameters prepared in advance for the composition of that photographed image are set as the retouching parameters for the photographed image newly selected as the subject of retouching.

A recommendation button that is operated when applying recommended retouching parameters may be provided on the retouching screen. When the recommendation button is pressed, retouching parameters that have been prepared in advance for the composition of the photographed image selected as the subject of retouching are set.

Recommendation buttons for each composition may be provided on the retouching screen. In the example described above, recommendation buttons are provided for each of the compositions "close-up," "bust-up," and "full body." For example, information indicating the type of composition of the photographed image selected to be retouched is displayed near the display area 31. The user can check the composition of the photographed image selected to be retouched and set appropriate retouching parameters by pressing the recommendation button for the same composition.

Retouching parameters may be recorded. For example, the retouching parameters set by the user are recorded in association with the user ID along with information on the type of composition of the captured image to be retouched.

Appropriate retouching parameters may be suggested using an inference model generated by machine learning. In this case, the retouching app 1A is equipped with an inference function that uses an inference model that takes a captured image as input and outputs retouching parameters.

<Applying retouching parameters to unselected images>
The five photographed images thus retouched in response to the user's operations are printed on photographic paper and provided to the user in the form of an album. The photographed images retouched in response to the user's operations are also provided in the form of data together with the photographed images not selected as retouching targets on the image selection screen (FIG. 4).

Hereinafter, a photographed image that has not been selected as a subject for retouching will be referred to as an unselected image, where appropriate. A photographed image that has been selected as a subject for retouching will be referred to as a selected image.

Unselected images are automatically retouched by the retouching application 1A without any operation by the user. Unselected images provided in the form of data also become retouched photographed images.

Figure 9 shows an example of applying retouching parameters.

As shown in FIG. 9, the retouching contents applied to each of the captured images P1 to P5 by user operations are reflected in each of the unselected images stored in the all folder. The retouching contents applied to each of the captured images P1 to P5 may be reflected only to the unselected images stored in the all folder, or may be reflected to all captured images stored in the all folder.

Which of the retouched images P1 to P5 should have its retouching applied to each unselected image is selected based on the state of the face in the selected image and the state of the face in the unselected image. A selected image in which a face appears in a state similar to the state of the face in each unselected image is selected, and the retouching applied to the selected selected image is reflected in the unselected image.

For example, the size of the face can be used as the state of the face in the selected image and unselected images. Specifically, the face ratio, which is the ratio of the face size to the size of the captured image, is calculated for each captured image (selected image and unselected image), and retouching is performed on each unselected image so as to reflect the retouching content of the selected image with a similar face ratio.

Figure 10 shows an example of calculating face ratio.

Figure 10A shows an example of calculating the face ratio for a captured image P1, which is a horizontally long image. Figure 10B shows an example of calculating the face ratio for a captured image P4, which is a vertically long image.

Face recognition is performed, and the area enclosed by a frame F is recognized as the area of the face. In addition, a face ratio Z is calculated based on the width X of the frame F and the length Y of the short side of the captured image, for example, by the following formula (1).
Face ratio Z = width X of frame F ÷ length Y of short side ... (1)

For the horizontally long captured image P1 shown in FIG. 10A, the face ratio is calculated based on the width of the frame F and the height of the captured image indicated by the dashed-dotted arrow. On the other hand, for the vertically long image shown in FIG. 10B, the face ratio is calculated based on the width of the frame F and the width of the image indicated by the dashed-dotted arrow.

The face ratio is calculated for each of the captured images P1 to P5. In addition, the face ratio is calculated for each of the unselected images.

For a given unselected image, the same retouching content applied to a retouched photographed image whose face ratio is calculated to be, for example, closest to the face ratio of the unselected image is reflected.

For example, if the face ratios of the captured images P1 to P5 are calculated as 0.30, 0.40, 0.50, 0.60, and 0.70, then the unselected image with a calculated face ratio of 0.32 is retouched in the same manner as the captured image P1. The unselected image with a calculated face ratio of 0.80 is retouched in the same manner as the captured image P5.

In studio photography, multiple images are usually taken with similar compositions. When a user selects an image with a certain composition to be retouched and performs retouching, the unselected images include images with a similar composition to the retouched image. By selecting which retouched image to apply the retouching content to based on the face condition, such as face ratio, it becomes possible to apply retouching to the unselected images in line with the user's intentions.

The user can receive unselected images that have been retouched in accordance with their intentions, without having to select each image one by one and retouch them manually. The user can efficiently adjust the appearance of the captured image.

The face ratio may be calculated by a method other than the method using the above formula (1). For example, the face ratio may be calculated based on the width of the frame F and the length of the long side of the image, or the face ratio may be calculated based on the area of the face range relative to the area of the image.

When there are multiple users as subjects and therefore multiple faces are captured in the image, the average face ratio of each person may be used as the face ratio for the entire image.

The state of the face in the captured image may be represented by information other than the face ratio and used as a criterion for which captured image the retouching content is to be reflected. It is possible to use face states such as face position, face direction, face brightness, and number of faces. A combination of multiple pieces of information regarding face state may also be used. In other words, at least one of the face size including face ratio, face position, face direction, face brightness, and number of faces can be used as the face state.

The retouching content of the captured image to be reflected in the unselected image may be selected based on criteria other than the face condition, such as face ratio. For example, the selection may be based on the eye condition, or the gender of the subject. As with the face condition, at least one of the eye size, eye position, and eye direction can be used as the eye condition.

It is also possible to select which captured image's retouching content is to be reflected in the unselected image based on the type of composition described above.

<Configuration and operation of retouching PC 1>
Configuration of Retouching PC 1 FIG. 11 is a block diagram showing an example of the configuration of the retouching PC 1. As shown in FIG.

CPU (Central Processing Unit) 101, ROM (Read Only Memory) 102, and RAM (Random Access Memory) 103 are interconnected by bus 104.

An input/output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a memory unit 108, a communication unit 109, and a drive 110 are connected to the input/output interface 105.

The input unit 106 is composed of a keyboard, mouse, etc.

The output unit 107 is composed of a display 11, etc.

The storage unit 108 is composed of a hard disk, a non-volatile memory, etc. The storage unit 108 stores various information such as programs executed by the CPU 101, such as the retouching application 1A, and captured images.

The communication unit 109 is an interface to a network such as the Internet or a LAN. For example, the communication unit 109 communicates with a storage PC connected to a camera in a photography studio, and receives a photographed image sent from the storage PC, in which the user about to begin retouching appears as a subject. The communication unit 109 also communicates with the image management server 2, and sends the retouched photographed image to the image management server 2.

The drive 110 controls writing of data to the removable medium 111 and reading of data from the removable medium 111. Import of captured images from a PC for storage may be performed via the removable medium 111.

Figure 12 is a block diagram showing an example of the functional configuration of retouching PC 1.

As shown in FIG. 12, in the retouching PC 1, an image processing unit 121 is realized by executing the retouching application 1A. The retouching PC 1 is an image processing device having the image processing unit 121.

The image processing unit 121 is composed of an image acquisition unit 131, a retouch processing unit 132, an output control unit 133, and a display control unit 134.

The image acquisition unit 131 accepts a user's operation on the image selection screen, and acquires the captured image selected by the user as the target for retouching from, for example, the storage unit 108. The captured image to be retouched acquired by the image acquisition unit 131 is supplied to the retouching processing unit 132 and the display control unit 134.

In addition, when retouching of each captured image to be retouched is completed, the image acquisition unit 131 acquires an unselected image, for example, from the memory unit 108, and outputs it to the retouching processing unit 132.

The retouching processing unit 132 performs face recognition to recognize the face appearing in the captured image to be retouched. The retouching processing unit 132 analyzes the features of each part of the recognized face and performs various image processing related to retouching in response to the user's operation on the retouching screen. The analysis of the features of each part may be performed using an inference model generated by machine learning.

For example, when an operation is performed to adjust the size or shape of the eyes, the retouching processor 132 performs image processing such as shifting the position of the eye outline in accordance with the user's selection. When an operation is performed to adjust the intensity of makeup, the retouching processor 132 performs image processing such as compositing an image of a blush of a specified color onto the cheeks, or compositing an image of a lipstick of a specified color onto the lips.

When an operation related to adjusting the skin type is performed, the retouching processing unit 132 performs image processing such as adjusting the brightness of the skin. When an operation related to adjusting the face size (feeling of a small face) or face shape is performed, the retouching processing unit 132 performs image processing such as shifting the position of the face outline according to the content selected by the user.

Information representing the results of image processing by the retouching processing unit 132 is supplied to the display control unit 134 and used to display the retouched screen.

When retouching of each photographed image to be retouched is completed, the retouching processing unit 132 analyzes each photographed image, including the unselected image, as described above, and reflects the retouching content applied to the retouched photographed image to the unselected image. The photographed images retouched by the retouching processing unit 132 are supplied to the output control unit 133.

The output control unit 133 saves the captured image that has been retouched in response to user operations in the album folder.

The output control unit 133 also outputs the retouched captured images (selected images and unselected images) to the communication unit 109 and transmits them to the image management server 2.

The display control unit 134 controls the display of various screens of the retouching application 1A, such as the image selection screen and the retouching screen.

Operation of Retouching PC 1 The retouching process performed by the retouching application 1A will now be described with reference to the flowchart of FIG.

In step S1, the display control unit 134 of the image processing unit 121 displays the image selection screen (Figure 4).

In step S2, the image acquisition unit 131 accepts the selection of the captured image to be retouched in response to a user's operation on the image selection screen. The image acquisition unit 131 acquires the captured image to be retouched.

In step S3, the image acquisition unit 131 determines whether or not the selection of the captured images to be retouched has been completed. If it is determined in step S3 that the selection of the captured images to be retouched has not been completed, the process returns to step S1, and image selection continues.

For example, if five captured images are selected and the end button 24 is pressed, it is determined in step S3 that the selection of captured images to be retouched has ended.

In step S4, the display control unit 134 displays a retouch screen (FIG. 5). The user uses the retouch screen to retouch the captured image.

In step S5, the retouching processing unit 132 performs image processing in response to the user's operation, and retouches the captured image selected as the target for retouching.

In step S6, the retouching processing unit 132 determines whether or not retouching is complete. If it is determined in step S6 that retouching is not complete, the process returns to step S4, and retouching continues for each captured image selected as the target for retouching.

For example, when all captured images selected as retouching targets have been retouched and the end button 37 is pressed, it is determined in step S6 that retouching is to be ended.

In step S7, the retouching processor 132 calculates the face ratio for each of the selected and unselected images.

In step S8, the retouching processing unit 132 applies to each unselected image the same retouching content as that applied to the selected image whose face ratio is calculated to be closest to the face ratio of the unselected image.

In step S9, the output control unit 133 saves the retouched unselected image.

In step S10, the communication unit 109 transmits the retouched selected images and the unselected images to the image management server 2. After that, the processing of the retouching application 1A ends.

By performing the above process, the user can adjust the appearance of the images taken of themselves as the subject. The user can receive an album of the images that have been taken to their liking.

In addition, by using the mobile terminal 3, the user can receive, in the form of data, photographed images that have been manually adjusted and photographed images (unselected images) that have been automatically retouched in the same manner as the manual adjustments, which are managed by the image management server 2.

<Modification>
- Examples of retouching locations In the above embodiment, retouching is performed using a PC installed in a photography studio, but retouching may also be performed using other devices, such as a mobile terminal 3 or a PC installed in the user's home.

When retouching using the mobile terminal 3 is possible, after a user has finished taking photos in a photography studio, the user operates the mobile terminal 3 to launch the image management app 3A and performs retouching on the screen of the image management app 3A. The image management app 3A has the same functions as the retouching app 1A.

The above-mentioned screen is displayed on the display of the mobile terminal 3 by the image management application 3A, and the photographed images to be retouched are selected and retouched from among the photographed images uploaded to the image management server 2. The photographed images that have been retouched using the functions of the image management application 3A are printed and provided to the user in the form of an album, or the retouched photographed images are provided to the user as is in the form of data.

This allows users to retouch photos in various locations, such as at home. Users can also retouch photos to their heart's content without worrying about time.

Switching Screen Display The above-described retouching function can also be applied to retouching other captured images, such as commemorative wedding photos, etc. In a captured wedding image, for example, two users, the bride and groom, may appear.

In this case, it may be possible to select on the retouching screen whether to retouch the person on the left side or the person on the right side. This makes it possible to apply different retouching parameters to the retouching of the groom and the bride. It may also be possible to retouch the groom and the bride simultaneously with a single operation.

In this way, the display of the retouching screen and the type of retouching available are switched depending on the type of commemorative photo. The type of commemorative photo, such as a coming-of-age ceremony photo or a wedding photo, may be selected before retouching begins.

- Warning for over-enhancing As described above, the user can adjust the appearance of his/her face in the photographed image according to his/her preference, and enhance his/her face in the photographed image. Commemorative photographs such as coming-of-age ceremonies and wedding photographs are important photographs of life events that will remain for a long time.

To prevent such important photographs from being overdone (retouched to an excessive degree), a warning to that effect may be displayed if the degree of retouching selected by the user is greater than a threshold degree. The warning to be careful not to overdone is displayed, for example, on the retouching screen while the user is retouching.

You can set combinations of retouching that result in excessive retouching, and display a warning when the user tries to select such a combination.

As described above, a predetermined number of captured images (e.g., 5 or 10 images) that have been adjusted by the user are printed on photographic paper and provided to the user in the form of an album. If there is variation in the quality of each captured image, the album as a whole loses its sense of unity.

To prevent the sense of unity of the album as a whole from being lost, the degree of retouching that can be selected may be limited. For example, one reference photograph is selected, and the degree of retouching that can be selected is limited based on the content of the retouching applied to that photograph.

It may also be possible to make it possible to compare retouched photographed images while retouching. For example, when a specified operation is performed on the retouching screen, the retouched photographed images are displayed side by side. This makes it possible to ensure a sense of unity in the appearance of each photographed image.

Others Although the number of images to be retouched can be selected to be the same as the number of copies to be printed, it is also possible to select a number of images greater than the number of copies to be printed. In this case, the user selects from among the retouched images the images to be actually printed and received in the form of an album.

Although still images are taken in a photography studio, moving images may also be taken. In this case, the moving images are taken as the captured images, and the captured images can be retouched by the user.

Regarding the program: The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware, or in a general-purpose personal computer.

The program to be installed is provided by being recorded on removable media 111 shown in FIG. 11, which may be an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. It may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be pre-installed in ROM 102 or storage unit 108.

The program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

The effects described in this specification are merely examples and are not limiting, and other effects may also exist.

The embodiment of this technology is not limited to the above-mentioned embodiment, and various modifications are possible without departing from the gist of this technology.

For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices over a network.

In addition, each step described in the above flowchart can be executed by a single device, or can be shared and executed by multiple devices.

Furthermore, when a single step includes multiple processes, the multiple processes included in that single step can be executed by a single device, or can be shared and executed by multiple devices.

1 Retouching PC, 2 Image management server, 3 Mobile terminal, 11 Display, 121 Image processing section, 131 Image acquisition section, 132 Retouching processing section, 133 Output control section, 134 Display control section

Claims (4)

On the computer,
selecting a selected image which is a part of the plurality of captured images in response to a user's operation on the first screen ;
adjusting the appearance of each of the selected images in response to an operation by the user on a second screen ;
After an operation for completing the adjustment of the appearance of the selected images is performed , the adjustment of the appearance of the selected image that is closest in at least one of the size, position, orientation, brightness, and number of faces is reflected in each of the unselected images that are the photographed images that were not selected as the target for the adjustment of the appearance.
A program that executes a process. The program according to claim 1 , wherein the adjustment of the appearance of the selected image includes at least adjustment of the size and shape of the eyes and adjustment of the size and shape of the face. The image processing device
selecting a selected image which is a part of the plurality of captured images in response to a user's operation on the first screen ;
adjusting the appearance of each of the selected images in response to an operation by the user on a second screen ;
After an operation for completing the adjustment of the appearance of the selected images is performed , the adjustment of the appearance of the selected image that is closest in at least one of the size, position, orientation, brightness, and number of faces is reflected in each of the unselected images that are the photographed images that were not selected as the target for the adjustment of the appearance.
Image processing methods. acquiring a selected image, which is a part of the plurality of captured images selected in response to a user operation on the first screen ;
adjusting the appearance of each of the selected images in response to an operation by the user on a second screen ;
After an operation for completing the adjustment of the appearance of the selected images is performed , the adjustment of the appearance of the selected image that is closest in at least one of the size, position, orientation, brightness, and number of faces is reflected in each of the unselected images that are the photographed images that were not selected as the target for the adjustment of the appearance.
An image processing device comprising an image processing unit.

Images