JP7651306B2 - Image capture device, image capture device control method and program - Google Patents

Description

The present invention relates to an imaging device, a control method for an imaging device, and a program.

In recent years, it has become possible to view multiple images posted on social network services (SNS) and the like in a list display. When multiple images displayed in a list have similar subjects, compositions, brightness, colors, etc., they can create a sense of unity and give a visually pleasing impression. As a technique for shooting multiple images, Patent Document 1 discloses a technique for displaying an image to be reshot and a live view image, which is a moving image being shot, side by side, allowing the user to create the pose and expression being shot while checking the image to be reshot.

JP 2017-116872 A

However, even if a live view image, which is a moving image being shot, is displayed alongside a previously shot image, it can be time-consuming to take a picture that is consistent with the previously shot image, such as changing the pose or position of the subject and taking the picture again.

The present invention aims to provide an imaging device that can easily generate an image group that includes multiple images with a unified look.

In order to achieve the above object, an imaging device according to the present invention comprises:
a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording, in a recording section, a photographed image of the subject photographed by the photographing means and the photographed image in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image ;
an uploading means for uploading the photographed image of the subject and the photographed image together with the arrangement information recorded in the recording unit to an external server;
having
The display means displays a live image of the subject captured using the shooting conditions changed by the change means, together with the plurality of captured images.

The present invention makes it easy to generate image groups that contain multiple images with a unified look.

FIG. 1 is a block diagram showing an example of the configuration of an imaging apparatus. FIG. 1 is a diagram illustrating an example of the appearance of an imaging apparatus. 4 is a flowchart illustrating a shooting process according to the first embodiment. 4A and 4B are diagrams for explaining the arrangement of live images and captured images. 5A to 5C are diagrams illustrating adjustment of a live image. 10 is a flowchart illustrating an image capturing process according to a second embodiment. 13A to 13C are views for explaining examples of screens in the photographing process according to the second embodiment. 13A to 13C are diagrams illustrating examples of metadata indicating features of an image according to the third embodiment. 13 is a flowchart illustrating a photographing process according to a third embodiment. 13 is a flowchart illustrating a captured image arrangement process according to a third embodiment. FIG. 13 is a diagram illustrating a UI according to the third embodiment. 13 is a flowchart illustrating a captured image arrangement process according to a fourth embodiment. FIG. 13 is a diagram illustrating a UI according to the fourth embodiment. 13 is a flowchart illustrating a captured image arrangement process according to a fifth embodiment. 20 is a flowchart illustrating a captured image arrangement process according to a sixth embodiment. 13A and 13B are diagrams illustrating examples of templates of display positions of captured images according to the seventh embodiment. FIG. 23 is a diagram illustrating a live view display screen according to the seventh embodiment. 20 is a flowchart illustrating an image capturing process according to an eighth embodiment. FIG. 23 is a diagram illustrating a live view display screen according to an eighth embodiment.

<Embodiment 1>
Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In the first embodiment, a live view display screen that displays a live image, which is a moving image of a subject, displays a captured image that has been captured in the past together with the live image. The display position at which the live image is displayed on the screen of the imaging device and the display position at which the captured image is arranged can be determined based on a user's operation. The user can capture an image that has a sense of unity with the captured image by adjusting the composition, brightness, color, etc. of the live image to match the captured image. There are various elements that make the user feel a sense of unity. For example, commonality in composition, such as the position and size of the subject in the image being similar to each other, or the position of the horizon in a landscape image being the same height, can be cited. Another example is commonality in the categories of subjects, such as food, people, landscapes, and animals. Furthermore, commonality in the color and gradation of the subject, and commonality in filters applied to the image, such as saturation emphasis, can be cited.

The imaging device according to the first embodiment will be described as a smartphone with a camera function, but is not limited to this. The imaging device may be other devices with an imaging function, such as a digital camera, a digital video camera, a mobile phone, or other portable electronic devices.

(Device configuration)
1 is a block diagram showing an example of the configuration of an image capture device 100 according to the first embodiment.

The control unit 101 is a processor such as a CPU or DSP. The control unit 101 reads out a control program for each block of the imaging device 100 from the ROM 102, and deploys it in the RAM 103 for execution. This allows the control unit 101 to control the operation of each block of the imaging device 100.

The ROM 102 is a non-volatile memory that can be electrically erased and recorded. The ROM 102 stores the operation programs of each block of the imaging device 100, as well as parameters used for the operation of each block.

RAM 103 is a rewritable volatile memory. RAM 103 is used to expand programs executed by control unit 101 and the like, and to temporarily store data generated by the operation of each block of imaging device 100.

The optical system 104 is composed of a group of lenses including a zoom lens and a focus lens, and forms an image of a subject on an imaging surface of the imaging unit 105. The imaging unit 105 is, for example, an imaging element such as a CCD or CMOS sensor. The imaging unit 105 photoelectrically converts the optical image formed on the imaging surface of the imaging unit 105 by the optical system 104, and outputs the obtained analog image signal to the A/D conversion unit 106.

The A/D conversion unit 106 converts the input analog image signal into digital image data. The digital image data output from the A/D conversion unit 106 is temporarily stored in the RAM 103.

The image processing unit 107 applies various image processing to the image data stored in the RAM 103 in order to develop, display, or record digital image data. Examples of image processing applied to the image data include pixel defect correction processing caused by the optical system 104 or the image sensor, demosaicing processing, white balance correction processing, color interpolation processing, gamma processing, etc.

The recording unit 108 records data including image data on a built-in recording medium. The display unit 109 includes a display device such as an LCD (Liquid Crystal Display), and displays images stored in the RAM 103 and images recorded in the recording unit 108 on the display device. The display unit 109 also displays an operational user interface, etc., for receiving instructions from the user.

The input unit 110 is an input interface including various physical operating members such as a touch panel and a shutter button, and accepts input of instructions from the user. The communication unit 111 wirelessly connects to an external server and transmits and receives data including image data and layout information (described below). An example of an external server is a server that manages a service such as a social network service (hereinafter referred to as SNS) that receives images taken by users and makes them available to viewers of the SNS.

Under the control of the control unit 101, the imaging device 100 performs live view display in which analog image signals sequentially output from the imaging unit 105 are sequentially displayed on a display device via the A/D conversion unit 106, RAM 103, image processing unit 107, and display unit 109. During the live view display, the user can adjust the composition in preparation for actual shooting, which assumes that the image data will be recorded on a recording medium. During the live view display, the user can also prepare for actual shooting by changing exposure conditions such as aperture and shutter speed, and image processing conditions for changing image quality such as brightness, gradation, and color.

Figure 2 is a diagram illustrating an example of the appearance of an imaging device. With reference to Figure 2, the appearance of a smartphone as the imaging device 100 will be described. Figure 2 (A) shows a front view of the imaging device 100 (smartphone), and Figure 2 (B) shows a rear view of the imaging device 100.

The imaging device 100 includes a display unit 109 and a camera unit 201. The camera unit 201 includes the optical system 104, imaging unit 105, and A/D conversion unit 106 described in FIG. 1.

(Photographing process according to the first embodiment)
Fig. 3 is a flowchart illustrating an example of an image capturing process according to the first embodiment. The image capturing process of the image capturing device 100 in this embodiment will be described in detail with reference to Fig. 3. Each step of the image capturing process is executed by the control unit 101 or by each block of the image capturing device 100 that receives an instruction from the control unit 101.

4 and 5, a live view display screen and an image group display screen displayed on the display unit 109 in the shooting process shown in Fig. 3 will be described. The live view display screens in Fig. 4(A) to Fig. 4(C), Fig. 5(A), and Fig. 5(B) are display screens before shooting that display live images and allow shot images to be arranged and the live images to be adjusted. Fig. 5(C)
The image group display screen in is a post-shooting display screen for displaying an image group consisting of a captured image obtained by capturing an adjusted live image and an arranged captured image. Fig. 4 is a diagram for explaining the arrangement of the live image and the captured image. Fig. 5 is a diagram for explaining the adjustment of the live image.

The photographing process shown in FIG. 3 is started when the user switches the imaging device 100 to a photographing mode that allows camera photography. For example, the user can switch to the photographing mode by starting a camera function app on a smartphone.

In step S301, the control unit 101 controls the optical system 104, the imaging unit 105, the A/D conversion unit 106, the RAM 103, the image processing unit 107, and the display unit 109 to start live view display.

During the live view display period, the imaging device 100 captures and sequentially acquires live images. The acquired live images are displayed on the display device of the display unit 109. While checking the sequentially displayed live images, the user can adjust the composition and change the exposure conditions and image processing conditions. Note that the processes of steps S302, 303, 304, 305, and 306 described below are processes executed during the live view display period.

In step S302, the control unit 101 displays on the display unit 109 a template of the display position where the image is to be placed. The template is a plurality of areas that are divided in advance. The control unit 101 displays a template that has been set in advance or a template that has been selected by the user.

The template for image display positions will be described with reference to FIG. 4(A). The template for image display positions includes multiple frames (areas) for arranging images. The user can select the display position of an image, for example, by dragging an image selected from unarranged images displayed on the display unit 109 into a specific area of the template.

Figure 4 (A) shows a live view display screen on which a display position template 401 is displayed on the display unit 109. Template 401 includes a total of nine rectangular areas with three rows and three columns. Each rectangular area indicates the position at which an image is to be placed. One image is placed in one rectangular area. In other words, nine images can be placed on template 401.

The display position template is not limited to the example in FIG. 4(A). For example, the display position template may be divided into four areas in which four images can be arranged. The divided areas are not limited to rectangular shapes, and may be of various shapes such as circular or polygonal. The display position template may be one in which the areas differ in size. Furthermore, the display position of an image may be determined by the user freely arranging the image within a specified area without using a template.

In step S303, the control unit 101 determines a display position for displaying a live image of a subject to be photographed from the display position template 401 displayed on the live view display screen. The control unit 101 can determine an area selected by the user from each area included in the template 401 as the display position for the live image.

With reference to Fig. 4(B), the determination of the display position of a live image will be described. With a display position template 401 displayed as shown in Fig. 4(A), the user selects a rectangular area in which the live image is to be displayed by a touch operation. The control unit 101 sets the rectangular area selected by the user as the display position of the live image, and displays the live image in the rectangular area. The live view display screen in Fig. 4(B) is a display screen in which the user selects a rectangular area 402 in the center of the nine rectangular areas.
as the display position of the live image. The control unit 101 determines a rectangular area 402 as the display position of the live image, and displays a live image of a plate and a slice of bread with a fried egg on it in the rectangular area 402. Note that the subject is not limited to food, and may be a person, an animal, a building, a landscape, or the like.

The display position of the live image is not limited to being selected and determined by the user. For example, the display position of the live image may be preset for each template.

In step S304, the control unit 101 places the captured image in the template 401 at the display position based on a user operation. The user operation is, for example, an operation of dragging and dropping an image selected from the unplaced captured images displayed on the display unit 109 to the position (area) in the template 401 where the image is to be placed. The control unit 101 places the captured image in the area to which the user has moved the image selected from the unplaced captured images by the drag-and-drop operation.

The selection and arrangement of captured images will be described with reference to FIG. 4(C). The live view display screen in FIG. 4(C) shows an example in which five captured images are arranged around a rectangular area 402, which is the display position of the live image, by the processing in step S304.

When the display position of the live image is determined in step S303, the control unit 101 displays a list of unarranged captured images 403 in a position separate from the template 401. In the example of FIG. 4(C), the list of unarranged captured images 403 displays five captured images that have not yet been arranged. The images displayed in the list of unarranged captured images 403 are obtained from the recording unit 108. The user can arrange the captured images one by one by selecting the desired captured image while checking the unarranged captured images displayed on the live view display screen and dragging and dropping it into the desired position on the template 401.

In order to create a sense of unity when the images in an image group are displayed in a list, the user selects and arranges from the list of unarranged photographed images 403 images that have similar subjects, composition, brightness, color, etc. to the live image and the arranged photographed image. In the example of FIG. 4(C), the images of chickens and flowers in the list of unarranged photographed images 403 have a different subject from the bread in the live image displayed in the rectangular area 402. For this reason, the user selects and arranges from the list of unarranged photographed images 403 images that have the same subject of bread as the live image, rather than the images of chickens and flowers. In this way, the control unit 101 arranges the photographed images in the template 401 based on the user's operation to arrange the desired image in the desired position.

In addition, the arrangement of captured images is not limited to being determined based on user operations. The control unit 101 may analyze each image in the list of captured images 403 that have not been arranged, and arrange images in the template 401 that have similar subjects, compositions, brightness, colors, etc. to the live image.

In step S305, the control unit 101 adjusts the shooting conditions of the live image based on instructions from the user so that the composition, brightness, color, etc. of the live image are similar to those of the arranged captured image. The shooting conditions are, for example, conditions related to image processing that can be adjusted when capturing a live image, such as composition, exposure conditions, color, and gradation.

A method of adjusting the composition will be described as an example of adjusting the shooting conditions with reference to Fig. 5(A) and Fig. 5(B). Fig. 5(A) and Fig. 5(B) show an example of a live view display screen on which eight captured images are arranged in step S304. A zoom bar 501 is a user interface (UI) for changing the zoom magnification of the live image.
The shooting button 502 is a UI for performing actual shooting by pressing (touch operation).

In the live view display screen of FIG. 5(A), the bread, which is the subject of the live image in rectangular area 402, appears smaller than the bread in the captured images arranged around it. If a photograph is taken in this state, the subject size of the live image and the captured image will differ, and the display of the image group including the captured image of the subject and the captured image will lack unity and will be undesirable.

The live view display screen in FIG. 5(B) shows an example in which the user increases the zoom magnification of the live image to create a sense of unity in the display of the image group. The user can increase the zoom magnification by moving the knob on the zoom bar 501 in the + direction. The size of the subject increases by increasing the zoom magnification. In FIG. 5(A), the subject of the live image is smaller than the subjects of the surrounding captured images, but in FIG. 5(B), by increasing the zoom magnification, the subject becomes approximately the same size as the surrounding captured images. By changing the zoom magnification of the live image to align the subject sizes of the live image and the captured images, the user can display the live image in a unified manner in the list of captured images and the captured images.

Note that, although Fig. 5(A) and Fig. 5(B) show examples of adjusting the composition, this is not limiting. The user may be able to adjust shooting conditions such as the brightness and color of the live image on the live view display screen. In other words, the live view display screen may include a UI for adjusting the brightness, color, and the like of the live image. By adjusting the composition, brightness, color, and the like, or a combination of these, the user can shoot an image that has a more unified look compared to previously shot images.

In addition, the shooting conditions do not necessarily have to be changed by the user, but may be automatically changed or suggested by the imaging device 100. For example, the control unit 101 can automatically change the brightness, color, etc. of the live image so that a sense of unity is created with respect to the captured images. Regarding composition, the control unit 101 can automatically change the zoom magnification of the live image so that the size of the subject is consistent between the live image and the captured image.

In the automatic adjustment of the shooting conditions, the imaging device 100 adjusts the shooting conditions so that the captured image and the live image have the same composition, brightness, and color. The shooting conditions are image processing conditions for changing the composition, exposure conditions, color, gradation, etc.

With reference to Fig. 5(A) and Fig. 5(B), a method of adjusting the composition will be described as an example of adjusting the shooting conditions. Fig. 5(A) and Fig. 5(B) show an example of a live view display screen on which eight captured images have been selected and arranged in step S304. The shooting button 502 is a UI for accepting shooting operations.

In the live view display screen of FIG. 5(A), the slice of bread in the live image displayed in rectangular area 402 is displayed at a smaller size than the slices of bread in the surrounding captured images. If a photograph is taken in this state, the subject size will be different between the live image in rectangular area 402 and the captured images, and the list display of these images may not have a consistent look.

In order to give a sense of uniformity to the size of the subject, the control unit 101 calculates the size of the subject in the captured image, for example, the vertical and horizontal lengths of a rectangle surrounding the subject. For example, the control unit 101 can set the average size of each captured image as the size of the subject in the live image, and adjust the size of the subject in the live image. Note that when the size of the area displaying the captured image and the area displaying the live image differ, the control unit 101 can set the size of the subject in the live image according to the area ratio of each area.

The control unit 101 adjusts the focal length of the optical system 104 so that the live image is displayed at the set subject size. In the example of FIG. 5(A), the subject of the live image displayed in the rectangular area 402 is smaller than the subjects of the surrounding captured images. Therefore, the control unit 101 changes the focal length of the lens to the telephoto side so that the subject size appears larger, as shown on the live view display screen in FIG. 5(B).

The imaging device 100 equipped with multiple optical systems 104 may switch between the optical systems 104 to be used depending on the size of the subject. The imaging device 100 equipped with a fixed focal length lens, such as a smartphone, can adjust the subject size of the live image to that of the captured image by cropping and enlarging the image (electronic zoom) rather than adjusting the focal length. The imaging device 100 can also adjust the subject size of the live image to that of the captured image by combining the focal length change of the optical system 104 and electronic zoom.

By adjusting the subject size, the subject of the live image, which in Figure 5(A) is smaller than the subjects of the surrounding captured images, can be enlarged to the same size as the subjects of the surrounding captured images, as in Figure 5(B). By automatically changing the zoom magnification, the subject size becomes uniform in multiple images, so the list of multiple images displayed on the live view display screen after shooting has a unified look.

The process of unifying the size of the subject in multiple images is not limited to adjusting the size of the subject in the live image based on the captured image. The control unit 101 may adjust the size of the subject in the captured image based on the live image. For example, if the subject in the captured image is smaller than the subject in the live image, the control unit 101 can increase the size of the subject in the captured image by cropping and enlarging the captured image.

The control unit 101 can provide a uniform feeling not only in the subject size but also in the blur (depth of field) by controlling the optical system, processing the image, or by combining these processes. For example, a list display of portrait images will be more preferable if the blur is unified. The depth of field changes depending on the shooting conditions such as the focal length, the subject distance, and the aperture value. The control unit 101 can make the depth of field of the live image the same as that of the captured image by automatically determining the shooting conditions for the live image based on the shooting conditions of the captured image. The control unit 101 can unify the blur of the list display by matching the depth of field of the live image to that of the captured image.

In step S306, the control unit 101 determines whether the adjustment of the shooting conditions in step S305 has been completed. The control unit 101 can determine whether the adjustment of the shooting conditions has been completed, for example, based on whether the user has completed the adjustment of the shooting conditions and pressed the shooting button 502. If the adjustment of the shooting conditions has been completed (step S306: Yes), the process proceeds to step S307. If the adjustment of the shooting conditions has not been completed (step S306: No), the process returns to step S302, and steps S302, S303, S304, and S305 are repeated until the adjustment of the shooting conditions is completed. In this way, the control unit 101 can capture an image that has a sense of unity with the captured images by adjusting the positioning, composition, brightness, and color of the live image based on the user's instructions.

In step S307, when the adjustment of the live image is completed in step S306 and the photographing button 502 is pressed, the control unit 101 performs actual photographing.
Since the image is captured in the state adjusted in step S305, the image has substantially the same composition, brightness, and color as the captured image arranged on the live view display screen.

The control unit 101 associates the captured images of the live images captured by the main shooting with the captured images arranged on the live view display screen as one image group. The control unit 101 also associates the captured images of the live images and the image data of the captured images with arrangement information including the display positions of the live images and the respective display positions of the captured images, and records them in the recording unit 108.

In step S307, the actual photographing is performed when the user presses the photographing button 502, but the actual photographing may also be performed automatically. When the control unit 101 determines in step S305 that the adjustment of the photographing conditions, such as the size of the subject of the live image, has been completed in accordance with the photographed image, it is also possible to automatically perform the actual photographing.

Furthermore, the control unit 101 may notify the user that the adjustment of the shooting conditions has been completed. For example, the control unit 101 displays a message on the display unit 109 notifying the user that the adjustment of the shooting conditions has been completed. The user can check the message displayed on the display unit 109 and perform actual shooting.

In step S308, the control unit 101 acquires the image data and arrangement information of the image group recorded in the recording unit 108, and displays the image data of the image group on the display unit 109 based on the acquired arrangement information.

With reference to FIG. 5(C), an image group display screen that displays image data within an image group will be described. The image group display screen in FIG. 5(C) shows an example in which an image of a subject captured during actual photography is displayed in a rectangular area 503, with previously captured images associated with the captured image displayed around it. The control unit 101 arranges the image data within the image group on the display surface of the display unit 109 based on the arrangement information of the image group. When each image is displayed on the image group display screen, the photographing process is completed.

In the above-described first embodiment, the user can adjust the composition, brightness, color, etc. of the live image while comparing it with the captured image arranged on the live view display screen. This allows the user to capture a live image that has a sense of unity with the captured image. Therefore, the user can easily generate a group of images that have a sense of unity.

In the first embodiment, the adjustment to match the composition of the live image with the captured image displayed on the live view display screen has been described, but the adjustment of the live image is not limited to this. For example, the adjustment of the live image includes adjustment of the exposure conditions, or adjustment of the image processing conditions to change the image quality such as brightness, gradation, and color tone.

Specifically, the adjustment of the exposure conditions may be a process of adjusting the brightness of the live image by adjusting the amount of exposure compensation. Also, the adjustment of the image processing conditions may be a process of adjusting the vividness of the live image by adjusting the amount of saturation emphasis.

In addition, the imaging device 100 may enable users to post image groups that have a uniform look by uploading the image data and arrangement information of the generated image groups to an external server that provides services such as SNS.

<Embodiment 2>
In the second embodiment, captured images to be displayed on a live view display screen are acquired from an external server that provides services such as SNS, etc. The configuration of the imaging device according to the second embodiment is similar to that of the imaging device 100 according to the first embodiment shown in FIG.

The external server receives image groups of captured images and corresponding arrangement information from multiple imaging devices 100 and records them in a recording unit of the external server. When the external server is accessed from a web browser or the like, it can display a list of captured images included in the image group based on the arrangement information.

(Photographing process according to the second embodiment)
Fig. 6 is a flowchart illustrating an example of an image capturing process according to the second embodiment. The image capturing process of the imaging device 100 in this embodiment will be described in detail with reference to Fig. 6. Each step of the image capturing process is executed by the control unit 101, or by each block of the imaging device 100 that receives an instruction from the control unit 101. Note that steps similar to those in the image capturing process according to the first embodiment shown in Fig. 3 are denoted by the same reference numerals, and detailed description thereof will be omitted.

Furthermore, with reference to FIG. 7, a live view display screen and an image group display screen displayed on the display unit 109 in the shooting process shown in FIG. 6 will be described. The live view display screens of FIG. 7(A) and FIG. 7(B) are screens on which captured images are arranged based on image data and arrangement information of the image group acquired from an external server. The image group display screen of FIG. 7(C) is a screen that displays images in an image group after shooting, and has a UI for sending a captured image of a subject and the corresponding arrangement information to an external server.

In step S301, the control unit 101 starts live view display. In step S601, the control unit 101 acquires, via the communication unit 111, captured images of an image group recorded in an external server and corresponding layout information.

In step S602, the control unit 101 arranges the captured images (image data in the image group) acquired in step S601 on the live view display screen based on the arrangement information.

The arrangement of captured images will be described with reference to FIG. 7(A). FIG. 7(A) shows an example in which seven captured images obtained from an external server are arranged on a live view display screen based on the corresponding arrangement information. Information on the template of the display position used to arrange the captured images is included in the arrangement information.

Figure 7 (A) shows an example in which seven captured images are displayed in seven locations of a template that includes nine rectangular areas. Of the nine rectangular areas, no captured images are placed in the bottom center and bottom right rectangular areas. In this way, the imaging device 100 can obtain the captured images of the image group recorded in the external server and the corresponding arrangement information, and arrange the captured images on the live view display screen based on the obtained arrangement information. In other words, the imaging device 100 can reproduce the image data in the image group on the live view display screen in the same arrangement as when the image data is viewed by accessing the external server.

The display position of a captured image on the live view display screen may be changed by the user. The user can change the position of a captured image by dragging and dropping the captured image to the desired position. If a captured image is already positioned at the desired position, the image to be moved may be swapped with the image at the desired position.

In step S603, the control unit 101 determines the display position of the live image. For example, the control unit 101 can determine an area selected by the user from a rectangular area in the template in which no captured image is located as the display position of the live image.

The determination of the display position of the live image will be described with reference to FIG. 7(B). In the example of FIG. 7(A), the user selects, by touch operation, one of the nine rectangular areas, the bottom center or bottom right rectangular area in which no captured image is located, as the display position of the live image. FIG. 7(B) shows an example in which the user selects the bottom center rectangular area 701. The control unit 101 displays the live image in the rectangular area 701 selected by the user.

In steps S305 and S306, the control unit 101 changes the shooting conditions of the live image based on the user's instruction, as in the first embodiment, and determines whether the adjustment of the shooting conditions is complete. If the adjustment of the shooting conditions is not complete (step S306: No), the process returns to step S601, and the processes of steps S601, S602, S603, and S305 are repeated until the adjustment of the shooting conditions is complete.

In steps S307 and S308, the control unit 101 performs the actual shooting, as in the first embodiment, and associates the captured images of the live image and the captured images arranged in the template as an image group, and displays them on the image group display screen.

In step S604, the control unit 101 determines, based on a user instruction, whether to send the image data in the image group and the corresponding placement information to an external server.

Referring to FIG. 7(C), the reception of an instruction to send image data and arrangement information of an image group will be described. FIG. 7(C) shows an example of a screen in which a UI for receiving an instruction as to whether or not to send the image data and arrangement information of an image group to an external server has been added to the image group display screen that displays the image group in step S308 of FIG. 6. The user can check the consistency of the composition, brightness, color, etc. within the image group, and determine whether or not to send the image data and arrangement information of the image group to an external server.

When the user wishes to send the image data and arrangement information of the image group to an external server, the user presses (touches) the "Yes" button. When the "Yes" button is operated, the control unit 101 determines that the image group and the corresponding arrangement information will be sent to the external server (step S604: Yes), and the process proceeds to step S605.

If the user does not want to send the image data and arrangement information of the image group to the external server, the user presses (touches) the "No" button. When the "No" button is operated, the control unit 101 determines that the image group and the corresponding arrangement information will not be sent to the external server (step S604: No), and the shooting process shown in FIG. 6 ends.

In step S605, the control unit 101 transmits the image data of the image group and the corresponding arrangement information to the external server via the communication unit 111, and the shooting process shown in FIG. 6 ends.

In the above-described second embodiment, the external server can generate a screen that displays the image data in the image group in the layout at the time of shooting, based on the image data of the image group and the corresponding layout information received from the imaging device 100. As a result, when the external server is accessed from a web browser or the like to view the image group, the image data can be viewed in the same layout as when the images were shot by the imaging device 100.

<Embodiment 3>
On the live view display screen, a captured image to be compared with a live image is selected by a user operation from the list of unarranged captured images 403. In the third embodiment, in order to reduce the user's effort in selecting and arranging images having the same characteristics from among captured images, the list of unarranged captured images 403 displays related captured images acquired based on the characteristics of a representative image selected by the user as selection candidates. The representative image is an image set as a representative image among captured images with the same tag. The captured images are classified by adding tags based on the characteristics of the images.

When the user selects a representative image, candidates for captured images are narrowed down based on the characteristics of the representative image. This reduces the user's effort in selecting captured images. The configuration of the imaging device according to the third embodiment is the same as the imaging device 100 according to the first embodiment shown in FIG. 1. Below, the processing that differs from the first embodiment will be described.

(Tag addition process)
The control unit 101 adds tags to captured images in advance and determines a representative image for each tag. First, the tag addition process for adding tags to captured images will be described.

When a new captured image is added to the recording unit 108, the control unit 101 adds a tag to the added target image. The control unit 101 extracts image features from the captured image added to the recording unit 108. Image features include, for example, the image composition, color, subject, etc.

In extracting composition features, the control unit 101 detects the contour of the subject from the captured image to be extracted. Based on the detected contour, the control unit 101 determines which of the predefined composition categories the composition falls into. Predefined composition categories include, for example, the Japanese flag composition, the rule of thirds composition, and the diagonal composition.

The control unit 101 can extract the compositional features by determining the compositional pattern based on the contours of the subject. Note that the control unit 101 may extract the compositional features by any method as long as the method is capable of extracting the compositional features. For example, the control unit 101 may extract the compositional features based on the position, size, etc. of the subject. The control unit 101 may also determine the composition by cutting out the periphery of the subject.

When extracting color characteristics, the control unit 101 extracts a representative color from the captured image of the extraction target. The representative color is the most frequently occurring pixel value in the captured image of the target. The control unit 101 may also extract multiple representative colors including the second most frequently occurring pixel value, the third most frequently occurring pixel value, and so on. This allows the control unit 101 to extract color characteristics.

The control unit 101 may extract color characteristics by any method as long as the method is capable of extracting color characteristics. For example, the control unit 101 may extract color characteristics by weighting the representative colors according to their appearance frequencies.

When extracting subject characteristics, the control unit 101 uses image recognition means to determine which of the predefined subject classifications the subject in the image belongs to, for the captured image to be extracted. Predefined subject classifications include, for example, faces, animals, food, landscapes, etc.

The control unit 101 can extract the characteristics of the subject by using an image recognition technique. The control unit 101 may extract the characteristics of the subject by any method as long as the method can extract the characteristics of the subject. For example, the control unit 101 may extract the characteristics of the subject based on the shape, color, etc. of the subject.

The control unit 101 adds image feature information extracted from the captured image to the captured image as a tag. The tag is added, for example, by adding the features of the target image to the target image data as metadata.

Figure 8 is a diagram showing an example of tags (metadata) that indicate characteristic information of an image. Metadata is data that has been added to captured image data. In the example of Figure 8, the tag for the composition of the target image is "Japanese flag", the tag for the color is "#E3E9DF, #DA4011", and the tag for the subject is "food".

(Representative image setting process)
The control unit 101 sets a representative image for the captured images to which tags have been added. The representative image setting process will be described below.

The control unit 101 sets a representative image for a group of images that have the same tag value as a newly captured image that has been added to the recording unit 108, for example. A representative image may be set for a group of images that have one tag value that is the same, or may be set for a group of images that have multiple tags that have the same values. Note that the tag values are not limited to being the same; for example, a group of images whose color characteristics are color codes of similar colors may be treated as a group of images with the same tag value.

The representative image may be selected by any method as long as it is possible to select one image that is representative of a group of images with the same tag value. For example, the control unit 101 may select a representative image based on the shooting date and time of each image included in the group of images with the same tag value. Specifically, the control unit 101 may select the image with the most recent shooting date and time as the representative image from among the images included in the group of images with the same tag value.

The control unit 101 can also set a representative image for a combination of multiple tags. For example, for captured images having the tag information shown in FIG. 8, a representative image is set for a group of images whose composition is "Japanese flag" and whose subject is "food." By combining multiple tags, the control unit 101 can set a representative image for a group of images with more specific conditions.

Information about the representative image is added to the image data as metadata, similar to tag information. Information about the representative image includes the type of tag and the value of that tag that are common to the group of images that belong to the representative image.

The representative image may be set when a new captured image is added to the recording unit 108, or when a predetermined number of images are added. In addition, if a representative image has already been set for a group of images with the same tag value, the control unit 101 may not execute the process of setting the representative image.

(Photographing process according to the third embodiment)
FIG. 9 is a flowchart illustrating the photographing process according to the third embodiment. FIG. 10 is a flowchart illustrating the photographed image arrangement process according to the third embodiment. The photographed image arrangement process is a subflow of the photographing process. The photographing process of the imaging device 100 in this embodiment will be described with reference to FIGS. 9 and 10. The photographing process shown in FIG. 9 is similar to the photographing process shown in FIG. 3 with respect to the processes other than step S901. The same processes as those in FIG. 3 are denoted by the same reference numerals and will not be described. The photographed image arrangement process shown in FIG. 10 illustrates the detailed process of step S901.

In step S1001, the control unit 101 accepts a selection operation of a representative image from the user. With reference to FIG. 11, the UI for accepting the selection of a representative image from the user will be described. FIG. 11(A) shows a state in which multiple representative images are displayed in the display area 1102 of the display unit 109 as selection candidates. In the example of FIG. 11(A), the display area 1102 displays five representative images 1101.

The control unit 101 accepts the selection of a representative image by a user operation on the display area 1102. For example, the user can select a desired representative image by a touch operation or a drag-and-drop operation. The control unit 101 places the selected representative image in any rectangular area of the display position template 401. The placement position of the selected representative image may be a position specified by the user, for example, the rectangular area to which the user has moved the representative image by a drag-and-drop operation.

If the desired representative image is not displayed in the display area 1102, the user can switch between candidate representative images displayed in the display area 1102. For example, the control unit 101 accepts an instruction to display another representative image by the user's sliding operation on the display area 1102. When the control unit 101 accepts an instruction from the user, it switches the representative image displayed in the display area 1102 to the other representative image. In this way, even if the desired representative image is not displayed, the user can display the desired representative image by switching the display of the display area 1102.

The control unit 101 may switch the display in the display area 1102 by accepting an operation to specify the tag type (composition, color, subject) of the image prioritized by the user. In this case, the control unit 101 displays a UI for accepting the tag type specification on the display unit 109. When the user specifies a tag type, the control unit 101 can display a representative image for the specified tag type in the display area 1102. This allows the control unit 101 to present to the user captured images that match the tags (image features) prioritized by the user in step S1004.

In step S1002, the control unit 101 determines whether or not the user has selected a representative image. If a representative image has been selected (step S1002: Yes), the process proceeds to step S1003. If a representative image has not been selected (step S1002: No), the process proceeds to step S1005. If a representative image has not been selected, for example, this occurs when the user has not performed an operation to select a representative image for a predetermined period of time.

When a representative image is selected, the selected representative image is placed in rectangular area 1105, for example, as shown in FIG. 11(B). When the user performs a drag-and-drop operation to move the representative image to one of the rectangular areas of template 401, control unit 101 places the selected representative image in the rectangular area specified by the user.

In step S1003, the control unit 101 acquires related photographed images from the recording unit 108 based on the representative image selected by the user. The related photographed images are images related to the representative image selected by the user, and are images presented to the user as candidates for photographed images to be placed in the template 401. The related photographed images can be photographed images with the same tag value as the representative image.

When acquiring related photographed images, the control unit 101 may accept an operation to specify the tag type (composition, color, subject) of the image prioritized by the user. In this case, the control unit 101 displays a UI for accepting tag type specification on the display unit 109. The control unit 101 preferentially acquires related photographed images having the same specified tag value. If the user selects multiple tags, the control unit 101 acquires related photographed images having the same selected tag values. This allows the control unit 101 to preferentially acquire related photographed images according to the tag type prioritized by the user from the recording unit 108.

In step S1004, the control unit 101 displays the related captured images acquired in step S1003 on the display unit 109 as candidates for captured images to be placed on the template 401.

Figure 11 (B) shows an example of the display on the display unit 109 that has changed from the state shown in Figure 11 (A) as a result of the processing in steps S1002 to S1004. In the example of Figure 11 (B), the representative image selected in step S1002 has been placed in the center left area, which is one of the rectangles in the template 401, by the user's drag-and-drop operation.

The control unit 101 displays the related photographed images 1103 acquired in step S1003 in the display area 1104 as candidates for photographed images to be placed in the template 401. In the example of FIG. 11(B), the control unit 101 displays five related photographed images 1103 as candidates. Note that the control unit 101 excludes related photographed images that have already been placed in the rectangular area of the template 401 from the candidates for the display area 1104, and displays another related photographed image that has not yet been placed in the display area 1104.

If the desired related photographed image is not displayed in the display area 1104, the user can switch the related photographed image displayed in the display area 1104. For example, the control unit 101 accepts an instruction to display another related photographed image by the user's sliding operation on the display area 1104. When the control unit 101 accepts an instruction from the user, it switches the related photographed image displayed in the display area 1104 to another related photographed image. In this way, even if the desired related photographed image is not displayed, the user can display the desired related photographed image by switching the display of the display area 1104.

If the number of related photographed images acquired in step S1003 is less than the number that can be displayed in the display area 1104, the control unit 101 may additionally display other photographed images. The other photographed images to be added are, for example, photographed images having the same value in any of the tags of the representative image, or photographed images having a closer shooting date/time or shooting location. This allows the user to easily select other photographed images even if the number of related photographed images is less than the number that can be displayed in the display area 1104. After the processing of step S1004 is completed, the processing proceeds to step S1006.

In step S1005, since the user has not selected a representative image, the control unit 101 displays the captured images as candidates for the captured images to be placed in the template 401. The candidate captured images in step S1005 may be displayed in any order of priority. For example, the control unit 101 may display the captured images in order of the shooting date and time. Note that the control unit 101 excludes the captured images that have already been placed in the rectangular area of the template 401 from the candidate display, and displays the unplaced captured images in the display area 1104.

In step S1006, the control unit 101 accepts a user instruction to place the (related) captured image displayed in the display area 1104 in steps S1004 and S1005 in the template 401. The process of placing the selected (related) captured image in the template 401 is the same as in step S304 in the first embodiment.

Each time one (related) photographed image is arranged on the template 401, the control unit 101 removes the (related) photographed image arranged on the template 401 from the candidate display in the display area 1104. When one (related) photographed image is arranged on the template 401, the control unit 101 displays an additional (related) photographed image in the display area 1104.

In step S1007, the control unit 101 determines whether or not the user has instructed to end the arrangement of the captured images. If an end instruction has been given (step S1007: Yes), the captured image arrangement process shown in FIG. 10 ends and proceeds to step S305 in FIG. 9. If an end instruction has not been given (step S1007: No), the process returns to step S1001.

Note that if a representative image has already been selected and placed in the template 401, the control unit 101 may return to step S1003 instead of step S1001 to continue processing.

In the third embodiment, the control unit 101 displays candidates for captured images to be placed on the template 401 based on the representative image selected by the user. This enables the imaging device 100 to reduce the user's effort in selecting captured images to compare with the live image.

In step S1006, the control unit 101 may switch, in response to a user instruction, between displaying captured images related to the representative image or displaying captured images under different conditions as candidates to be displayed in the display area 1104. Captured images under different conditions include, for example, images captured within a certain period of time, images captured at a specified location, and all captured images. This allows the control unit 101 to present captured images under different conditions to the user even if the captured image desired by the user is not displayed as a candidate.

Furthermore, when the control unit 101 places the related photographed image selected by the user in step S1006 on the template 401, the control unit 101 may update the candidates for related photographed images to be displayed in the display area 1104. For example, the control unit 101 finds tags whose values are common between the tag of the representative image selected by the user in step S1002 and the tag of the related photographed image selected by the user in step S1006. The control unit 101 preferentially displays photographed images having the same values as these common tags as new related photographed images in the display area 1104. By updating the candidates to be displayed in the display area 1104 based on the related photographed image selected by the user, the control unit 101 can easily display the related photographed image desired by the user as a candidate.

Furthermore, related captured images are not limited to captured images with the same tag value, but may be captured images with tag values in a similar range. The similar range of tag values can be determined in advance. For example, in a composition tag, a three-division (vertical) composition and a two-division (vertical) composition can be defined as being in a similar range. In a color tag, color codes of similar colors can be defined as being in a similar range.

<Embodiment 4>
In the third embodiment, in order to reduce the user's effort in selecting a captured image to compare with a live image, the relevant captured images related to the representative image selected by the user are displayed as candidates for the captured images on the display unit 109. In contrast, in the fourth embodiment, the candidates for the captured images are acquired from the recording unit 108 based on the acquisition conditions for the captured images selected (designated) by the user, and are displayed as candidates on the display unit 109.

In the fourth embodiment, the user specifies acquisition conditions for candidate captured images, and the candidates for captured images are narrowed down based on image features (tags) related to the acquisition conditions. Since candidates according to the specified acquisition conditions are displayed, the user can reduce the effort required to select captured images. The configuration of the imaging device according to the fourth embodiment is the same as that of the imaging device 100 according to the first embodiment shown in FIG. 1. Furthermore, in the fourth embodiment, tag addition processing is executed in the same manner as in the third embodiment. A description of the processing similar to that in the third embodiment will be omitted.

(Photographing process according to the fourth embodiment)
The photographing process of the imaging device 100 according to the fourth embodiment will be described with reference to Fig. 9 and Fig. 12. The photographing process of the imaging device 100 according to the fourth embodiment is similar to the photographing process according to the third embodiment except for step S901 in Fig. 9.

Figure 12 is a flowchart illustrating the captured image arrangement process according to the fourth embodiment. The captured image arrangement process is a subflow of the capture process. The captured image arrangement process shown in Figure 12 illustrates the detailed process of step S901. Among the steps in Figure 12, the same processes as those in Figure 10 are denoted by the same reference numerals and the description thereof is omitted.

In step S1201, the control unit 101 accepts an operation from the user to specify acquisition conditions for a captured image. The acquisition conditions for a captured image include, for example, image composition conditions, image color conditions, and image subject conditions.

Referring to FIG. 13, the UI displayed on the display unit 109 for accepting the user's designation of acquisition conditions for captured images will be described. FIG. 13(A) shows a UI for allowing the user to designate each item of the acquisition conditions for captured images. A selection box 1301 is a UI for designating composition conditions. In response to the user's tap operation on the selection box 1301, the control unit 101 displays the selection items for the composition conditions shown in FIG. 13(B). The control unit 101 accepts the selection of the composition conditions through the user's tap operation.

The select box 1302 is a UI for specifying color conditions. In response to a tap operation by the user, the control unit 101 displays the color condition selection items shown in FIG. 13(C). The control unit 101 accepts the selection of the color condition by the user's tap operation.

The selection box 1303 is a UI for specifying the conditions of the subject. In response to a tap operation by the user, the control unit 101 displays the selection items of the conditions of the subject shown in FIG. 13(D). The control unit 101 accepts the selection of the conditions of the subject by the tap operation by the user.

In this way, the control unit 101 can receive instructions from the user specifying the acquisition conditions for the captured image. Note that the acquisition conditions for the captured image, the UI for specifying the acquisition conditions, and the operation method by the user are not limited to the above examples. Instructions for the acquisition conditions for the captured image may be received from the user in any manner.

For example, the composition condition in FIG. 13(B) may be selected from an illustration representing the composition, rather than the name of the composition. Also, the color condition in FIG. 13(C) may be selected from a color sample, rather than specifying RGB. Also, the subject condition in FIG. 13(D) may be selected from an illustration representing the subject, rather than the name of the subject.

In step S1202, the control unit 101 determines whether or not the user has specified the acquisition conditions for the captured image. If the acquisition conditions for the captured image have been specified (step S1202: Yes), the process proceeds to step S1203. If the acquisition conditions for the captured image have not been specified (step S1202: No), the process proceeds to step S1005. If the acquisition conditions for the captured image have not been specified, for example, the user has not performed an operation to specify the acquisition conditions for the captured image for a predetermined period of time.

In step S1203, the control unit 101 acquires related captured images from the recording unit 108 based on the captured image acquisition conditions specified by the user. The control unit 101 acquires, as related captured images, captured images to which a tag with the same value as the captured image acquisition conditions specified by the user is added. The processing from step S1004 to step S1007 is the same as that in FIG. 10.

In the fourth embodiment, the control unit 101 displays candidates for captured images to be placed on the template 401 based on the conditions for acquiring the captured images specified by the user. This enables the imaging device 100 to reduce the user's effort in selecting captured images to compare with the live image.

<Embodiment 5>
In the third and fourth embodiments, the representative image selected by the user and the related captured images related to the acquisition conditions of the captured images specified by the user are displayed as candidates for captured images on the display unit 109. In contrast, in the fifth embodiment, in order to reduce the user's effort in selecting captured images to compare with a live image, candidates for captured images are acquired from the recording unit 108 based on the characteristics of the live image and displayed as candidates on the display unit 109.

Since candidates for captured images are narrowed down based on the characteristics of the live image, the user can reduce the effort required to select captured images similar to the live image. The configuration of the imaging device according to the fifth embodiment is similar to that of the imaging device 100 according to the first embodiment shown in FIG. 1. Furthermore, in the fifth embodiment, tag addition processing is executed in the same manner as in the third embodiment. A description of the processing similar to that in the third embodiment will be omitted.

(Photographing process according to the fifth embodiment)
The photographing process of the imaging device 100 in this embodiment will be described with reference to Fig. 9 and Fig. 14. The photographing process of the imaging device 100 according to the fifth embodiment is similar to the photographing process according to the third embodiment except for step S901 in Fig. 9. Fig. 14 is a flow chart illustrating the photographed image arrangement process according to the fifth embodiment. The photographed image arrangement process is a subflow of the photographing process. The photographed image arrangement process shown in Fig. 14 illustrates the detailed process of step S901.

In step S1401, the control unit 101 receives an instruction from the user to update the candidate display of captured images. The display unit 109 displays a UI such as an update button that receives an instruction from the user to update the candidate display of captured images. The control unit 101 receives an instruction to update the candidate display of captured images by the user tapping the update button.

In step S1402, the control unit 101 determines whether or not there has been an instruction from the user to update the candidate display of captured images. If there has been an instruction to update from the user (step S1402: Yes), the process proceeds to step S1403. If there has not been an instruction to update from the user (step S1402: No), the process proceeds to step S1005.

In step S1403, the control unit 101 extracts features of the live image. The method of feature extraction may be the same as the method of feature extraction in the tag addition process described in embodiment 3. The control unit 101 can obtain information on features related to the composition, color, subject, etc. of the live image using a method similar to feature extraction in the tag addition process.

Note that the live image changes over time, but the control unit 101 can analyze the image characteristics using the live image at the time when the user issues an instruction to update the candidate display of captured images in step S1402.

In step S1404, the control unit 101 acquires related captured images from the recording unit 108 based on the features of the live image. The control unit 101 acquires captured images to which a tag with the same value as the feature of the live image extracted in step S1403 is added as the related captured images.

In the fifth embodiment, the control unit 101 displays candidates for captured images to be placed on the template 401 based on the live image. This allows the imaging device 100 to reduce the user's effort in selecting captured images to compare with the live image.

In the description of FIG. 14, the control unit 101 uses the live image at the timing when the user instructs the update of the candidate display of captured images for feature extraction, but this is not limited to this. The control unit 101 may use the live image at the timing when a specified condition is satisfied for feature extraction. The specified condition is satisfied, for example, when the amount of change in the live image over time is equal to or less than a threshold value. This enables the imaging device 100 to reduce the effort required by the user to instruct the update of the candidate display of captured images. In addition, it becomes easier to extract features of the live image at the timing when the user decides on the composition to be captured.

<Embodiment 6>
In the third embodiment, the related photographed images related to the representative image selected by the user are displayed as candidates for the photographed images on the display unit 109. In contrast, in the sixth embodiment, in order to further reduce the user's effort in arranging the photographed images, the related photographed images related to the representative image selected by the user are automatically arranged in the area of the template 401.

In the sixth embodiment, related captured images are automatically arranged in the areas within the template 401, reducing the effort required for the user to select and arrange them. The configuration of the imaging device according to the sixth embodiment is the same as that of the imaging device 100 according to the first embodiment shown in FIG. 1. Also, in the sixth embodiment, tag addition processing and representative image setting processing are executed in the same manner as in the third embodiment. A description of the processing similar to that in the third embodiment will be omitted.

(Photographing process according to the sixth embodiment)
The photographing process of the imaging device 100 in this embodiment will be described with reference to Figs. 9 and 15. The photographing process of the imaging device 100 according to the sixth embodiment is similar to the photographing process according to the third embodiment except for step S901 in Fig. 9. Fig. 15 is a flow chart illustrating the photographed image arrangement process according to the sixth embodiment. The photographed image arrangement process is a subflow of the photographing process. The photographed image arrangement process shown in Fig. 15 illustrates the detailed process of step S901. Among the steps in Fig. 15, the same processes as those in Fig. 10 are denoted by the same reference numerals and will not be described.

In step S1501, the control unit 101 arranges the related photographed images acquired in step S1003 in each area in the template 401. Specifically, the control unit 101 acquires a number of related photographed images that can fill the empty areas in the template 401, and arranges them in the empty areas of the template 401.

For example, as in the example screen of FIG. 4(C), if there are three areas where no captured images are placed, the control unit 101 obtains three related captured images from the recording unit 108 and places them in the free areas.

The control unit 101 controls so that a live image that has been placed by the user is not automatically replaced with a related captured image. The control unit 101 places the related captured image in an area of the template 401 where there is no live image placed by the user.

The control unit 101 may also perform control so that a photographed image that has been placed by the user is not automatically replaced with a related photographed image. The control unit 101 manages a flag for determining whether the placed photographed image was placed by the user or automatically placed by the control unit 101 through the processing of step S1501. For example, the control unit 101 adds a flag of 0 to the placed photographed image if it was placed by the user's instruction in step S1006, and a flag of 1 if it was placed by the control unit 101 in step S1501. By referring to the value of the flag, the control unit 101 can perform control so that a photographed image placed by the user (flag = 0) is not replaced with a related photographed image.

The control unit 101 may also change the display form of the arranged captured image depending on whether the arranged captured image was arranged by the user or arranged by the control unit 101 in step S1501. By changing the display form, the user can easily distinguish whether the arranged captured image was arranged by the user or arranged automatically by the imaging device 100. The control unit 101 can change the display form, for example, by the color of the frame of the rectangular area of the template 401.

Also, if the number of related captured images is less than the number of images that can fill each area in the template 401, the control unit 101 may arrange only the related captured images acquired in step S1003. In this case, the control unit 101 may arrange the related captured images by prioritizing the areas that are closer to the live image. By arranging the related captured images from the areas that are closer to the live image, it becomes easier for the user to compare the live image and the captured images.

In the sixth embodiment, the control unit 101 automatically arranges the related captured images acquired based on the representative image selected by the user in the template 401. This enables the imaging device 100 to reduce the user's effort in selecting and arranging captured images to be compared with the live image.

The related photographed image that the control unit 101 placed in the area of the template 401 in step S1501 may be replaced with another photographed image by a user operation in step S1006. This allows the user to replace the related photographed image placed by the control unit 101 with a photographed image of the user's choice if the related photographed image placed by the control unit 101 is not the desired image.

<Embodiment 7>
When displaying a list of images, arranging images having the same characteristic can give the list of images a sense of unity. In addition, the list of images may be given a sense of unity by regularly arranging images having a first characteristic and images having a second characteristic.

Embodiment 6 is an embodiment in which related captured images having the same characteristics as a representative image selected by the user are automatically arranged in an area within template 401. In contrast, embodiment 7 is an embodiment in which multiple groups of captured images having different characteristics are automatically and regularly arranged.

In the seventh embodiment, a plurality of groups of captured images having different characteristics can be automatically arranged in a regular manner. This allows the user to realize a list display of various variations with a simple operation. The configuration of the imaging device according to the seventh embodiment is the same as that of the imaging device 100 according to the first embodiment shown in FIG. 1. Moreover, the same reference numerals are used for the same processes as those in the sixth embodiment, and detailed description thereof will be omitted.

(Photographing process according to the seventh embodiment)
9 and 15, the photographing process of the image capturing apparatus 100 in this embodiment will be described. Differences from the photographing process in the sixth embodiment will be specifically described.

In S302 of FIG. 9, the control unit 101 accepts a user instruction to select a template that allows the user to specify the display position of a captured image of a group having a first characteristic and the display position of a captured image of a group having a second characteristic. Hereinafter, the group having the first characteristic will be referred to as the first group, and the group having the second characteristic will be referred to as the second group.

An example of a template for the display position of a captured image will be described with reference to FIG. 16. FIGS. 16(A) and 16(B) are examples of templates for the display position selectable by the user. A number of areas included in the template are divided into a number of groups and arranged in a regular pattern.

In Figures 16(A) and 16(B), rectangular area 1601 (which also includes a filled rectangular area) is a rectangular area indicating the display position of template 401 in which the first group of images are arranged. Also, rectangular area 1602 (which also includes a shaded rectangular area) is a rectangular area indicating the display position of template 401 in which the second group of images are arranged.

In FIG. 16(A), rectangular regions 1601 and 1602 are arranged in a regular checkerboard pattern. In FIG. 16(B), rectangular regions 1601 and 1602 are arranged in a regular vertical stripe pattern.

The control unit 101 displays a template of the display position as shown in FIG. 16 on the display unit 109 and presents it to the user. The control unit 101 accepts the selection of the template from the user. The control unit 101 may make the display form of the rectangular area 1601, which is the display position of the first group of images, different from the display form of the rectangular area 1602, which is the display position of the second group of images. This allows the user to easily determine which group each rectangular area belongs to.

In step S1001 of FIG. 15, the control unit 101 accepts the selection of a representative image by the user. The user can move the desired representative image to one of the rectangular areas of the template 401, for example, by a drag-and-drop operation. The control unit 101 detects the user's operation and places the representative image selected by the user in the area specified by the drag-and-drop operation. This allows the control unit 101 to accept the selection and placement of the representative image by the user.

In step S1002 of FIG. 15, the control unit 101 determines whether or not the user has selected a representative image. If a representative image has been selected (step S1002: Yes), the control unit 101 records in the recording unit 108 which feature group the rectangular area in which the representative image is located belongs to, and the process proceeds to step S1003. If a representative image has not been selected (step S1002: No), the process proceeds to step S1005.

In step S1501 of FIG. 15, the control unit 101 automatically places the related captured image acquired in step S1003 in a rectangular area of the same feature group as the feature group of the rectangular area in which the representative image is placed.

An example of a live view display screen on which related captured images are arranged will be described with reference to Fig. 17. Fig. 17(A) shows an example in which a template for the display position shown in Fig. 16(A) is selected in step S302. In step S303, the live image is arranged in the rectangular area 402 in the center of the template 401. In step S1001, the representative image 1701 is arranged in the rectangle in the upper left of the template 401.

In step S1501, the control unit 101 reads from the recording unit 108 which feature group the rectangular area in which the representative image 1701 is placed belongs to. In the example of FIG. 17(A), the rectangular area in which the representative image 1701 is placed belongs to the first group. The control unit 101 places the related captured image acquired in step S1003 in a rectangular area at another display position that belongs to the first group, which is the same feature group as the representative image.

Figure 17 (B) illustrates a state in which the control unit 101 arranges related captured images in rectangular areas at other display positions belonging to the first group, starting from the state in Figure 17 (A). Note that the control unit 101 controls so that the user does not replace the arranged representative image and live image with the related captured image.

FIG. 17C shows an example in which the processes from step S1001 to step S1501 are repeated again from the state of FIG. 17B, and the representative image and related captured images are also arranged in the rectangular areas of the second group. In this way, the control unit 101 can configure a live view display screen in which the image features are unified for each feature group.

In the seventh embodiment, the control unit 101 can easily arrange multiple groups of captured images having different characteristics in the template 401. This allows the imaging device 100 to reduce the user's effort in regularly arranging captured images of multiple characteristic groups for comparison with a live image.

The control unit 101 may determine the conditions for the characteristics of the image to be placed in the rectangular area of the second group based on the characteristics of the image of the first group. The conditions for the image to be placed in the rectangular area of the second group may be, for example, a condition that the subject is smaller in the composition than in the first group, or a condition that the color is compatible with the color of the first group.

The control unit 101 can also select captured images that belong to each feature group based on tag information related to the image features. This allows the imaging device 100 to reduce the user's effort in regularly arranging captured images of multiple feature groups for comparison with a live image.

<Embodiment 8>
In the eighth embodiment, the size of the display area (each area in the template) of the captured image and the live image is adjusted to make it easier to check the details of the live image, such as the focus or the angle of view. The user can check the details of the live image and adjust the composition, brightness, and color of the live image to match the captured image, and then capture the image.

The configuration of the imaging device according to the eighth embodiment includes an acceleration sensor that measures the speed of the imaging device 100 in addition to the configuration of the imaging device 100 according to the first embodiment shown in FIG. 1. Explanations of operations and processes similar to those of the first embodiment will be omitted, and the characteristic processes of the eighth embodiment will be explained with reference to FIGS. 18 and 19.

(Photographing process according to the eighth embodiment)
The photographing process of the imaging device 100 in this embodiment will be described with reference to Fig. 18. The photographing process according to the eighth embodiment is similar to the flowchart shown in Fig. 3 except for step S1801. Therefore, the same processes are denoted by the same reference numerals and will not be described.

In step S1801, the control unit 101 accepts an operation from the user to adjust the shooting conditions so that the arranged captured image and the live image have the same composition, brightness, and color. The control unit 101 also accepts an operation by the user to check the details of the live image, such as the focus and angle of view.

The shooting conditions are image processing conditions for changing the composition, exposure conditions, color, gradation, etc. As an example of adjusting the shooting conditions, a method of adjusting the composition will be described with reference to Figures 5(A) and 5(B).

FIGS. 5(A) and 5(B) show examples of a live view display screen in which eight captured images have been selected and arranged in step S304. In addition, in FIGS. 5(A) and 5(B), a zoom bar 501 is a UI for changing the zoom magnification of the live image, and a capture button 502 is a UI for capturing images by pressing or touching the button.

In the live view display screen of FIG. 5(A), the slice of bread in the live image arranged in the rectangular area 402 appears smaller than the slice of bread in the surrounding captured images. If a photograph is taken in this state, the subject size of the live image and the captured image will differ, resulting in an undesirable display of a list of these images with no consistency.

To match the subject size of the live image to that of the captured image, the user increases the zoom magnification of the live image. In the example live view display screen of FIG. 5(B), in order to increase the subject size of the live image, the user can move the knob of the zoom bar 501 in the + direction to increase the zoom magnification. Through the zoom operation, the subject size of the live image, which was smaller than the surrounding captured images in FIG. 5(A), is enlarged to the same size as the surrounding captured images, as shown in FIG. 5(B).

By changing the zoom magnification, the subject size in multiple images becomes consistent, so the list of multiple images displayed on the live view screen after shooting has a unified look.

With reference to Fig. 19(A) to Fig. 19(C), a method for checking whether the image is in focus will be described as an example of checking the details of a live image. Fig. 19(A) shows an example of a live view display screen in which a live image is placed in the center and eight captured images are placed around it. In Fig. 19(A), the display areas for the live image and the captured images are the same size. Having the display areas for the live image and the captured images the same size is suitable for making adjustments to create a sense of unity.

However, with the size of the multiple divided areas, it is difficult to check whether the live image is in focus, and the user may end up taking an image that is out of focus. To make it easier to check whether the image is in focus, the control unit 101, for example, accepts a pinch-out operation from the user and enlarges the rectangular area 402 that displays the live image, as shown on the live view display screens of Figures 19(B) and 19(C). This makes it easier to check whether the live image is in focus.

After checking the focus, the user pinches in. The control unit 101 accepts the pinch in from the user and returns the size of the rectangular area 402 displaying the live image to its original size, as shown on the live view display screen in Fig. 19(A) . By changing the display size of the live image in response to the user's operation, the user can easily check the focus.

Furthermore, a method for checking the angle of view will be described with reference to Figs. 19(B) and 19(C). For example, when the user moves the imaging device 100 at a speed equal to or greater than a predetermined threshold, the control unit 101 determines that the user is checking the rough angle of view of the live image. The speed of the imaging device 100 can be measured by an acceleration sensor provided in the imaging device 100.

When the control unit 101 determines that the user is checking the angle of view of the live image, it automatically enlarges the rectangular area 402 displaying the live image, as shown in Figs. 19(B) and (C). The user can easily check the angle of view by changing the display size of the live image.

When the speed of the imaging device 100 falls below a predetermined threshold, the control unit 101 determines that the user has finished checking the angle of view. When the control unit 101 determines that the user has finished checking the angle of view, it automatically restores the size of the rectangular area 402 displaying the live image to its original size, as shown in the live view display screen of FIG. 19(A). By changing the size of the rectangular area 402 displaying the live image according to the speed of the imaging device 100, the user can easily check the rough angle of view.

In the eighth embodiment, in step S1801, the control unit 101 adjusts the size of the rectangular area 402 that displays the live image so that the user can check the details of the live image. This allows the user to easily check the details of the live image, such as the focus and the rough angle of view.

When the size of the rectangular area 402 displaying the live image is increased, the number and arrangement of the captured images are the same as those in FIG. 19(A) in the live view display screens of FIG. 19(B) and FIG. 19(C), but are not limited to the same. For example, in the example of FIG. 19(D), the live image is enlarged and some of the captured images are displayed below the live image in a reduced size. In the example of FIG. 19(E), the live image is enlarged and displayed in the center, and the captured images are displayed above and below the live image in a reduced size. In the example of FIG. 19(F), the live image is enlarged and displayed in the center, and the captured images are not displayed. As exemplified in FIG. 19(D) to FIG. 19(F), the number and arrangement of the captured images may be freely changed according to the size of the live image.

<Other embodiments>
The first to eighth embodiments (including modifications) are merely examples, and the present invention includes configurations obtained by appropriately modifying or changing the configurations of the respective embodiments within the scope of the gist of the present invention. In addition, the present invention also includes configurations obtained by appropriately combining the configurations of the respective embodiments.

For example, the first to eighth embodiments can be combined with the second embodiment. In this case, the imaging device 100 displays a live image and a captured image acquired from an external server such as an SNS on the live view display screen during shooting.

Specifically, when the eighth embodiment is combined with the second embodiment, the imaging device 100 displays the captured image acquired from the external server on the display unit 109 together with the live image. The imaging device 100 adjusts the size of the area in which the live image is displayed, allowing the user to check the details of the live image. The user can adjust the composition, brightness, and color of the live image to match the captured image from the external server, and then capture the image.

The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.

100: imaging device, 101: control unit, 105: imaging unit, 108: recording unit, 109: display unit

Claims (18)

a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording, in a recording section, a photographed image of the subject photographed by the photographing means and the photographed image in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image;
an uploading unit for uploading the photographed image of the subject and the photographed image together with the position information recorded in the recording unit to an external server;
The display means displays a live image of the subject captured using the shooting conditions changed by the change means, together with the plurality of captured images. 2. The imaging apparatus according to claim 1, further comprising a determination unit that determines display positions of the live image of the subject and the captured image. 2. The imaging apparatus according to claim 1, wherein the display unit adjusts a live image of the subject in accordance with the changed photographing conditions and displays the adjusted image on the display unit. 4. The imaging apparatus according to claim 1, wherein the photographing conditions include at least one of a condition related to a focal length, a condition related to a subject distance, a condition related to a subject depth, and a condition related to an exposure. 5. The imaging apparatus according to claim 1, wherein the display unit displays the photographed image of the subject and the already-photographed image on the display unit based on the arrangement information. 6. The imaging device according to claim 1, wherein the display position of the photographed image of the subject and the display position of the already-photographed image are areas selected from a plurality of areas that are partitioned in advance. 7. The imaging apparatus according to claim 1, wherein the captured image is an image selected from the images acquired from the recording section. a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording the photographed image of the subject photographed by the photographing means and the photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image,
The display means displays a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images,
The imaging device, wherein the captured image is an image selected from images acquired from an external server. 9. The imaging apparatus according to claim 8, further comprising a transmission unit that transmits, to the external server, layout information including a display position of the photographed image of the subject and a display position of the already-photographed image. 10. The imaging apparatus according to claim 9, wherein the display unit displays the photographed image of the subject and the already-photographed image on the display unit based on the arrangement information acquired from the external server. a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording the photographed image of the subject photographed by the photographing means and the photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image,
The display means includes:
displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images;
An imaging apparatus comprising: a display unit that displays related captured images obtained based on features of a representative image selected by a user as candidates for the captured images. a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording the photographed image of the subject photographed by the photographing means and the photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image,
The display means includes:
displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images;
An imaging apparatus, comprising: an imaging device that displays, on the display unit, related captured images acquired based on acquisition conditions designated by a user, as candidates for the captured images. a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording the photographed image of the subject photographed by the photographing means and the photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image,
The display means includes:
displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images;
An imaging apparatus, comprising: a display unit that displays related captured images obtained based on features of the live image of the subject as candidates for the captured images. The imaging device described in any one of claims 11 to 13, characterized in that the display means displays a template on the display unit divided into a plurality of areas for displaying the live image of the subject and the captured image, and automatically places the captured image selected from the related captured images in an area among the plurality of areas in which the live image of the subject and the captured image are not placed. The regions of the template are divided into groups;
15. The imaging apparatus according to claim 14, wherein the display means automatically arranges the captured image selected from the related captured images acquired for each group in an area of the corresponding group. 1. An imaging device, comprising:
a display means for displaying a live image of a subject and a plurality of captured images on a display unit;
a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject;
An imaging means for imaging the subject;
a recording means for recording the photographed image of the subject photographed by the photographing means and the photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image,
The display means includes:
displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images;
1. An imaging device, comprising: an imaging device which changes a display size of a live image of the subject based on a user operation or a speed of the imaging device. displaying a live image of the subject and a plurality of captured images on a display unit;
changing a photographing condition of the object based on the plurality of photographed images displayed on the display unit together with a live image of the object;
displaying a live image of the object captured using the changed shooting conditions together with the plurality of captured images;
taking an image of the subject;
a step of recording the photographed image of the subject and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image;
a step of uploading the photographed image of the subject and the photographed image together with the arrangement information recorded in the recording unit to an external server. A program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 16.

Images