JP7599864B2 - Simulation device, computer program, and storage medium - Google Patents

Description

The present invention relates to a simulation device that uses a virtual space.

To efficiently photograph events such as athletic meets and weddings, it is effective to do location hunting (hereinafter referred to as location scouting), where you visit the venue in advance and consider shooting conditions such as shooting position and angle of view to obtain the desired images. Location scouting is the process of bringing multiple pieces of filming equipment to the event venue in advance and taking repeated test shots to ensure smooth filming on the day of the event.

On the other hand, simulation technology that uses three-dimensional measurement data to recreate spaces such as event venues is becoming more widespread. For example, Patent Document 1 discloses a simulation technology that uses images of an event venue taken from multiple viewpoints to generate image data of the event venue viewed from any position and any direction. Simulation services that allow you to view an event venue from any position and any direction have also appeared.

JP 2012-113281 A

However, in simulations using the above-mentioned arbitrary viewpoint image synthesis technology, although it was possible to check the state of the venue, it was difficult to consider the position and size of the actual subjects in the shot image. In addition, in order to find the shooting conditions such as the shooting position, direction, and angle of view that would result in the desired shot image, it was necessary to search around the simulation screen using input operations such as the mouse and keyboard, which was time-consuming.

Furthermore, it was not possible to confirm the detailed shooting conditions, such as ISO sensitivity, aperture value, shutter speed, and combinations of shooting equipment (camera, lens, etc.), required to obtain brightness and background blur close to the desired captured image.
An object of the present invention is to provide a simulation device that can quickly obtain appropriate shooting conditions, etc. by using a virtual space.

The simulation device according to the present invention comprises:
an information input means for inputting information on the event to be photographed, photographing information, and a desired photographed image;
a virtual space constructing means for constructing a virtual space that imagines the event using a subject model based on the information input by the information inputting means;
a simulation image generating means for generating a simulation image using the virtual space from an image obtained by photographing the event;
an evaluation means for evaluating a similarity between the simulation image and a desired captured image in terms of at least one of brightness and blur amount;
and a notification means for notifying a user of a combination of photographing equipment required for taking the simulation image having the degree of similarity evaluated by the evaluation means that is equal to or greater than a predetermined value, and of the photographing conditions corresponding to the combination .

The present invention provides a simulation device that can quickly obtain appropriate shooting conditions, etc., using a virtual space.

1 is a block diagram showing a configuration example of a simulation device according to a first embodiment; FIG. 13 is a diagram showing an example of a venue layout for an athletic meet in the first embodiment. FIG. 11 is a diagram showing an example of background information of a venue. FIG. 4 is a diagram illustrating an example of light source information. 11 is a diagram for explaining an example of subject position information in a footrace; FIG. FIG. 11 is a diagram showing an example of a method for specifying the subject direction. FIG. 11 is a diagram illustrating an example of a format of subject position information. FIG. 11 is a diagram illustrating an example of shooting position information. FIG. 4 is a diagram showing an example of shooting information. FIG. 2 is a diagram showing an example of a desired captured image. 13 is a flowchart for constructing a virtual space. FIG. 13 is a diagram showing an example of coordinate transformation of a venue layout. FIG. 1 is a diagram showing an image of a subject model. FIG. 2 is a diagram showing an example of subject model placement in a virtual space. 13 is a flowchart of a simulation image generation process. FIG. 13 is a diagram illustrating an example of a projected image. FIG. 13 is a diagram showing an example of a simulation image in which a background is added to a projected image. FIG. 13 is a diagram showing an example of a simulation image whose brightness has been adjusted. FIG. 13 is a diagram showing an example of a simulation image to which background blur has been added. 13 is a flowchart of a similarity calculation process. FIG. 13 is a diagram showing an example of a similarity calculation graph. 11A and 11B are diagrams illustrating an example of a method for calculating a similarity regarding a face direction. FIG. 11 is a diagram illustrating an example of a two-dimensional filter characteristic for detecting the amount of blur. FIG. 13 is a diagram illustrating an example of a weighting coefficient for similarity calculation. 10 is a flowchart showing an example of a flow for a shooting condition evaluation process of the simulation device. FIG. 13 is a diagram showing an example of shooting information (combination 1). FIG. 13 is a diagram showing an example of shooting information (combination 2). FIG. 13 is a diagram showing an example of shooting information (combination 3). FIG. 13 is a diagram showing an example of shooting information (combination 4). FIG. 13 is a diagram showing an example (full size) of a simulation image to which a background has been added. FIG. 13 is a diagram showing an example of a simulation image (APS-C) to which a background has been added. 13 is a flowchart showing an example of a flow for calculating an optimal equipment combination. FIG. 13 is a diagram showing an example of score calculation for each equipment combination.

Below, preferred embodiments of the present invention will be described using examples with reference to the attached drawings. Note that in each drawing, the same members or elements are given the same reference numbers, and duplicate descriptions will be omitted or simplified.

1 is a block diagram showing an example of the configuration of a simulation device according to embodiment 1. In this embodiment, an explanation will be given assuming an athletic meet at an elementary school or the like as an event, but the present invention can be similarly applied to other events.
In FIG. 1, the simulation device has an information input means 100 for inputting information on an event to be photographed, photographing information, a desired photographed image, and the like.
The system also includes a virtual space construction means 101 for constructing a virtual space that reproduces (assumes) an event situation using a subject model based on information inputted by the information input means 100. The system also includes a simulation image generation means 102 for generating a simulation image using the virtual space from an image obtained by photographing the event venue.

Furthermore, the apparatus includes an evaluation means 103 for evaluating the similarity between the simulation image and a desired captured image, and a notification means 104 for collecting shooting conditions according to the similarity so that the evaluation result by the evaluation means 103 is high, and notifying the user of the shooting conditions according to the similarity.
In this embodiment, the evaluation means 103 is configured to evaluate the similarity of at least one of brightness and the amount of blur (of the background) between the simulation image and the desired captured image.

The control means 105 controls the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, and the notification means 104 based on the information input by the information input means 100. The control means 105 has a built-in CPU as a computer, and executes the operations of the entire device, such as the flowcharts in Figures 11, 15, 20, 25, and 32, based on a computer program stored in memory.

Moreover, the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, the notification means 104, etc. execute their respective functions by the computer of the control means 105 executing a computer program. Note that the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, the notification means 104, etc. may be partly or entirely included in the control means 105, or each function may be configured by a discrete circuit.
Each of the components in FIG. 1 will be described in detail below.

The information input means 100 is an input means for inputting event information, shooting information, desired shooting images, etc. The event information includes at least one of the layout of the venue where the event to be shot is held, background information of the venue, light source information of the venue, event program information related to the event content, event order, etc.

The information input means is capable of inputting at least one of information relating to the brightness of the event venue and a background image of the event venue.
An event refers to an event that brings together a large number of people, such as athletic meets, races, plays, school performances, festivals, live music events, weddings, etc.

The venue layout includes an overhead view of the venue where the event will take place.
Fig. 2 is a diagram showing an example of a venue layout for an athletic meet in Example 1. Reference numeral 200 in Fig. 2 denotes a playground (schoolyard) that serves as the event venue, and is represented in a two-dimensional (X, Y) coordinate space with the upper left corner as the origin (0, 0). Reference numeral 201 in Fig. 2 denotes a track that is used as a running course or boundary line for each event in the athletic meet, 202 denotes a school building, and 203 to 205 denote objects that serve as the background during photography, such as trees planted in the playground (hereinafter referred to as background objects).

Figure 3 is a diagram showing an example of background information for a venue. Background information for a venue is a group of images taken of the venue where the event is held and background objects located in the surrounding area. In the case of the venue layout for an athletic meet shown in Figure 2, background information would be, for example, the images shown in Figures 3(A) and (B) which show a school building 202 located in the schoolyard, or the images shown in Figures 3(C) and (D) which show trees 203, 204, and 205 planted in the surrounding area.

As shown in Figure 3, by capturing background information (background images) from multiple different viewpoints and inputting them, a background image from any viewpoint can be generated using a synthesis process. Images can be captured in advance at the event venue using a highly portable camera such as a smartphone, or, if images related to the event venue are already available on the cloud, they can be downloaded to obtain the images.

It is assumed that the input image has the appropriate brightness and is in focus. There are various known techniques for synthesizing an image from an arbitrary viewpoint based on images from multiple viewpoints, and for capturing multiple images suitable for synthesis without excess or deficiency, so detailed explanations will be omitted here.
The light source information is information about the lighting conditions (for example, brightness and color temperature) of the venue where the event is held.

Fig. 4 is a diagram showing an example of light source information, and a list of EV (Exposure Value) values corresponding to the lighting conditions of an event venue as shown in Fig. 4 is created in advance, and the light source information is selected from that list. In the case of an outdoor event, for example, where the lighting conditions on the day may be affected by the weather, multiple lighting conditions may be selected and a simulation may be performed under each of the conditions.
The brightness information as the light source information may be automatically set by downloading data such as a weather forecast for the date and time of the event. At that time, color temperature information may also be input as light source information according to the weather forecast.

The event program information includes the contents of the event, the order of the event, subject position information indicating the movement information of subjects participating in the event, and the like.
The subject position information includes information regarding at least one of the position of the subject and the orientation of the subject at each time that the event is taking place.
FIG. 5 is a diagram for explaining an example of subject position information in a footrace. An overview of subject position information will be explained using FIG. 5 by taking a footrace at an athletic meet as an example.

In the figure, 501, 502, 503, and 504 represent four students (subjects) participating in the race. Also, as the time progresses from T0, T1, T2, ... TN, the figure shows how each student moves from the start of a track 505 drawn in a playground 500 toward the goal.
Here, the position of each student is expressed by XY coordinates, similar to the layout of the venue in FIG.

FIG. 6 is a diagram showing an example of a method for specifying the orientation of a subject. As shown in FIG. 6, the orientation of a student is expressed by the directions of arrows (D0 to D15) that equally divide 360 degrees.
FIG. 7 is a diagram showing an example of the format of subject position information. As shown in FIG. 7, the subject position information input to the information input means 100 is composed of a set of information on the positions and orientations of a plurality of subjects at each time.

The subject position information can be stored as multiple preset data (registered data) in the information input means 100 according to the event program (in the case of an athletic meet, according to the type of sport, such as a foot race or tug-of-war). This allows the user to select pre-stored subject position information from multiple preset data according to the event program.

The content and order of events will be entered as event program information, but the User Interface (hereafter abbreviated as UI) used for this will not be mentioned here. Pre-quantified data can be input, or the user can enter it via a Graphical User Interface (hereafter abbreviated as GUI). Or it can be downloaded via the internet, etc.

Next, the shooting information will be described. The shooting information includes the shooting position, shooting direction, shooting angle of view, and at least one of the aperture value, shutter speed, ISO sensitivity, and white balance data (color temperature data).
The shooting positions include information about areas at the event venue where photographers or shooting equipment such as drones (hereinafter referred to as cameras) can enter (enter).

FIG. 8 shows an example of the shooting position information. FIG. 8(A) shows an overview of the shooting positions, with P0 to P15 indicated by black circles around a playground 800 being positions where shooting is possible.
Each shooting position is expressed using a list of XY coordinates as shown in Fig. 8B. The shooting positions may be treated as area information rather than point information within the event venue.
The shooting direction is the direction in which the camera takes pictures, and in this embodiment, is expressed by the directions of arrows (D0 to D15) that equally divide 360 degrees, as in FIG.
The photographing angle of view is an index that indicates the photographing range of the lens attached to the camera. For example, it is expressed as the focal length (mm) when converted into a sensor size of 35 mm.

9A to 9C are diagrams showing examples of shooting information. In this embodiment, as shown in FIG. 9A, discrete angles of view from wide angle (16 mm) to super telephoto (800 mm) are expressed in correspondence with F0 to F12.
Aperture value is an index for adjusting the amount of exposure when a camera takes a photograph, and is expressed using the F-number. The smaller the F-number, the brighter the lens (the more exposure), but the smaller the F-number, the narrower the area in front of and behind the subject that is in focus (depth of field).

The range of the minimum and maximum F-numbers varies depending on the lens attached to the camera. In this embodiment, the F-number N is expressed in association with an AV (Aperture Value) value by the following formula 1.
AV=2log ₂ N...(Formula 1)
By using the AV value, it is possible to express discrete aperture values from F1.0 to F32 in correspondence with continuous values from AV0 to AV10, as shown in FIG. 9B.

Shutter speed is an index that indicates the exposure time of a camera that takes a picture. In a general camera, the setting range is from a few seconds to a few thousandths of a second. The slower the shutter speed, the brighter the image can be taken even in a dark place, but the effect of camera shake or subject blur during shooting becomes greater, so the possibility of failing to take a picture increases. In this embodiment, the shutter speed t is expressed in correspondence with a TV (Time Value) value by the following formula 2.
TV=log ₂ (1/t)...(Formula 2)
By using the TV value, it is possible to express discrete shutter speeds from 1 second to 1/8000 seconds in correspondence with continuous values from Tv0 to Tv13, as shown in FIG. 9C.

ISO sensitivity is the sensitivity setting of the sensor mounted on the camera used to take the photo. It is expressed with ISO at the beginning of the number, such as ISO 100 or ISO 20.
In general cameras, the ISO sensitivity setting ranges from several hundred to several tens of thousands. The higher the value, the brighter the image you can take even in a dark place, but the image quality will deteriorate due to the increased effect of noise.

In this embodiment, the ISO sensitivity S is expressed in association with a SV (Sensitivity Value) value by the following formula 3.
SV=log ₂ (S/100)...(Formula 3)
By using the SV value, it is possible to express discrete ISO sensitivities from ISO 100 to ISO 51200 in correspondence with continuous values from SV0 to SV9, as shown in FIG. 9D.

Next, the desired photographed image will be described. The desired photographed image is information on a sample image of an image that the user wants to photograph, and includes image information as a sample, and is stored in association with at least one of the number of subjects, face positions, face sizes, and face orientations.
FIG. 10 is a diagram showing an example of a desired photographed image.

In this embodiment, a large number of sample images such as those shown in Fig. 10A and 10B are prepared as preset data. Then, the number of subjects, face positions, face sizes, face orientations, and background object positions, sizes, and orientations are linked (associated) with each sample image and saved. The saving location may be a cloud on the Internet, etc.
There are various methods for inputting the desired photographed image, but photographed images or illustrations of the same event held in the past are used as input.
The user can download a plurality of sample images saved as preset data, for example via the Internet, and select a desired photographed image from among them.

Information associated with the selected sample image is stored as input information. A plurality of desired captured images may be selected.
It should be noted that the brightness information and the blur information of the face are also linked to each sample image and stored.Furthermore, the color and color temperature of the subject may also be linked to each sample image and stored.
Next, a description will be given of the virtual space construction means 101. The virtual space construction means 101 generates a virtual space in which an event situation is reproduced by a subject model, based on the event information inputted by the information input means 100.

FIG. 11 is a flowchart for constructing a virtual space, FIG. 12 is a diagram showing an example of coordinate transformation of a venue layout, and FIG. 13 is a diagram showing an image of a subject model. The flow of generating a virtual space will be explained using FIGS.
First, in step S1101, the virtual space construction means 101 converts the two-dimensional venue layout input to the information input means 100 into a three-dimensional coordinate space (xyz coordinate system) as shown in FIG.

Next, in step S1102, the virtual space construction means 101 places a subject model in three-dimensional coordinate space based on the event program information input to the information input means 100. An example of a subject model to be placed is shown in FIG. 13. In FIG. 13, reference numeral 1301 denotes a subject model of a student who will be the subject of the sports day, and is represented in three-dimensional coordinate space as a polyhedron (polygon). Reference numeral 1302 in the figure is a front mark that indicates the direction of the student's face. Multiple subject models of FIG. 13 are placed in three-dimensional coordinate space according to the subject position information.

14 is a diagram showing an example of subject model placement in a virtual space, displaying the virtual space near the start line at time T0. The positions and orientations of subject models 1401, 1402, 1403, and 1404 change with time based on subject position information.
Furthermore, the virtual space construction means 101 can make the subject model pose based on the contents of the subject position information, and can also place spectators such as students not participating in the competition and parents in the virtual space.

The method of making a polygonal subject model assume a desired pose is similar to that used in 3D games and the like, and is already widely known, so a detailed explanation will be omitted here.
Next, the simulation image generating means 102 will be described with reference to Fig. 15. The simulation image generating means 102 generates simulation images obtained when photographing is performed under various photographing conditions in the virtual space.

FIG. 15 is a flowchart of the generation of a simulation image.
First, in step S1501, the simulation image generating means 102 determines the shooting conditions (shooting position, shooting direction, angle of view, aperture value, shutter speed, ISO sensitivity, lighting conditions, etc.) when shooting within the three-dimensional coordinate space in which the subject model is placed. The shooting position is selected from P0 to P15 in FIG. 8 input to the information input means 100.

The shooting direction is selected from D0 to D15 in FIG. 6, which are input to the information input means 100. The angle of view is selected from F0 to F12 in FIG. 9(A), which are input to the information input means 100. The aperture value is selected from AV0 to AV10 in FIG. 9(B), which are input to the information input means 100. The shutter speed is selected from TV0 to TV13 in FIG. 9(B), which are input to the information input means 100. The ISO sensitivity is selected from SV5 to AV14 in FIG. 9(D), which are input to the information input means 100.

The illumination conditions are selected from the light source information EV1 to EV14 of FIG.
Next, in step S1502, the simulation image generating means 102 performs a projection process to obtain a two-dimensional image (projected image) obtained when the virtual space is photographed at the photographing position, orientation, and angle of view selected in step S1501. The projection process can be performed by a generally known technique such as perspective projection, and therefore a detailed description thereof will be omitted.

Figure 16 shows an example of a projected image, which is obtained when shooting is performed at time T1 from shooting position P14 in shooting direction D4 with a field angle of F6. An image is obtained in which subject models 1601, 1602, and 1603 representing athletes are arranged on screen 1600. The brightness of the subject models and virtual space in the projected image is generated with appropriate brightness.

Next, in step S1503, the simulation image generating means 102 adds a background image to the projection image generated in step S1502. The background image is generated by synthesizing image data from multiple viewpoints according to the shooting position, direction, and angle of view set in step S1501 based on the background information of the venue input to the information input means 100.
That is, the simulation image generating means is configured to be able to add a background image to the simulation image based on image data of the background of the event venue.

The generation of image data using a plurality of viewpoints is known as an arbitrary viewpoint image synthesis technique and is therefore not described in detail here.
FIG. 17 is a diagram showing an example of a simulation image in which a background is added to a projected image.
Background objects 1701 and 1702 corresponding to the trees 204 and 205 planted in the schoolyard in Fig. 2 are added to the projected image. Since the brightness of the images from multiple viewpoints before synthesis is adjusted to an appropriate brightness in advance, the brightness of the image after synthesis is also appropriate.

Next, in step S1504, the simulation image generating means 102 adjusts the brightness of the projected image to which the background was added in step S1503. The brightness of the image is expressed as appropriate when the entire image looks natural. Images that have been intentionally photographed brightly are expressed as over (N steps). The value of N is expressed as over 1 step and over 2 steps when the brightness (luminance value) is doubled or quadrupled based on the image brightness when it is appropriate.

In addition, images that are intentionally taken dark are expressed as underexposed (M steps). The M value is expressed as underexposed 1 step and underexposed 2 steps when the brightness (luminance value) is 1/2 and 1/4 of the standard brightness of the image.
The brightness (L) of an image is calculated by the following formula 4 when the illumination condition (EV), aperture value (AV), shutter speed (TV), and ISO sensitivity (SV) are given.
L=(EV+SV)-(AV+TV)...(Formula 4)

When L is 0, the captured image has appropriate brightness. In an image with a background added, both the subject model and the background object are output with appropriate brightness, so when L is 0, no brightness adjustment is performed.
When L is a value greater than 0, the captured image will be over L levels brighter. In this case, the luminance value Y(x, y) of each pixel in the image to which the background has been added is converted to a luminance value Y'(x, y) using the following Equation 5.
Y'(x,y)=2 ^L *Y(x,y)...(Formula 5)

When L is a negative value, the captured image will be L steps below the brightness level. In this case, the luminance value Y(x, y) of each pixel in the image to which the background has been added is converted to a luminance value Y'(x, y) using the following Equation 6.
Y'(x,y)=(1/2 ^|L| )*Y(x,y)...(Formula 6)
As described above, the simulation image generating means can adjust the brightness of the simulation image based on the input shooting conditions, and outputs an image with the brightness adjusted in accordance with the shooting conditions as a simulation image.

FIG. 18 is a diagram showing an example of a simulation image whose brightness has been adjusted.
FIG. 18 shows an example of a simulation image in which the brightness of the image is adjusted under the lighting conditions of outdoor cloudy weather (EV12), aperture value F4.0 (AV4), shutter speed 1/500 (TV9), and ISO sensitivity 100 (SV0).
Since the brightness L of the image is −1 (one step under), the brightness (luminance value) of the entire screen is reduced by half compared to the image in FIG.

In this embodiment, a uniform brightness adjustment was performed on all images, but the brightness may be adjusted taking into account the reflectance of each subject. In this case, the reflectance of the subject and background objects may be linked to the subject model and the background information of the venue input to the information input means 100.

In addition, when the sun is the light source, such as outdoors, the irradiation angle may vary depending on the time of day. In such cases, by inputting not only the lighting conditions (EV value) but also the irradiation angle for each time of day as light source information, it is possible to precisely reproduce the shadows of the subject and the brightness of the face.
Regarding the method of adjusting brightness taking into account the reflectance of each subject and the illumination angle of the light source, known techniques such as ray tracing and photon mapping may be used, and details will not be mentioned here.

Next, in step S1505, blur is added to the image whose brightness has been adjusted in step S1504. If the pixel value of a point P(X,Y) on the image before blurring is P(X,Y) and the value after blurring is P'(X,Y), the relationship between P' and P is expressed by the following formula 7 using a point spread function PSF that indicates the optical transfer characteristic of the lens attached to the camera that takes the image.
P'(X,Y)=P(X,Y)*PSF(F,d,D,AV)...(Formula 7)

Here, F is the focal length (photographing angle of view), d is the distance from the photographing position to the main subject (the subject to be focused on), D is the distance from the photographing position to the background object, and AV is the aperture value.
Once the shooting time and shooting position are determined, the distances to the main subject and background objects can be uniquely calculated from the venue layout in Fig. 2 and the event program information in Fig. 5. By performing the calculation of Equation 7 above for all pixels in the simulation image, a simulation image with background blur added can be obtained.

In this manner, in this embodiment, the simulation image generating means can add blur to at least one of the main subject and background image of the simulation image based on the input photographing conditions.
Generally, if the focal length and aperture value of the lens attached to the camera are the same, the greater the difference between the distance to the subject and the distance to the background object, the greater the amount of blur. Also, if the distance to the subject and the distance to the background object are the same, the longer the focal length of the lens attached to the camera and the smaller the aperture value, the greater the amount of blur.

FIG. 19 is a diagram showing an example of a simulation image to which background blur has been added.
19, blur is added to objects 1902 and 1903 running behind a main object 1901, which is the focus position. In terms of the amount of blur, a larger amount of blur is added to object 1903, which is farther away from the main object 1901, than object 1902. Similarly, blur is added to background objects 1904 and 1905 according to their distance from the main object 1901.

Next, a description will be given of the evaluation means 103. The evaluation means 103 evaluates the similarity between the simulation image generated by the simulation image generation means 102 and the desired photographed image input and selected by the information input means 100.
FIG. 20 is a flow chart showing the calculation of the similarity. The evaluation of the similarity by the evaluation means 103 will be described with reference to the flow chart of FIG.
First, in step S2001, the evaluation unit 103 calculates the number of people similarity (Mn) and face position similarity (Mp) for the person area in the simulation image, as well as the face size similarity (Ms), face direction similarity (Md), face brightness similarity (Ml), and blur amount similarity (Mb).

The similarity (Mn) of the number of subjects is defined as the absolute value of m-n, where n is the number of subjects in the simulation image and m is the number of subjects linked to the desired captured image. The similarity is calculated from this difference in the number of subjects.
Fig. 21 shows an example of a graph for calculating similarity, and Fig. 21(A) shows a graph with the difference in the number of people on the horizontal axis and the similarity on the vertical axis. Fig. 21(A) shows a graph in which the larger the difference in the number of people is, the smaller the similarity becomes linearly, but this is just an example and a curve may also be used.

The similarity of face positions (Mp) is calculated by finding the Euclidean distance between the center position (Xc, Yc) of the subject's face in the simulation image and the center position (Xs, Ys) of the face linked to the desired captured image, and then calculating the similarity from this Euclidean distance. Figure 21 (B) shows a graph with the Euclidean distance on the horizontal axis and the similarity on the vertical axis.

The curve shown in the graph of FIG. 21(B) is an example, and other shapes may be used. If the simulation image and the desired captured image contain multiple faces, pairing is performed with a combination that provides the smallest Euclidean distance, and the similarity of each face position is calculated. The average value of the calculated similarities of each face position is set as the similarity of the face position for the entire image (Mp).

The similarity in face size (Ms) is calculated using the face size indicated by the radius from the center of the face. If the face size of the subject in the simulation image is Rc and the face size associated with the desired captured image is Rs, then the absolute value of Rs-Rc is defined as the face size difference Rd. Figure 21 (C) shows a graph with the face size difference Rd on the horizontal axis and the similarity on the vertical axis.

The curve shown in the graph of FIG. 21(C) is an example, and other shapes are acceptable. If the simulation image and the desired captured image contain multiple faces, the similarity in the size of each face is calculated for each paired combination when calculating the similarity in the face positions. The average value of the calculated similarity in the size of each face is set as the face size similarity (Ms) for the entire image.

The similarity (Md) of the face direction is evaluated by considering the face direction of the subject in the simulation image and the face direction associated with the desired captured image as directional vectors in three-dimensional space.
Fig. 22 is a diagram for explaining an example of a method for calculating similarity regarding face orientation. For example, as shown in Fig. 22, an XY plane 900 including the center coordinates of a face of interest is extracted, a vector indicating the face orientation is projected onto the plane, and the direction in which the projected vector faces is determined. The face orientation of the subject in the simulation image is superimposed on the face orientation associated with the desired captured image, and the smaller angle between the two vectors is determined.

The absolute value of the angle difference is between 0° and 360°, but if it exceeds 180°, the smaller angle is found by subtracting the angle from 360°. The smaller angle value of 0° to 180° corresponds to 100% to 0%, which becomes the similarity on the X-Y plane 900. The similarity is similarly found for the X-Z plane 901, and the similarity on the X-Y plane 900 and the similarity on the X-Z plane 901 are multiplied together to obtain the similarity between the orientations of the two faces.

When multiple faces are included in the simulation image and the desired captured image, the similarity of the orientation of each face is calculated for each pair of faces that were paired when the similarity of the face positions was calculated. The average value of the calculated similarities of the orientation of each face is set as the similarity (Md) of the orientation of the face for the entire image.
A graph in which the angle between the vectors of the face direction is plotted on the horizontal axis and the similarity is plotted on the vertical axis is shown in Fig. 21(D). The curve shown in the graph in Fig. 21(D) is just an example, and other shapes may be used.

The similarity in facial brightness (Ml) is calculated from the difference between the average facial brightness Ls of the subject in the simulation image and the average facial brightness Lr associated with the desired captured image. The absolute value of Ls-Lr is defined as the facial brightness difference Ld. When brightness values are expressed in 8 bits, the possible range of Ld is from 0 to 255. Figure 21 (E) shows a graph with the facial brightness difference Ld on the horizontal axis and the similarity on the vertical axis.

It should be noted that the curve shown in the graph of FIG. 21E is merely an example, and a different shape may also be used.
When multiple faces are included in the simulation image and the desired captured image, the brightness similarity of each face is calculated for each pair of faces that were paired when the face position similarity was calculated. The average value of the calculated brightness similarities of each face is set as the brightness similarity (Ml) of the face in the entire image.

The degree of similarity (Mb) of the amount of blur is calculated from the difference between the amount of blur Bs of the subject's face in the simulation image and the amount of blur Br of the face associated with the desired photographed image.
FIG. 23 is a diagram showing an example of a two-dimensional filter characteristic for detecting the amount of blur.
The amount of blur is defined as the inverse of the value obtained by performing a two-dimensional high-frequency extraction BPF filter process with weighting characteristics as shown in Figure 23 on the pixel values P(i, j) of the facial contour, and dividing the extracted high-frequency components by the number of pixels (N) of the facial contour.

For example, it is expressed as in Equation 7 below.
Br=N/ΣP(i,j)*BPF...(Formula 7)
The absolute value of Bs-Br is defined as the difference Bd in the amount of blur between the faces. A graph in which the horizontal axis represents the difference Bd in the amount of blur between the faces and the vertical axis represents the degree of similarity is shown in FIG.

The curve shown in the graph of FIG. 21(F) is an example, and may have a different shape. If the simulation image and the desired captured image contain multiple faces, the similarity of the amount of blur for each face is calculated for the combination paired when calculating the similarity of the face positions. The average value of the calculated similarity of the amount of blur for each face is set as the similarity (Mb) of the amount of blur for the faces in the entire image.

Next, in step S2002, the evaluation means 103 calculates the number similarity (M'n), position similarity (M'p), size similarity (M's), orientation similarity (M'd), brightness similarity (M'l), and blur amount similarity (M'b) for the background objects in the simulation image. The method of calculating the similarity is the same as for calculating the similarity of the person area in step S2001. When there are multiple background objects, the similarity of each background object is calculated as in step S2001, and then the average value is calculated, and this value is used as the similarity of the background objects.

Next, in step S2003, the evaluation unit 103 calculates a similarity Mtotal of the entire image using the similarity of each item of the person area and background area calculated in steps S2001 and S2002 according to the following formula 8.
Mtotal=Kn*Mn+Kp*Mp+Ks*Ms
+Kd*Md+Kl*Ml+Kb*Mb
+K'n*M'n+K'p*M'p+K's*M's
+K'd*M'd+K'l*M'l+K'b*M'b... (Formula 8)

Here, Kn, Kp, Ks, Kd, Kl, and Kb are coefficients for adjusting the weights for the similarity of each item in the person region, and K'n, K'p, K's, K'd, K'l, and K'b are coefficients for adjusting the weights for the similarity of each item in the background region.

Figure 24 shows an example of weighting coefficients for similarity calculation. As in the example of Figure 24, by making the coefficients Kn, Kp, Ks, and Kd for the number of subjects, position, size, and orientation larger than the brightness Kb and the amount of blur Kb, it is possible to perform a similarity evaluation that places more emphasis on the composition. Also, by making the coefficient for the person area larger than the coefficient for the background object, it is possible to perform a similarity evaluation that places more emphasis on people.

By linking and setting these coefficients according to the characteristics of each desired captured image, it becomes possible to perform an appropriate similarity evaluation for each image. Note that in this embodiment, the similarity is calculated focusing on the facial area of the subject, but similarity may also be calculated for the size, position, orientation, and pose of the subject's entire body using a similar method. Furthermore, it is possible to evaluate similarity including color by applying a method similar to that used to calculate brightness similarity to each color component of each pixel.

In this way, by linking face color information, color temperature information, and the like to a desired photographed image and storing it, the similarity of face color and color temperature information may also be included in the similarity evaluation.
That is, the evaluation means may calculate the similarity between the simulation image and the desired captured image for at least two of the following: the number of subjects, the position of the subjects, the size of the subjects, the orientation of the subjects, the brightness of the subjects, the background image, the amount of background blur, and color.

Next, the process of the notification means 104 will be described in detail.
The notification means 104 compares the similarity (Mtotal) calculated by the evaluation means 103 with a preset threshold value (Th), and if Mtotal is greater than Th, notifies the user of the simulation image and the shooting conditions used to generate it. If Mtotal is not greater than Th, the user may be notified that the similarity is low or that no image with a high similarity exists in the simulation image. Notification methods include output to a display device of the user terminal, output by printing, data output to an external medium or a device connected to a network, etc.

Next, the details of the shooting condition evaluation process performed by the control means 105 will be described. The control means 105 sequentially evaluates the simulation images obtained when the virtual space is photographed under various conditions based on the various information input to the information input means 100, such as event program information, shooting information, desired photographed image, venue background information, light source information, etc. Then, it notifies the user of the evaluation results.

FIG. 25 is a flow chart showing an example of a flow for the imaging condition evaluation process of the simulation device, and an example of sequential execution of evaluation will be described with reference to FIG.
Steps S2501 to S2502 are a loop for varying (scanning) various shooting conditions by assigning them in predetermined steps. In Fig. 25, the shooting conditions are time, position, direction, and focal length, and are indicated by T for time, P for position, D for direction, and F for focal length, respectively. In this example, they are assigned in order in a brute force manner, but the order in which the shooting conditions are assigned is not limited. It is also possible to assign only the necessary conditions efficiently using some algorithm.

Step S2501 is a loop for comprehensively evaluating T (time) in the range of T0 to TN, and step S2502 is a loop for comprehensively evaluating P (position) in the range of P0 to PN. Step S2503 is a loop for comprehensively evaluating D (direction) in the range of D0 to DN, and step S2504 is a loop for comprehensively evaluating F (focal length) in the range of F0 to FN. Step S2505 is a loop for comprehensively evaluating AV (aperture value) in the range of AV0 to AVN.

Step S2506 is a loop for comprehensively evaluating TV (shutter speed) in the range of TV0 to TVN, and step S2507 is a loop for comprehensively evaluating SV (ISO sensitivity) in the range of SV0 to SVN.
In step S2508, the virtual space construction means 101 constructs a virtual space under conditions based on the loop variable T. In step S2509, a simulation image is generated when shooting is performed under shooting conditions based on the loop variables P, D, F, AV, TV, and SV.

In step S2510, the evaluation unit 103 calculates an evaluation value between the simulation image generated in step S1709 and the desired captured image.
Steps S2511 and S2512 are processes in which the notification means 104 notifies the user of a simulation image and shooting conditions based on the evaluation result calculated in step S2510.
That is, if the similarity is equal to or greater than a predetermined value, the simulation image and the corresponding photographing conditions are notified to the user.

Steps S2513 to S2519 represent a determination of whether to continue the loop processing for time T, position P, direction D, focal length F, aperture value AV, shutter speed TV, and ISO sensitivity SV, and the processing from steps S2508 to S2512 is repeated until all conditions are covered.

As described above, according to this embodiment, the shooting conditions for obtaining a desired photographic image can be automatically assigned, evaluated, and extracted using simulation, so that the shooting conditions for obtaining a desired photographic image can be extracted more accurately without having to bring a large amount of photographic equipment to the site in advance.
In this way, depending on the desired captured image, the similarity is calculated for at least two of the number of subjects, the position of the subject, the size of the subject, the orientation of the subject, the brightness of the subject, the background image, the amount of background blur, and the color of the subject.

The similarity is evaluated using different weights for the person region and the background region. That is, the evaluation means calculates an evaluation value by weighting and adding at least two calculated similarities, thereby making it possible to extract shooting conditions for more effectively obtaining an image close to a desired shot image.
Therefore, since it is possible to know in advance what position and under what shooting conditions the image should be taken, the efficiency of the image taking work can be improved. Also, since the image taking equipment required to obtain the desired image is known, it is not necessary to bring in unnecessary image taking equipment.

Next, a second embodiment of the present invention will be described.
In the second embodiment, by inputting information about the photographic equipment into the simulation device, it becomes possible to extract in advance the optimum combination of photographic equipment in addition to the photographic conditions for obtaining a desired photographic image.

The basic configuration is the same as in the first embodiment, but the information input means 100 allows additional input of information relating to photographic equipment.
The information about the photographing equipment includes the camera's sensor size (the size of the light receiving surface of the image sensor), shutter speed, ISO sensitivity, lens focal length, aperture value, and the like.

As information on photographic equipment, information on multiple cameras and multiple lenses is input. As information on photographic equipment, the user may input information on cameras and lenses that he or she owns, or information on cameras and lenses that can be rented from a rental service, etc. Alternatively, information on cameras and lenses that the user is considering purchasing may be input. As information on photographic equipment, information on multiple cameras and lenses may be prepared as presets in the information input means in advance, and the information may be selected from the presets.

Depending on the combination of the camera and lens in the input information on the photographing equipment, the possible ranges of the photographing conditions (photographing angle of view, aperture value, shutter speed, ISO sensitivity) described in FIG. 9 of the first embodiment are determined. For example,
Camera 1: Sensor size: Full size, Shutter speed: 1 second to 1/8000 second, ISO 100 to ISO 51200,
Camera 2: Sensor size: APS-C size, Shutter speed: 1 second to 1/4000 second, ISO 100 to ISO 12800,

Lens 1: focal length 300 mm, F value 4.0,
Lens 2: focal length 135 mm, F value 2.0,
When the above information on the photographing equipment is given, photographing conditions are obtained by combining the photographing conditions shown in, for example, FIGS.

Figure 26 is a diagram showing an example of shooting information (combination 1), Figure 27 is a diagram showing an example of shooting information (combination 2), Figure 28 is a diagram showing an example of shooting information (combination 3), and Figure 29 is a diagram showing an example of shooting information (combination 4).
Here, combination 1 is a combination of camera 1 and lens 1, combination 2 is a combination of camera 1 and lens 2, combination 3 is a combination of camera 2 and lens 1, and combination 4 is a combination of camera 2 and lens 2.

The shooting angle of view is determined by the combination of the lens focal length and the camera's sensor size. If the sensor size is full size, the lens focal length is used as is as the focal length (mm) when converted to a 35mm sensor size. If the sensor size is APS-C, the lens focal length is multiplied by 1.6 to convert it to the focal length for a 35mm sensor size.

For example, as shown in Fig. 28, when the sensor size is APS-C, the focal length of the lens 1, 300 mm, is multiplied by 1.6 to obtain 480 mm, which is used as the focal length of the shooting angle of view. Note that the value of 1.6 is just an example, and a magnification appropriate to the sensor and lens is used.
Fig. 30 is a diagram showing an example of a simulation image (full size) to which a background has been added, and Fig. 31 is a diagram showing an example of a simulation image (APS-C) to which a background has been added. The shooting conditions for Fig. 30 and Fig. 31 are the same except for the sensor size. When the same lens (focal length) is used, the subject appears larger with a sensor size of APS-C than with a full size.

The amount of background blur also differs depending on the sensor size. Generally, if the camera sensor size is the same, the longer the focal length of the lens, the more the background will be blurred. Also, if the focal length of the lens is the same, the larger the camera sensor size, the more the background will be blurred. Such differences in background blur due to combinations of sensor size and focal length can be expressed in a simulation image by changing the PSF in Equation 7 for adding background blur, explained in Example 1, according to the combination of camera and lens.

The flow of simulation image generation and evaluation in FIG. 25 described in the first embodiment is performed for the combinations 1 to 4, and the evaluation results calculated for each combination are tallied and compared to extract the optimal combination of equipment for obtaining the desired captured image.
FIG. 32 is a flowchart showing an example of a flow for calculating an optimal equipment combination. A flow for extracting an optimal combination of photographing equipment based on input information on the photographing equipment will be described with reference to FIG.

First, in step S3201, the control unit 105 generates shooting conditions corresponding to the combination of camera and lens from information related to the photographic equipment as shown in FIGS.
Next, in step S3202, the control unit 105 executes a series of processes for generating and evaluating a simulation image, which has been described with reference to FIG. 25 in the first embodiment, for each of the generated combinations.
For example, a case where the flow of FIG. 25 is performed for combination 1 will be described.
Steps S2501 and S2502 form a loop for setting various shooting conditions.

In Fig. 25, the time, position, direction, and focal length are assigned (scanned) as shooting conditions, where time is indicated by T, position by P, direction by D, and focal length by F. In this example, they are assigned in order in a brute force manner, but the order in which the shooting conditions are assigned is not limited to this. It is also possible to assign only the necessary conditions efficiently using some algorithm.
Step S2501 is a loop for comprehensively evaluating T (time) in the range of T0 to TN, step S2502 is a loop for comprehensively evaluating P (position) in the range of P0 to PN, and step S2503 is a loop for comprehensively evaluating D (direction) in the range of D0 to DN.

Step S2504 is a loop for comprehensively evaluating F (focal length) in the range of F0 to FN, but since the F-value for combination 1 is only F8 (300 mm), this loop is executed only once.
Step S2505 is a loop for comprehensively evaluating AV (aperture value) in the range of AV0 to AVN, but since the F-value of combination 1 is only AV4 (F4.0), this loop is executed only once.

Step S2506 is a loop for comprehensively evaluating TV (shutter speed) in the range TV0 to TVN, and step S2507 is a loop for comprehensively evaluating SV (ISO sensitivity) in the range SV0 to SVN. Step S2508 is a process in which the virtual space construction means 101 constructs a virtual space under conditions based on the loop variable T.

Step S2509 is a process for generating a simulation image when shooting under shooting conditions based on the loop variables P, D, F, AV, TV, and SV.
In step S2510, the evaluation unit 103 calculates an evaluation value between the simulation image generated in step S1709 and the desired captured image.
Steps S2511 and S2512 are processes in which the notification means 104 notifies the user of a simulation image and shooting conditions based on the evaluation result calculated in step S2510.

Steps S2510 to S2513 represent a determination of whether to continue loop processing regarding time T, position P, direction D, focal length F, aperture value AV, shutter speed TV, and ISO sensitivity SV, and steps S2508 to S2512 are repeated until all conditions are covered.
Next, in step S3203, the control unit 105 determines whether or not the generation and evaluation of simulation images have been performed for all combinations. If the generation and evaluation of simulation images have been completed for all combinations, the process proceeds to step S3204.

Next, in step S3204, the control unit 105 compiles the evaluation results of the simulation images performed for all combinations.
FIG. 33 is a diagram showing an example of score calculation for each equipment combination. As shown in FIG. 33, for camera-lens combinations 1 to 4, a score for each desired captured image and a total score, which is the sum of the scores, are calculated.

The score is calculated by integrating the evaluation values (similarity) for each combination when the similarity (Mtotal) is equal to or greater than a threshold value (Th) for a given desired captured image. The total score is calculated by integrating all the scores for each desired captured image. The equipment combination with the highest total score is notified to the user via notification means 104.

As described above, by using this embodiment, it is possible to notify the user of the optimum combination of photographic equipment for obtaining the desired photographic image, so that the user can more accurately select the photographic equipment for obtaining the desired photographic image without having to bring a large amount of photographic equipment to the site in advance.
In addition, since the simulation can be used to automatically extract the shooting conditions for obtaining the desired photographic image, it is possible to extract the shooting conditions for obtaining the desired photographic image more accurately without having to bring a large amount of photographic equipment to the site in advance.

In addition, depending on the desired captured image, similarities in the number of subjects, face position and size, orientation, brightness, blur, etc. are evaluated using different weights for the person area and background area. This makes it possible to extract shooting conditions that will more effectively obtain an image close to the desired captured image.

Next, a third embodiment of the present invention will be described.
In the third embodiment, a method of extracting a combination of photographic equipment that can obtain optimal photographic results within the total weight of the photographic equipment desired by the user by taking the weight of the photographic equipment into consideration will be described.
The basic configuration is the same as in the first embodiment, but the information input means 100 is capable of inputting weight information as desired information on the photographic equipment.

That is, in this embodiment, the information about the photographic equipment includes the camera sensor size, shutter speed, ISO sensitivity, camera weight, lens focal length, aperture value, lens weight, etc. The desired weight information is the total weight of the photographic equipment desired by the user. Here, for example, 1,500 g is input.

The photographic equipment information is information about multiple cameras and multiple lenses. The photographic equipment information may be information about cameras and lenses that the user owns, or information about cameras and lenses that can be rented from a rental service, etc. Also, information about cameras and lenses that the user is considering purchasing may be input.
Information regarding the photographic equipment may be prepared in advance as presets for a plurality of cameras and lenses in the information input means, and the information may be selected from the presets.

The possible ranges of the shooting conditions (shooting angle of view, aperture value, shutter speed, ISO sensitivity) described in FIG. 9 of the first embodiment are determined according to the combination of camera and lens in the input information about the shooting equipment.
for example,

Camera 1: Sensor size: Full size, Shutter speed: 1 second to 1/8000 second, ISO 100 to ISO 51200, Weight: 800g
Camera 2: Sensor size: APS-C, shutter speed: 1 second to 1/4000 second, ISO 100 to ISO 12800, 400g
Lens 1: focal length 300 mm, F value 4.0, weight: 1100 g
Lens 2: focal length 135 mm, F value 2.0, weight: 700 g
Given the following information about the equipment:

The shooting conditions for combination 1 (camera 1 + lens 1) are as shown in FIG.
The shooting conditions for combination 2 (camera 1 + lens 2) are as shown in FIG.
The shooting conditions for combination 3 (camera 2 + lens 1) are as shown in FIG.
The shooting conditions for combination 4 (camera 2 + lens 2) are as shown in FIG.

The total weight for each combination is then calculated.
The total weight of combination 1 (1 camera + 1 lens) is 1,900g.
The total weight of combination 2 (camera 1 and lens 2) is 1,500g.
The total weight of combination 3 (camera 2 and lens 1) is 1,500g.
The total weight of combination 4 (2 cameras and 2 lenses) is 1,100 g.
It becomes.

Next, the total weight of each combination is compared with the desired weight information (1,500 g). Because the total weight of combination 1 is heavier than the desired weight information, it is decided to exclude combination 1 from the generation of a simulation image. Because the total weights of combinations 2 to 4 are less than the desired weight information, it is decided to generate a simulation image.

The flow of generating and evaluating simulation images (Figure 25) explained in Example 1 is carried out for combinations 2 to 4, and the evaluation results calculated for each combination are compiled and compared to extract the optimal combination of equipment for obtaining the desired photographic image. The rest of the process is the same as in Example 2, so the explanation is omitted.

In addition, in this embodiment, the total weight of each combination is simply compared with the desired weight information, but the difference Mg (=Mg-Tg) between the desired weight information Ig and the total weight Tg may be multiplied by a weighting coefficient Kg and added to the similarity of the entire image. In this case, the similarity Mtotal2 is calculated using Equation 9.

Mtotal2=Kn*Mn+Kp*Mp+Ks*Ms
+Kd*Md+Kl*Ml+Kb*Mb
+K'n*M'n+K'p*M'p+K's*M's
+K'd*M'd+K'l*M'l+K'b*M'b+Kg*Mg... (Formula 9)

Since the other elements than Kg and Mg are the same as those in the first embodiment, the explanation will be omitted.
If the total weight is heavier than the desired weight, Mg will be a negative value and Mtotal2 will be lower, and if the total weight is lighter than the desired weight, Mg will be a positive value and Mtotal2 will be higher.

Mtotal2 may be compared with a preset threshold value (Th), and only when Mtotal2 is greater than Th, the user may be notified of the simulation image and information about the photographing equipment and photographing conditions used when generating the simulation image. In other words, when the similarity is equal to or greater than a predetermined value, the notifying means may notify the user of information about the combination of camera and lens.
The notification method may be output to a display device of the user terminal, output by printing, or data output to an external medium or a device connected to a network.

As described above, according to this embodiment, the user can be notified of the optimum combination of photographic equipment taking into consideration the weight required to obtain the desired photographic image, so that the user can more accurately select the photographic equipment required to obtain the desired photographic image without having to bring a large amount of photographic equipment to the site in advance.
In addition, since the shooting conditions for obtaining the desired photographic image can be automatically extracted using simulation, the shooting conditions for obtaining the desired photographic image can be extracted in advance more accurately without having to bring a large amount of photographic equipment to the site in advance.

In addition, depending on the desired captured image, the number of subjects, the position and size of the face, the direction, brightness, and the degree of similarity in the blur are evaluated using different weights for the person area and the background area. This makes it possible to extract in advance the shooting conditions that will more effectively obtain an image close to the desired captured image.

The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments. Various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention.
A computer program for implementing all or part of the control in this embodiment may be supplied to the simulation device via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the simulation device may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention.

100 Information input means 101 Virtual space construction means 102 Simulation image generation means 103 Evaluation means 104 Notification means 105 Control means

Claims (20)

an information input means for inputting information on the event to be photographed, photographing information, and a desired photographed image;
a virtual space constructing means for constructing a virtual space that imagines the event using a subject model based on the information input by the information inputting means;
a simulation image generating means for generating a simulation image using the virtual space from an image obtained by photographing the event;
an evaluation means for evaluating a similarity between the simulation image and a desired captured image in terms of at least one of brightness and blur amount;
a notification means for notifying a user of a combination of photographing equipment required for taking the simulation image having a similarity evaluated by the evaluation means that is equal to or greater than a predetermined value , and of photographing conditions corresponding to the combination. The simulation device according to claim 1, characterized in that the event information includes subject position information representing the content of the event, the order of the event, and movement information of subjects participating in the event. The simulation device according to claim 2, characterized in that the subject position information includes information regarding at least one of the subject's position and subject's orientation at each time. The simulation device according to claim 2 or 3, characterized in that the subject position information can be selected from a plurality of preset data. The simulation device according to any one of claims 1 to 4, characterized in that the shooting information includes the shooting position, shooting direction, shooting angle of view, and information regarding at least one of aperture value, shutter speed, ISO sensitivity, and white balance data. The simulation device according to any one of claims 1 to 5, characterized in that the desired captured image includes image information and is associated with at least one of the number of subjects, face position, face size, and face direction. The simulation device according to any one of claims 1 to 6, characterized in that the desired captured image can be selected from a plurality of sample images stored as preset data. The simulation device according to any one of claims 1 to 7, characterized in that the information input means is capable of inputting at least one of information regarding the brightness of the venue where the event is held and a background image of the event venue. The simulation device according to any one of claims 1 to 8, characterized in that the simulation image generating means adds a background image to the simulation image based on image data of the background of the venue where the event is held. The simulation device according to any one of claims 1 to 9, characterized in that the simulation image generating means adjusts the brightness of the simulation image based on the input shooting conditions. The simulation device according to any one of claims 1 to 10, characterized in that the simulation image generating means adds blur to at least the background image of the simulation image based on the input shooting conditions. The simulation device according to any one of claims 1 to 11, characterized in that the evaluation means calculates the similarity between the simulation image and the desired captured image for at least two of the following: the number of subjects, the positions of the subjects, the sizes of the subjects, the orientations of the subjects, the brightness of the subjects, the background image, the amount of background blur, and the colors of the subjects. The simulation device according to claim 12, characterized in that the evaluation means calculates an evaluation value by weighting and adding the at least two calculated similarities. The simulation device according to any one of claims 1 to 13, characterized in that the notification means notifies the user of the simulation image and the corresponding shooting conditions when the similarity is equal to or greater than a predetermined value. The simulation device according to any one of claims 1 to 14, characterized in that the information input means is capable of inputting information about the photographic equipment. The simulation device according to claim 15, characterized in that the information about the photographic equipment includes information about at least one of the camera's sensor size, shutter speed, ISO sensitivity, and camera weight. The simulation device according to claim 16, characterized in that the information about the photographic equipment includes information about the focal length of the lens, the aperture value, and the weight of the lens. The simulation device according to claim 17, characterized in that the notification means notifies the user of information regarding the camera and lens combination when the similarity is equal to or greater than a predetermined value. A computer program for controlling each means of the simulation device according to any one of claims 1 to 18 by a computer. 20. A computer readable storage medium storing the computer program according to claim 19.

Images