JP7615554B2 - Photo movie generating system, photo movie generating device, photo movie generating method, and program - Google Patents

Description

The present invention relates to a photo movie generation system that generates and provides photo movies.

Patent Document 1 proposes an example of a photography service at a tourist spot or theme park, in which a photo rally is held in which users are photographed by cameras installed at multiple locations in the tourist spot, etc., and an image printing system is proposed in which the photographed images are printed out after the rally is over. By using this photography service, users can preserve their memories in photographs.

Cloud photo storage services also provide a service that automatically creates photo movies using photos stored in cloud storage. For example, in the cloud photo storage service of Non-Patent Document 1, a user selects photos and videos that they want to make into a movie from the images stored in cloud storage, and selects a desired theme from several pre-prepared themes, and the movie is automatically generated by the service.

JP 2018-074281 A

Google Photos, "Create a movie", [online], [Searched on July 8, 2020], Internet <URL: https://photos.google.com/movies/create>

If users could be provided with movies with music (BGM) using photos taken with photography services at theme parks, event venues, tourist spots, etc., they would be able to enjoy the photos even more, which would likely increase the appeal of the photography service.

Regarding the music to be added to the movie, the movie creation function of Non-Patent Document 1 allows the user to select the music, but the music that can be selected is either a default category (dramatic, electronic, rockin', upbeat, etc.) or music that is DRM (Digital Rights Management) protected from music in the music application in the user's terminal.
The service only allows songs that are not protected by Digital Rights Management (Digital Rights Management). Therefore, users cannot select songs related to the places they have visited, and it is not sufficient for enjoying memories of theme parks, etc.

The present invention has been made in consideration of the above problems, and aims to provide a photo movie creation system that allows users to fully enjoy their memories of theme parks, etc.

The first invention for solving the above-mentioned problem is a photo movie generating system in which a photo movie generating device and a user terminal are connected to each other via a network so as to be able to communicate with each other, the photo movie generating device stores movie configuration information including effect patterns in a storage unit, which indicates a configuration of screen changes in the generated photo movie in accordance with music, and when a request for generating a photo movie is received from the user terminal, the photo movie generating device has an image acquiring means for acquiring images taken by the user at a shooting spot to be used in the photo movie, a music acquiring means for acquiring acoustic feature data of a music piece related to the shooting spot when a request for generating a photo movie is received from the user terminal, and a music acquiring means for acquiring the acoustic feature data of the music piece related to the shooting spot and the music piece related to the shooting spot. and a photo movie generation means for generating a photo movie to be played on a user terminal using images acquired by the image acquisition means and transmitting the generated photo movie to the user terminal , wherein the photo movie generation means generates a photo movie in which the screen changes in time with music based on acoustic feature data acquired by the music acquisition means, and when generating a new photo movie to be played along with the music, determines whether or not the movie composition information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie composition information exists, generates a new photo movie having a pattern different from the pattern of the movie composition information .
For example, the shooting spot is set up in a theme park, a tourist attraction, or an event venue, and the photo movie generation system further includes an image storage that stores images taken by the user at the shooting spot in association with the user ID of the user, and a photography device that is set up at the shooting spot and takes photos of the user, the photography device reads the user ID when taking photos of the user, and if the user visits a theme park, tourist attraction, or event venue multiple times, the image storage stores the images taken by the photography device during each visit in association with the same user ID.
It is also preferable that the image acquisition means acquires images provided by businesses of the visited places, in addition to images taken by the user at photography spots in the visited places.

According to the first invention, when a request to generate a photo movie is sent from a user terminal to a photo movie generating device, the photo movie generating device acquires images of the user taken at a photography spot, generates a photo movie from the images that is played along with music related to the photography spot, and sends the photo movie to the user terminal. This allows the photo movie generating system to provide the user with a photo movie in which images taken at a photography spot such as a theme park are played along with music related to the theme park, etc., allowing the user to fully enjoy their memories and improving the appeal of the photography service.

Furthermore, in the first invention , it is possible to generate an impressive photo movie in which the screen changes in accordance with the feature amount of the music.

Furthermore, in the first invention , photo movies with different expressions for the same music can be provided, so that the user can be provided with a variety of entertainment.

It is also desirable that the photo movie generation means generates the photo movie without adding the audio of the song, links it to identification information of the song and transmits it to the user terminal, and that the user terminal is equipped with a playback means for acquiring audio data of the song corresponding to the identification information and playing the acquired audio data in synchronization with the photo movie.
This makes it possible to generate photo movies with smaller file sizes than photo movies with music, thereby saving on storage space and communication bandwidth.

It is also desirable that the photo movie production device further includes an image material generation means for cutting out an object within an image acquired by the image acquisition means and generating image material, and that the photo movie production means generates the photo movie using the image material generated by the image material generation means.
By generating a photo movie using image material generated from an image, a more interesting and enjoyable photo movie can be provided.

It is also preferable that the user terminal comprises a music selection means for accepting a selection of music to be used in the photo movie and transmitting the selection information of the music to the photo movie generating device together with a request to generate the photo movie.
Furthermore, it is preferable that the user terminal comprises an image designation means for accepting designation of images to be used in the photo movie and transmitting the image designation information to the photo movie production device together with a request to produce the photo movie.
This allows the user to specify his/her favorite music and images and generate a photo movie.

A second invention is a photo movie generating device connected to a user terminal via a network so as to be able to communicate with the user terminal, which stores movie configuration information in a memory unit, including effect patterns indicating the configuration of the screen changes in the generated photo movie in accordance with music, and when a request to generate a photo movie is received from the user terminal, it acquires images taken by the user at a shooting spot to be used for the photo movie, and when a request to generate a photo movie is received from the user terminal, it acquires acoustic feature data of a song related to the shooting spot, and a photo movie generating means uses the images acquired by the image acquisition means to generate a photo movie to be played on the user terminal together with the song related to the shooting spot , and transmits the photo movie to the user terminal, wherein the photo movie generating means generates a photo movie in which the screen changes in accordance with music based on the acoustic feature data acquired by the song acquisition means, and when generating a new photo movie to be played together with the song, it determines whether the movie configuration information of the photo movie generated in the past to be played together with the same song is present in the memory unit, and if the movie configuration information is present, it generates a new photo movie having a pattern different from the pattern of the movie configuration information .

A user terminal communicatively connected to a photo movie generating device via a network is characterized by comprising: a photo movie generation request means for sending a request for generating a photo movie to the photo movie generating device; and a receiving means for receiving from the photo movie generating device a photo movie that is generated using images taken by a user at a shooting spot and is played on the user terminal together with music related to the shooting spot.

The third invention is a photo movie generating method in a photo movie generating system in which a photo movie generating device and a user terminal are connected to each other so as to be able to communicate with each other via a network, the photo movie generating device storing movie configuration information including effect patterns indicating the configuration of the screen changes in the generated photo movie in accordance with music in a memory unit , when receiving a request to generate a photo movie from the user terminal, executes the following steps: acquiring images taken by the user at a shooting spot to be used for the photo movie; acquiring acoustic feature data of a song related to the shooting spot when receiving a request to generate a photo movie from the user terminal; generating a photo movie to be played on the user terminal together with the song related to the shooting spot using the images and transmitting the generated photo movie to the user terminal; the photo movie generating device generates a photo movie in which the screen changes in accordance with the music based on the acoustic feature data; and when generating a new photo movie to be played together with the song, the photo movie generating device determines whether the movie configuration information of a photo movie generated in the past to be played together with the same song exists in the memory unit, and if the movie configuration information exists, generates a new photo movie having a pattern different from the pattern of the movie configuration information .

A fourth invention relates to a photo movie generating device that is communicably connected to a user terminal via a network and stores movie configuration information including effect patterns in a storage unit, the effect patterns indicating a configuration of screen changes in the generated photo movie in accordance with music, the photo movie generating device including: an image acquiring means that acquires images taken by a user at a shooting spot to be used in the photo movie when a request for generating a photo movie is received from the user terminal; a music acquiring means that acquires acoustic feature data of a music piece related to the shooting spot when a request for generating a photo movie is received from the user terminal; and a photo movie to be played on the user terminal together with the music piece related to the shooting spot, using the images acquired by the image acquiring means. and a photo movie generation means for generating a photo movie based on acoustic feature data acquired by the music acquisition means and transmitting the generated photo movie to the user terminal, wherein the photo movie generation means generates a photo movie in which the screen changes in time with music based on acoustic feature data acquired by the music acquisition means, and when generating a new photo movie to be played along with the music, determines whether or not the movie configuration information of a previously generated photo movie to be played along with the same music is present in the memory unit, and if the movie configuration information is present, generates a new photo movie having a pattern different from the pattern of the movie configuration information.This is a program for causing the photo movie generation device to function as a photo movie generation device .

A program of a user terminal communicatively connected to a photo movie generating device via a network causes the user terminal to function as a user terminal equipped with a photo movie generation request means for sending a request to generate a photo movie to the photo movie generating device, and a receiving means for receiving from the photo movie generating device a photo movie that is generated using images taken by the user at a shooting spot and is played on the user terminal together with music related to the shooting spot .

The present invention makes it possible to provide a photo movie generation system that allows users to fully enjoy their memories of theme parks, etc.

1 is a diagram showing the overall configuration of a photo movie production system 1. FIG. 2 is a diagram showing the hardware configuration of the photo movie generating apparatus 2. FIG. 2 is a diagram showing the hardware configuration of a user terminal 3. FIG. 2 is a diagram showing the data structure of an image storage 5. FIG. 2 is a diagram showing the data structure of a music database 60. 3A shows an example of the data structure of music information 63, and FIG. 3B shows an example of the data structure of acoustic feature data 64. FIG. 1 shows the functional configuration of a photo movie production system 1. 4 is a flowchart showing the flow of processing executed by the photo movie production system 1. FIG. 4 is a diagram showing an example of an operation screen 8. FIG. 2 shows an example of an effect template 221. FIG. 4 shows an example of an effect correspondence table 222. 11 is a flowchart showing the flow of a photo movie generating process. 7A to 73 are diagrams showing examples of photo movies 71, 72, and 73. FIG. 2 shows an example of movie configuration information 223.

The following describes in detail a preferred embodiment of the present invention with reference to the drawings.

Figure 1 is a diagram showing the overall configuration of a photo movie production system 1 according to an embodiment of the present invention. The photo movie production system 1 generates a photo movie using images (photos) taken by a user at a photography service such as a theme park, and provides the photo movie to the user.

Theme park photography services involve photographing users with photography devices installed at locations visited by users, such as theme parks such as amusement parks, tourist spots, or event venues (these locations are collectively referred to as theme parks, etc.). With this photography service, the photographed images can be printed out and sold to users, or the image data can be uploaded to storage so that it can be downloaded later on the user's smartphone, etc.

In this embodiment, it is assumed that photography spots A, B, C, ... are provided in a theme park, and photography services are provided by photography devices 4a, 4b, 4c, ... installed at photography spots A, B, C, .... Photography devices 4a, 4b, 4c, ... are stationary photography devices equipped with, for example, a camera, a user ID reader, a communication interface, etc.

As shown in FIG. 1, in the photo movie production system 1, a photo movie production device 2, a user terminal 3, an image storage 5, a music distribution server 6, etc. are connected so as to be able to communicate with each other via a network 7.

In this embodiment, the photography devices 4a, 4b, 4c, ... are also connected to the image storage 5 via the network 7 so as to be able to communicate with them. The user is issued a user ID by the theme park by registering in advance, and when taking pictures with the photography devices 4a, 4b, 4c, ..., the photography devices 4a, 4b, 4c, ... read the user ID displayed as a two-dimensional code on the display screen of the user terminal 3 or on the face of the ticket. This allows the images taken with the photography devices 4a, 4b, 4c, ... to be linked to the user ID and stored in the image storage 5. If the user has visited the theme park multiple times, the images taken with the photography devices 4a, 4b, 4c, ... during each visit can be linked to the same user ID and stored.

In response to a request from the user terminal 3, the photo movie generating device 2 generates a photo movie using the above images and transmits it to the user terminal 3.

Figure 2 is a diagram showing the hardware configuration of the photo movie generating device 2. As shown in Figure 2, the photo movie generating device 2 can be realized by, for example, a server computer configured by connecting a control unit 201, a storage unit 202, a communication unit 203, etc. via a bus 204, etc. However, this is not limited to this, and various configurations can be used as appropriate, and the photo movie generating device 2 can also be configured by multiple computers.

The control unit 201 includes a CPU (Central Processing Unit).
The control unit 201 is made up of a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The CPU loads programs stored in the storage unit 202, ROM, and other recording media into a work memory area on the RAM and executes them, controlling the operation of each unit connected via the bus 204. The ROM permanently stores programs such as the computer's boot program and BIOS, as well as data. The RAM temporarily stores loaded programs and data, and also has a work area that the control unit 201 uses to perform various processes. The control unit 201 functions as each means of the photo movie creation device 2 by reading and executing the programs.

The storage unit 202 is a recording medium such as a hard disk drive, solid state drive, or flash memory. The storage unit 202 stores programs executed by the control unit 201, data required for program execution, an operating system, and the like. The storage unit 202 also stores application programs for performing the processes described below (S105 to S106, S109 to S110, and S113 to S116 in FIG. 8), and the control unit 201 of the photo movie generating device 2 performs the processes described below in accordance with these application programs.

As shown in FIG. 7, the storage unit 202 of the photo movie generating device 2 also stores data related to the generation of the photo movie, such as an effect template 221, an effect correspondence table 222, and movie configuration information 223. Details of these data will be described later.

The communication unit 203 has a communication control device, a communication port, etc., and controls communication via the network 7. The network 7 may be a LAN (Local Area Network) or a WAN (Wide Area Network) that is connected for communication over a wider area.
The network 7 may be a wired or wireless network.

The bus 204 is a path that mediates the exchange of control signals, data signals, etc. between each part of the photo movie generating device 2.

Returning to the explanation of FIG. 1, the user terminal 3 is a terminal used by the user, and may be, for example, a smartphone, tablet, PC, music player, game console, etc. The user terminal 3 is equipped with a dedicated application program (hereinafter referred to as a photo movie app) for using the photo movie production system 1. Alternatively, the user terminal 3 may be equipped with a browser capable of browsing a website set up on the network 7 by the photo movie production device 2, and perform processing related to the generation of a photo movie via the website.

Figure 3 is a diagram showing the hardware configuration of the user terminal 3. As shown in the figure, the user terminal 3 can be realized by a computer or the like configured by connecting, for example, a control unit 301, a memory unit 302, a communication unit 303, a display unit 304, an input unit 305, a peripheral device I/F unit 306, and an audio output unit 307 via a bus 308 or the like. However, the present invention is not limited to this, and various configurations can be used as appropriate.

The control unit 301, storage unit 302, communication unit 303, and bus 308 are substantially similar to the control unit 201, storage unit 202, communication unit 203, and bus 204 of the photo movie generating device 2. Note that the storage unit 302 of the user terminal 3 stores a photo movie application for performing the processes described below (S101 to S104, S117, S118, S121, and S122 in FIG. 8), and the control unit 301 of the user terminal 3 executes the processes described below in accordance with the photo movie application.

The display unit 304 is composed of a display device such as a liquid crystal panel or a CRT monitor, and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display device, and displays display information input under the control of the control unit 301 on the display device. The display unit 304 may be a touch panel display in which the touch panel, which is the input unit 305, is integrally provided with the display device.

The input unit 305 is, for example, an input device such as a touch panel, a keyboard, a pointing device such as a mouse, or a numeric keypad, and outputs input data to the control unit 301.

The peripheral device I/F (interface) unit 306 is a port for connecting peripheral devices, and data is sent and received to and from the peripheral devices via the peripheral device I/F unit 306. The peripheral device I/F unit 306 is configured with a USB or the like, and typically has multiple peripheral device I/Fs. The connection with the peripheral devices may be wired or wireless.

The audio output unit 307 outputs the audio data input from the control unit 301 from a speaker.

Returning to the explanation of FIG. 1, as described above, the image storage 5 is a storage that accumulates and stores images of users taken with the photographic devices 4a, 4b, 4c, .... In the image storage 5, images taken by users with the photographic devices 4a, 4b, 4c, ... are stored in association with user IDs.

Figure 4 is a diagram showing the data structure of the image storage 5. As shown in Figure 4, in the image storage 5, image data 53 is stored in association with a user ID 51 and an image ID 52.

User ID 51 is user identification information issued by the theme park upon completing the user registration described above.

The image ID 52 is identification information for the image data 53 and is issued by the photography device 4a, 4b, 4c, ... or the image storage 5.

Image data 53 is an image of the user taken by photography device 4a, 4b, 4c, ....

In addition, image data 53 (images of the theme park's buildings, scenery, characters, etc.) provided by the theme park operator can also be stored in the image storage 5 in association with image ID 52 as images to be used as material for the photo movie.

Returning to the explanation of FIG. 1, the music distribution server 6 transmits music sound source data and the like in response to requests from the photo movie generating device 2 and the user terminal 3. The music distribution server 6 stores music related to theme parks in a music database 60.

As shown in FIG. 5, the music database 60 stores music sound source data 62, music information 63, and acoustic feature data 64 in association with a music ID 61.

The song ID 61 is identification information for the song and is issued by the music distribution server 6.

The sound source data 62 is sound source data for songs related to the theme park, such as theme songs, character songs, sound effects, background music, etc.

As shown in FIG. 6(a), song information 63 is additional information about the song, such as the song title, artist name, lyrics information, and playback time.

As shown in FIG. 6B, the acoustic feature data 64 is a feature of the sound source data 62, such as the music composition (A melody, B melody, chorus, etc.), the tempo of the music (BPM; Beats Per Minute), beat (time signature), the timing of pronunciation of a specific instrument, chord progression (the flow of the melody of the music), volume change (audible volume change), frequency center of gravity (degree of excitement of the music), and pitch power separation (complexity of the music). The acoustic feature data 64 is obtained by known analysis methods (for example, CRI Middleware Co., Ltd., "Ultra-fast and high-precision music analysis middleware BEATWIZ (registered trademark)", [onine], [searched July 8, 2020], Internet, <URL: https://www.cri-mw.co.jp/product/amusement/beatwiz/index.html>, or "Song
The items of the music information 63 and the acoustic feature data 64 shown in FIG. 6 are merely examples, and the present invention is not limited to these.

Next, the functional configuration of the photo movie production system 1 will be described with reference to FIG. 7.

As shown in FIG. 7, in the photo movie production system 1, the photo movie production device 2 includes, in addition to the above-mentioned storage unit 202, a music acquisition unit 21, an image acquisition unit 23, an image material production unit 24, a photo movie production unit 25, etc. The user terminal 3 also includes a music designation unit 31, an image designation unit 32, a photo movie production request transmission unit 33, a reception unit 34, a sound source data acquisition unit 35, a playback processing unit 36, etc.

The music selection unit 31 (music selection means) of the user terminal 3 accepts the selection of music to be used in the photo movie. The music selection unit 31 accepts the selection of music from among the music stored in the music distribution server 6 through the user's operation on the operation screen 8 (see FIG. 9) described later, and transmits the music ID 61 (selection information) to the photo movie production device 2 together with a request to produce a photo movie.

The image designation unit 32 (image designation means) accepts designation of images to be used in the photo movie. The image designation unit 32 accepts designation of an image from among images stored in the image storage 5 through user operation on the operation screen 8, and transmits the image ID 52 (designation information) to the photo movie creation device 2 together with a request to create a photo movie.

The photo movie generation request sending unit 33 (photo movie generation request means) sends a request to generate a photo movie to the photo movie generating device 2 together with the above-mentioned music ID 61 and image ID 52. If a photo movie type (such as "photo movie where the screen changes along with music" or "slide show with BGM") described below is specified on the operation screen 8, the photo movie type specification information is also sent to the photo movie generating device 2 together with the request to generate a photo movie.

When the music acquisition unit 21 (music acquisition means) of the photo movie generating device 2 receives a request to generate a photo movie from the user terminal 3, it acquires music-related data such as sound source data 62, music information 63, and acoustic feature data 64 linked to the above-mentioned music ID 61 from the music distribution server 6.

When the image acquisition unit 23 (image acquisition means) receives a request to generate a photo movie from the user terminal 3, it acquires image data 53 linked to the above image ID 52 from the image storage 5.

The image material generating unit 24 (image material generating means) generates image material to be used in the photo movie from the image data 53 acquired by the image acquiring unit 23. The image material generating unit 24 detects objects such as people, animals, objects, etc. from the image data 53 such as a photograph, cuts out the detected objects, and creates individual image material for each object. The image material generating unit 24 outputs the generated image material to the photo movie generating unit 25.

The photo movie generating unit 25 (photo movie generating means) generates a photo movie using the image data 53 acquired by the image acquiring unit 23 and the image material generated by the image material generating unit 24.

Here, the photo movie generating unit 25 can use the music information 63 and the acoustic feature data 64 to generate a "slide show with background music" or a "photo movie in which the screen changes in sync with the music" according to the photo movie type specified above.

A "slide show with BGM" is a photo movie that uses a specified song as background music and displays specified images in a slide show format. The photo movie generator 25 generates a "slide show with BGM" based on the song information 63. Specifically, for example, the "play time" of the song information 63 is divided equally by the number of images, and images are replaced every equal amount of time.

The "photo movie in which the screen changes in sync with the music" is a photo movie that is generated by replacing images and applying effects such as movement and rotation in response to changes in the music detected from the "music composition" and "timing of the pronunciation of specific instruments" in the acoustic feature data 64. The photo movie generating unit 25 references the effect correspondence table 222 and effect template 221 stored in the storage unit 202 to generate the "photo movie in which the screen changes in sync with the music".

In this embodiment, the photo movie generated by the photo movie generating unit 25 is a photo movie without music. A photo movie without music is a photo movie that contains only images without the addition of music sound source data 62. As described later, the photo movie without music is played back in synchronization with the music sound source data 62 on the user terminal 3.

When the photo movie generating unit 25 generates a photo movie, it compresses and encodes the data using the MPEG4 format or the like, and converts it into a data format that can be played on the user terminal 3. Then, it links movie configuration information 223, which is information that indicates the configuration of the photo movie, to the music ID 61 and stores it in the storage unit 202, and also links the photo movie data to the music ID 61 and transmits it to the user terminal 3. The movie configuration information 223 will be described later.

The receiving unit 34 (receiving means) of the user terminal 3 receives the photo movie sent from the photo movie generating device 2.

The audio source data acquisition unit 35 (audio source data acquisition means) acquires audio source data 62 of a song corresponding to a song ID 61 linked to a photo movie from the music distribution server 6.

The playback processing unit 36 (playback means) plays the photo movie. In this embodiment, the acquired sound source data 62 and the photo movie (photo movie without music) received from the photo movie generating device 2 are played in synchronization.

Next, the flow of processing executed by the photo movie production system 1 will be described with reference to FIG. 8.

When a user launches the photo movie application on the user terminal 3 and inputs the user ID 51, the user terminal 3 displays the operation screen 8 illustrated in FIG. 9, and accepts the specification of images to be used in the photo movie (step S101), the specification of music (step S102), and the specification of the photo movie type (step S103).

The images that can be specified in step S101 are image data 53 stored in image storage 5, and in addition to image data 53 taken by the user with photography devices 4a, 4b, 4c, ..., image data 53 provided by the theme park operator can also be specified. On operation screen 8, thumbnails of these images are displayed in a horizontally scrollable image list 83 along with a message 81 requesting image selection, such as "Please select images to use in the photo movie." If the user visits the theme park multiple times and uses the photography service, it is desirable to be able to specify image data 53 taken with photography devices 4a, 4b, 4c, ... not only during the most recent visit, but also during past visits. The image data 53 specified by the user is displayed in selected image display area 82.

In addition to the user's designation, the user may be allowed to select "automatic" to leave the designation of images to the photo movie generating device 2. In the "automatic" case, the photo movie generating device 2 extracts a predetermined number of images from the image data 53 taken by the photographing devices 4a, 4b, 4c, ... based on, for example, the image ratings (quality of smile level, contrast, brightness, composition, etc.), and displays them in the selected image display area 82 in descending order of rating or in chronological order.

The songs that can be selected in step S102 are those stored in the music database 60 of the music distribution server 6, and a message 84 is displayed on the operation screen 8 prompting the user to select a song, such as "Please select music to use in the photo movie," along with a list of these songs 86, 87, 88 displayed in a pull-down menu.

Specifiable songs are displayed by category, such as song list 86 being "theme songs," song list 87 being "character songs," and song list 88 being "instrumental." Alternatively, specifiable songs may be displayed in alphabetical order or in alphabetical order. Songs specified by the user are displayed in selected song display area 85. The "Preview" button 85a in FIG. 9 is the button that is operated when previewing a specified song.

As described above, in addition to the user's choice, the user may also be allowed to select "automatic selection," which allows the photo movie generating device 2 to select the music. In the "automatic selection," the photo movie generating device 2 may randomly select a music piece from those stored in the music database 60, or may select a music piece predetermined by the theme park operator depending on the time of year, such as "Character Song 2" from June to August.

The photo movie types that can be specified in step S103 are the above-mentioned "photo movie with screen changes in sync with music" and "slide show with BGM," and the operation screen 8 displays a message 89 such as "Please select the type of photo movie" to indicate that the photo movie type is to be specified, along with radio buttons 91 and 92 that can be specified. The photo movie type selected by the user is displayed in the selected type display area 90.

Once the images, music, and photo movie type to be used in the photo movie have been specified in this manner, the user terminal 3 sends a photo movie generation request to the photo movie generation device 2 along with the music ID 61 of the specified music, the image ID 52 of the specified image, and the specified photo movie type (step S104).

When the photo movie generating device 2 receives a request to generate a photo movie (step S105), it requests the music distribution server 6 to transmit the song information 63 and acoustic feature data 64 corresponding to the specified song ID 61 (step S106). When the music distribution server 6 receives a request from the photo movie generating device 2 (step S107), it responds to this request by transmitting the song information 63 and acoustic feature data 64 corresponding to the song ID 61 to the photo movie generating device 2 (step S108). The photo movie generating device 2 receives and acquires the song information 63 and acoustic feature data 64 (step S109).

The photo movie production device 2 also requests the image storage 5 to transmit image data 53 corresponding to the specified image ID 52 (step S110). When the image storage 5 receives the request from the photo movie production device 2 (step S111), it responds to this request by transmitting the image data 53 corresponding to the image ID 52 to the photo movie production device 2 (step S112). The photo movie production device 2 receives and acquires the image data 53 (step S113).

When the image data 53 is acquired, the photo movie generating device 2 executes an image material generating process for the acquired image data 53 as necessary, and extracts an image material from the image data 53 (step S114). Note that the received image data 53 may be used as an image material as it is.

Next, the photo movie generating device 2 generates a photo movie of the type specified above (step S115).

If the specified photo movie type is "slide show with BGM," the photo movie generating device 2 will, for example, divide the "playback time" of the music information 63 equally by the number of images, and generate a photo movie in slide show format in which images are replaced every equally divided time.

When the specified photo movie type is "a photo movie in which the screen changes in sync with the music," the photo movie generating device 2 generates a photo movie by replacing images and applying effects such as movement and rotation according to changes in the music detected from the "music composition" and "timing of specific instruments being played" in the acoustic feature data 64. The effects to be used are defined in the effect correspondence table 222 and effect template 221 in the storage unit 202.

The effect template 221 is a program for applying a specific effect to an image, and as shown in FIG. 10, multiple effect templates 221 are stored in association with the effect number and the number of images to be used.

Effect templates 221 include, for example, "image switching" for switching between multiple displayed images, "monochrome to color" for changing an image from monochrome to color, "transmittance change" for changing the transmittance of an image, "screen movement" for moving the display position of an image on the screen, "rotation" for rotating an image, "enlargement" for enlarging an image, "reduction" for reducing an image, "display ratio change" for changing the display ratio of an image on the display screen, "appearance" for making an image appear, "enlargement movement" for moving an image while enlarging it, "enlargement rotation" for rotating an image while enlarging it, "reduction movement" for moving an image while reducing it, "reduction rotation" for moving an image while reducing it, "small image increase" for gradually increasing the number of small images, "multiple image single screen display" for displaying multiple images on one screen, "multiple image single screen display + slide" for sliding multiple images displayed on one screen, and "alternating display" for displaying images alternately on the screen.

Note that the effect template 221 shown in FIG. 10 is an example, and may include other effects. Also, multiple patterns of parameters such as the amount of movement and the speed of change may be prepared for each effect. For example, for the effect "Enlarge", "Enlarge 1", "Enlarge 2", "Enlarge 3", etc. may be prepared depending on the enlargement rate, and for the effect "Transmittance change", "Transmittance change 1", "Transmittance change 2", "Transmittance change 3", etc. may be prepared depending on the speed of change.

When generating a photo movie, the photo movie generating device 2 refers to the effect correspondence table 222 to determine which effect to use. FIG. 11 is a diagram showing an example of the effect correspondence table 222. The effect correspondence table 222 is a table that associates music components (intro, A melody, B melody, C melody, solo, chorus, etc.) with the appropriate effect template 221, and as shown in FIG. 11, multiple patterns are defined.

In the example of Figure 11, in the case of pattern (1), the effect "monochrome to color" is applied to the "intro" of the song, the effect "move" is applied to the "verse" of the song, the effect "small image" is applied to the "verse" of the song, the effect "multiple image display" is applied to the "verse" of the song, the effect "enlarge" is applied to the "solo" of the song, and the effect "image switch" is applied to the "chorus" of the song.

In the example of pattern (2) in FIG. 11, the effect "image switching" is applied to the "intro" of the song, the effect "shrink rotation" is applied to the "verse verse" of the song, the effect "enlarge rotation" is applied to the "verse verse" of the song, the effect "screen movement" is applied to the "verse verse" of the song, the effect "enlarge" is applied to the "solo" of the song, and the effect "image switching" is applied to the "chorus" of the song. In this way, multiple patterns of effect application examples are defined in the effect correspondence table 222.

In step S115, the photo movie generating device 2 determines which pattern of the effect correspondence table 222 to use to generate the photo movie, for example, by the process shown in FIG. 12.

As shown in FIG. 12, the photo movie generating device 2 first determines whether movie configuration information 223 corresponding to the song ID 61 specified by the user is stored in the storage unit 202 (step S201). The movie configuration information 223 is data indicating the configuration of screen changes according to music, such as images to be used in the photo movie, the timing of their switching, and effects to be applied to the images. For songs for which photo movies have already been generated by the photo movie generating device 2, the movie configuration information 223 is stored in the storage unit 202 in association with the song ID 61.

In step S201, if the movie configuration information 223 corresponding to the song ID 61 specified by the user is not stored in the storage unit 202 (step S201; No), there is no history of creating a photo movie for that song, so the photo movie generating device 2 reads out an arbitrary pattern from the effect correspondence table 222 (step S202) and generates a photo movie according to the effect correspondence table 222 of the read pattern (step S203).

On the other hand, in step S201, if movie configuration information 223 corresponding to the song ID 61 specified by the user is stored in the storage unit 202 (step S201; Yes), the photo movie generating device 2 reads an unused pattern from the effect correspondence table 222 (step S204), and generates a photo movie according to the effect correspondence table 222 of the read pattern (step S203). By referencing the movie configuration information 223 in this way, for a song for which a photo movie has been created before, a photo movie with a different configuration is created from the second time onwards. This makes it possible to generate different movie expressions for the same song, providing the user with a variety of enjoyment.

Figure 13 shows examples of photo movies 71, 72, and 73 generated in S203.

The photo movie 71 in FIG. 13(a) is generated by directly using the images 531, 532, 533, etc. acquired in S113, and the images displayed are changed when the composition of the music changes (time t1 when the A melody changes to the B melody, time t2 when the B melody changes to the chorus, etc.).

The photo movie 72 in FIG. 13(b) is obtained by applying the effect template 221 to the images 531, 532, 533, ... acquired in S113. The images are changed when the music composition changes, and a predetermined effect is applied to the images in each section (t0-t1, t1-t2, t2-).

In the example of FIG. 13(b), the effect "monochrome to color" is applied to image 531 in the section t0 to t1 of the "A melody," gradually changing the monochrome image 531 to the color image 531. Next, at time t1 when the A melody changes to the B melody, the image is switched and the effect is also changed to "shrink and rotate." In the section t1 to t2, image 531 is used in the background and image 532 in the foreground, and an animation is generated in which foreground image 532 shrinks and rotates. Next, at time t2 when the B melody changes to the chorus, the image is switched. The effect remains "shrink and rotate," but from section t2 onwards, image 531 is used in the background and image 533 in the foreground, and an animation is generated in which foreground image 533 shrinks and rotates.

The photo movie 73 in FIG. 13(c) was generated using images 534, 536, 539, etc. acquired in S113 as background images and image materials 535, 537, 540, 541 generated in S114 as foreground images. The background image changes when the music composition changes, and a predetermined effect is applied to the foreground image in each section (t0-t1, t1-t2, t2-).

In the example of FIG. 13(c), image 534 is displayed as the background during section t0-t1 of the A melody, and the "move" effect is applied to foreground image material 535, causing it to move as indicated by the arrow. Next, at time t1, when the A melody changes to the B melody, the background image is switched to image 536, and the "move + color change" effect is applied to foreground image material 537, generating an animation in which the color changes and the image moves as indicated by the arrow. Next, at time t2, when the B melody changes to the chorus, the background image is switched to image 539, and the foreground images are also changed to image materials 540 and 541. During section t2 onwards, image materials 540 and 541 are displayed alternately in time with the bass drum sounding timing.

When the photo movie generating device 2 generates a photo movie, it links the movie configuration information 223 of the generated photo movie to the music ID 61 and stores it in the storage unit 202. As shown in FIG. 14, the movie configuration information 223 is linked to the music ID 61 and records the music configuration, time, effect template, number of images used, and file names of the images used.

The example in Figure 14 is movie configuration information 223 for a photo movie that uses the song with song ID "00001" as background music. The duration of the "intro" section of the music composition is "0-15," and the effect template "monochrome to color" is applied. This effect template uses one image, and the image file name used is "image15."

The duration of the "A melody" section of the song is "15-50", and the effect template "Enlarge and rotate" is applied. This effect template uses two images, and the image file names used are "image8" for the foreground and "image2" for the background.

The duration of the "B melody" section of the song is "50-70", and the effect template "shrink and rotate" is applied. This effect template uses two images, and the image file names used are "image8" for the foreground and "image17" for the background.

The duration of the "chorus" section of the song is "70-120", and the effect template "Image Switch" is applied. This effect template uses five images, and the file names of the images used are "image10" for the first image, "image11" for the second image, "image12" for the third image, "image13" for the fourth image, and "image14" for the fifth image.

The times (hours) of the song composition "Bass drum timing" are "70, 110, 111, 112, 113", and the effect template "Image display" is applied at these times. This effect template uses six images, and the image file names used are "image15" for the background, and the image "image1" is displayed at the first (1st) bass drum timing, the image "image2" at the second (2nd) bass drum timing, the image "image3" at the third (3rd) bass drum timing, the image "image4" at the fourth (4th) bass drum timing, and the image "image5" at the fifth (5th) bass drum timing.

The "music composition" and "time" items in the movie composition information 223 correspond to the acoustic feature data 64 of the music. The "effect template" item is obtained from the effect correspondence table 222 shown in FIG. 11, and the "number of images used" item is obtained from the effect template 221 shown in FIG. 10. The photo movie generating device 2 determines which effect correspondence table 222 to use by executing the process of FIG. 12. In addition, the "file name of images used" item is arbitrarily determined by the photo movie generating device 2 from the image data 53 obtained in S113 and the image material generated in S114.

Returning to the explanation of FIG. 8, when the photo movie is generated by the processing of step S115, the photo movie generating device 2 links the generated photo movie with the specified music ID 61 and transmits it to the user terminal 3 (step S116).

When the user terminal 3 receives the photo movie sent from the photo movie generating device 2 (step S117), it plays the photo movie.

The user terminal 3 then requests the music distribution server 6 to transmit the audio data 62 of the above-mentioned music ID 61 to be used in the photo movie (step S118). When the music distribution server 6 receives a request to transmit the audio data 62 from the user terminal 3 (step S119), it transmits the audio data 62 of the requested music ID 61 to the user terminal 3 (step S120).

When the user terminal 3 receives the sound source data 62 (step S121), it plays the photo movie in synchronization with the sound source data 62 (step S122). By playing the photo movie and the sound source data 62 in synchronization with each other, the user terminal 3 plays a photo movie in which the screen changes in time with the music.

As described above, in the photo movie generation system 1 of this embodiment, when a request to generate a photo movie is sent from the user terminal 3 to the photo movie generation device 2, the photo movie generation device 2 obtains images of the user taken at photo spots A, B, C, etc. from the image storage 5, etc., generates a photo movie from the images that is played along with music related to photo spots A, B, C, etc., i.e., theme parks, and sends it to the user terminal 3. Since the photo movie combines images taken at a theme park with music related to the theme park, it is possible to strongly remind the user of the theme park, allowing the user to fully enjoy their memories.

The photo movie generating device 2 of this embodiment can also generate impressive photo movies in which the screen changes in accordance with the characteristics of the music, and can generate image materials to be used in the photo movie from the user's images, so that a more interesting and fun photo movie can be provided to the user.

In addition, the photo movie generating device 2 accumulates and stores movie configuration information 223 for songs for which photo movies have been generated previously, and generates photo movies with different configurations, so that photo movies with different expressions can be provided for the same song, providing the user with a variety of entertainment.

Furthermore, the photo movie generating device 2 generates a photo movie without music and plays it synchronously with the sound source data 62 on the user terminal 3 side, so that the file size can be reduced by not adding the sound source data 62 to the photo movie, and storage capacity and communication capacity can be reduced.

In addition, the user terminal 3 can specify the music and images to be used in the photo movie, so a photo movie can be generated using music and images that the user prefers.

However, the present invention is not limited to the above embodiment. For example, in this embodiment, the photo movie generating device 2 transmits a photo movie without music to the user terminal 3, but a photo movie with music may also be transmitted to the user terminal 3. A photo movie with music is a photo movie to which audio data 62 of a piece of music specified by the user terminal 3 is added. In other words, it is a photo movie in which the audio data 62 is played together with the video.

In this case, the photo movie generating device 2 receives and acquires the audio data 62 corresponding to the music ID 61 specified by the user terminal 3 from the music distribution server 6 in S109, and generates a photo movie in S115 in which the music and video of the audio data 62 are played in sync. The user terminal 3 plays this photo movie, and the playback method may be streaming playback or download playback.

In addition, in this embodiment, the music distribution server 6 stores the acoustic feature data 64 obtained by analysis in advance, but the photo movie generating device 2 may perform acoustic analysis to obtain the acoustic feature data 64 each time it acquires the sound source data 62.

Furthermore, the operation screen 8 in FIG. 9 is only an example, and the operation screen for specifying images, etc. is not limited to this. For example, in FIG. 9, the image, music, and photo movie type are specified on the same screen, but each may be specified on a separate screen.

The photo movie production system 1 of the present invention can be applied not only to theme parks, but also to tourist spots, event venues, etc., and can be applied to any place or area where photography is done at a photography spot, without any particular limitations.

The above describes preferred embodiments of the present invention with reference to the attached drawings, but the present invention is not limited to these examples. It is clear that a person skilled in the art can come up with various modified or revised examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention.

1: Photo movie production system 2: Photo movie production device 3: User terminal 4a, 4b, 4c: Photography device 5: Image storage 6: Music distribution server 7: Network 21: Music acquisition section 23: Image acquisition section 24: Image material production section 25: Photo movie production section 31: Music designation section 32: Image designation section 33: Photo movie production request transmission section 34: Reception section 35: Sound source data acquisition section 36: Playback processing section 51: User ID
52: Image ID
53: Image data 60: Music database 61: Music ID
62: Sound source data 63: Music information 64: Acoustic feature data 71, 72, 73: Photo movie 221: Effect template 222: Effect correspondence table 223: Movie configuration information

Claims (10)

A photo movie production system in which a photo movie production device and a user terminal are communicably connected via a network,
The photo movie generating device comprises:
storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit;
an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal;
a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal;
a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal;
Equipped with
The photo movie generating means includes:
generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means;
When generating a new photo movie to be played along with the music, the photo movie generation system determines whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, generates a new photo movie with a pattern different from the pattern of the movie configuration information . The photography spot is set up in a theme park, a tourist spot, or an event venue,
The photo movie production system includes:
An image storage that stores an image taken by a user at the photography spot in association with the user ID of the user;
A photography device is provided at the photography spot and takes a photograph of the user;
Further comprising:
The photographing device reads the user ID when photographing the user,
The photo movie generation system described in claim 1, characterized in that when a user visits a theme park, tourist attraction, or event venue multiple times, the image storage stores images taken with the photography device during each visit, linked to the same user ID. The photo movie generation system according to claim 1 or claim 2, characterized in that the image acquisition means acquires images provided by businesses of the visited places in addition to images taken by the user at photo spots in the visited places. the photo movie generating means generates the photo movie without adding a sound source of the music, associates the photo movie with identification information of the music, and transmits the photo movie to the user terminal;
The photo movie production system described in any one of claims 1 to 3, characterized in that the user terminal is equipped with a playback means for acquiring audio data of a song corresponding to the identification information and playing the acquired audio data in synchronization with the photo movie. The photo movie production device further comprises image material generation means for extracting an object from the image acquired by the image acquisition means and generating an image material;
5. The photo movie generating system according to claim 1, wherein the photo movie generating means generates the photo movie using the image material generated by the image material generating means. The photo movie production system according to any one of claims 1 to 5, characterized in that the user terminal is provided with a music selection means for accepting a selection of music to be used in the photo movie and transmitting the selection information of the music to the photo movie production device together with a request to generate the photo movie. The photo movie production system according to any one of claims 1 to 6, characterized in that the user terminal is provided with an image specification means for accepting the specification of images to be used in the photo movie and transmitting the image specification information to the photo movie production device together with a request to generate the photo movie. A photo movie generating device communicably connected to a user terminal via a network,
storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit;
an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal;
a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal;
a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal;
Equipped with
The photo movie generating means includes:
generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means;
When generating a new photo movie to be played along with the music, the photo movie generating device determines whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, generates a new photo movie with a pattern different from the pattern of the movie configuration information . A photo movie generating method in a photo movie generating system in which a photo movie generating device and a user terminal are communicably connected via a network, comprising:
The photo movie generating device stores movie configuration information including effect patterns, which indicates a configuration of changes in screens in sync with music in the generated photo movie, in a storage unit .
receiving a request to generate a photo movie from the user terminal, acquiring images taken by the user at a photography spot to be used in the photo movie;
receiving a request to generate a photo movie from the user terminal, acquiring acoustic feature data of a song related to the photography spot;
generating a photo movie using the images to be played on the user terminal together with music related to the shooting spot, and transmitting the photo movie to the user terminal;
Run
The photo movie generating device comprises:
generating a photo movie in which the screen changes in accordance with music based on the acoustic feature data;
A photo movie generation method characterized in that, when generating a new photo movie to be played along with the music, it is determined whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, a new photo movie having a pattern different from the pattern of the movie configuration information is generated . a photo movie generating device that is communicably connected to a user terminal via a network and stores movie configuration information including effect patterns that indicate a configuration of screen changes in sync with music in a generated photo movie;
an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal;
a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal;
a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal;
A photo movie generating device comprising:
The photo movie generating means includes:
generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means;
A program for functioning as a photo movie generating device, characterized in that when generating a new photo movie to be played along with the music, the program determines whether the movie composition information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie composition information exists, generates a new photo movie with a pattern different from the pattern of the movie composition information .

Images