JP6769616B2 - Sheet music provision method and program - Google Patents

Description

The present invention relates to a score providing method and a program, and more specifically, to provide and manage a score, and to provide a comprehensive musical experience by associating the provided score with a video of a performance. Regarding the method and program of providing music scores.

In recent years, the digitization of sheet music has progressed, and it has been provided, managed, and used in the form of electronic content. Not only newly created music, but also music that was provided as paper music in the past. Many of them have also been provided as electronic sheet music. For example, many classical music and the like have already been converted into electronic musical scores by companies and individuals as electronic musical scores without any copyright problems.

In such a situation, companies and sites that organize and provide electronic sheet music that is being accumulated, or posting sites that post digitized sheet music by individuals and companies have appeared, and moreover, simply provide electronic sheet music. Not only that, a system for identifying electronic musical scores while devising a search for music has also been proposed.

For example, for the purpose of providing a score display system that can easily obtain information on the score of music of a video, a plurality of video data representing each video including music is stored, and the selected video data is supplied. The moving image data supply unit MP, the musical score data supply unit SP that stores a plurality of musical score data representing the musical score and supplies the selected musical score data, one moving image data among the plurality of moving image data, and the plurality of said. The score data search unit SF that acquires the score data of the above and searches for the score data corresponding to the music of the video represented by the acquired video data among the plurality of score data, and displays the video based on the video data. A musical score display system including a musical score data corresponding to the music of the moving image to be displayed, and an image display unit GD for displaying information related to the musical score data selected by using the search result of the musical score data search unit SF. Has been proposed (see Patent Document 1).

On the other hand, various videos are distributed from various video distribution sites such as posting sites such as YouTube, and viewers can enjoy streaming or downloading videos from such sites by connecting to the Internet or the like. In addition to being able to do it, you can post and publish videos that you have taken and acquired yourself, and why you can enjoy it with your acquaintances or send it to the public.

Japanese Unexamined Patent Publication No. 2004-233698

However, the conventional score display system searches for the score data corresponding to the music of the video represented by the acquired video data, and displays the information about the searched score data together with the video, and simply the video being viewed. You can only watch the score of the song at the same time as the video. That is, such a conventional system simply searches for and provides a score for playing the music for a song that the user happens to be interested in. For example, in a series of services that provide the score. There is a problem that it does not provide a comprehensive music experience such as a music performance environment by associating videos with each other.

The present invention has been made in view of the above-mentioned conventional problems, and not only provides an electronic musical score, but also provides a service that supports various experiences desired by a user in relation to the provided musical score. For example, if the user receives the electronic score and the user provides the information of the video of playing the score with respect to the provided electronic score, the score and the video can be associated with each other for viewing and practice. The purpose is to provide music score providing methods and programs that support the improvement of music scores.

The invention according to claim 1 is a method of providing a musical score in association with a moving image in a system including a musical score providing system for providing a musical score and a moving image providing server for providing a moving image, and is a terminal via a network. In order to generate the correspondence information between the score transmission step of transmitting the score to the score, the position on the transmitted score, and the playback time of the video played in relation to the score, the identification information for receiving the video is provided. A video identification step received from the terminal, a score video mapping step that acquires a video using the identification information transmitted from the terminal and stores the correspondence information generated from the score and the video in a database, and a performance from the terminal. When the specified start position of is received, the playback time of the video corresponding to the specified position on the score is acquired from the generated correspondence information, and the video is played from the playback time of the video acquired on the video providing server. It is characterized by including a reproduction instruction step for instructing.

The invention according to claim 2 is a moving image posting step in which, in the method according to claim 1, a moving image played in relation to a musical score is transmitted from a terminal to a moving image providing server, and identification information of the transmitted moving image is received. It is characterized by further providing.

According to the third aspect of the present invention, in the method according to the first aspect, the video providing server is accessed from the terminal, the video played in relation to the score is selected, and the identification information of the selected video is received. It is characterized by further including a moving image selection step.

The invention according to claim 4 is a method of providing music in association with music in a system including a music score providing system for providing music and a content providing server for providing music, and is a terminal via a network. In order to generate the correspondence information between the score transmission step of transmitting the score to the music, the position on the transmitted music, and the playing time of the music played in relation to the music, the identification information for receiving the music is provided. A music identification step received from the terminal, a score music mapping step that acquires music using the identification information transmitted from the terminal and stores the correspondence information generated from the score and the music in a database, and a score music mapping step from the terminal. When the designation of the start position of the performance is received, the playback time of the music corresponding to the position on the specified score is acquired from the generated correspondence information, and the music playback time from the music playback time acquired in the content providing server is acquired. It is characterized by including a reproduction instruction step for instructing reproduction.

The invention according to claim 5 is a method of managing the display of a musical score on a terminal, in which a step of receiving a designation of a musical score to be displayed from the terminal via a network and a performance related to the designated musical score are performed. is a video identification step for acquiring identification information of the moving images, a reproduction instruction step for instructing the reproduction of the moving image by the identification information acquired in the video providing server receives the current playback time from the moving image distribution server, the videos received It is characterized by including a position acquisition step for acquiring a position on a score corresponding to a playback time, and a terminal display step for displaying only a part of the score including the acquired position on the terminal.

According to the present invention, in order to generate association information between a score transmission step of transmitting a score to a terminal via a network, a position on the transmitted score, and a playback time of a moving image played in connection with the score. The identification information for receiving the moving image is acquired by using the moving image identification step for receiving the moving image from the terminal and the identification information transmitted from the terminal, and the correspondence information generated from the score and the moving image is stored in the database. When the score video mapping step and the designation of the start position of the performance are received from the terminal, the playback time of the video corresponding to the specified position on the score is acquired from the generated mapping information and acquired by the video providing server. Since it is equipped with a playback instruction step that instructs the playback of the video from the playback time of the video, when the user provides the information of the video of the performance of the electronic score with respect to the electronic score provided from the server, the score and the video are displayed. Based on the organic correspondence of, it is possible to provide users with various experiences related to listening to music and practicing, and organic services that support the improvement of overall usability, not just the provision of sheet music. Can be provided.

It is a whole system block diagram of one Embodiment of this invention. It is a figure which shows the flow of a series of services of the whole system of one Embodiment of this invention. It is a functional block diagram of the server of one Embodiment of this invention. It is a figure which shows the association of the bar on the musical score and the corresponding reproduction position in one Embodiment of this invention. It is a flowchart which shows the process of acquiring the association of the bar on the musical score and the corresponding reproduction position of one Embodiment of this invention. It is a figure for demonstrating the operation which instructes the moving image reproduction from the score display screen of one Embodiment of this invention. It is a figure for demonstrating the display of the position on the musical score corresponding to the reproduced image on the moving image of one Embodiment of this invention. It is a figure which shows the association of the bar on the musical score and the corresponding reproduction position in another embodiment of this invention. It is a flowchart which shows the process which specifies the moving image reproduction position of another embodiment of this invention. It is a figure for demonstrating the operation which instructes the moving image reproduction from the score display screen of another embodiment of this invention. It is a figure for demonstrating the display of the position on the musical score corresponding to the reproduced image on the moving image of another embodiment of this invention. It is a figure for demonstrating the process which manages the display of the musical score on the terminal screen of another embodiment of this invention.

Hereinafter, embodiments of the music score providing method and program of the present invention will be described with reference to the drawings. Even in different drawings, the same reference numerals are used to indicate the same processing and configuration.

In the present embodiment, when providing an electronic score via the Internet and then viewing a performance video of a song posted on a video posting site, the start position of video playback can be specified, or the entire performance of the image being played can be specified. In the following embodiments, a musical score or a total score is used to display the position in, but the present invention is not limited to this, and any form can be displayed as long as some drawing showing the progress of the performance can be displayed on the terminal. But it can be handled. Here, basically, on the premise that some of the performers who posted the video are associated with the score used for the performance and set in the server in advance, the following embodiments will be described, but the present invention is not limited to this. This embodiment can be implemented if the music being played and the score are substantially the same. For example, a person who recognizes a video distributed for some reason can set it on the server in advance in association with the score corresponding to the music being played, and the score can be set by other methods known in the present technical field. Can be associated with the video.

In addition, it is not necessary for the user who received the score to post to the video site, but the video obtained by searching for the performance video of the music of the score obtained by the user who received the score on the posting site. There may be. In other words, when the score of a certain song, including classical music, is published, the video of the performance of that song is often shot and posted by one of the performers, and in fact, the same song has multiple videos. Is often present. Therefore, even if the user who receives the electronic score does not post the video, the user can easily find such a video and obtain the URL of the video by searching the Internet or the like. By specifying such a URL, the server can access the performance video of the score provided to the user.

Further, the musical score data of the electronic musical score used in the present embodiment is MusicXML, and is basically stored in the MusicXML data format in the server. The score written in MusicXML can be sent to the terminal as it is and displayed on the browser, or it can be converted to another display format on the server, for example PDF, and sent to the terminal for display on the browser of the terminal. .. As described above, MusicXML is used for the score data of the present embodiment, but the present invention is not limited to this, and any format can be used as long as it can generate a score for display by performing some conversion or processing.

(System configuration)
The operation and processing of a specific system of the score providing method used in one embodiment of the present invention will be described below. FIG. 1 is an overall system configuration diagram of an embodiment of the present invention. This system includes a score providing server 101, a communication server 102, and a video distribution server 104 as servers for distributing score data and controlling the entire system, and are connected via a network 103. .. The terminal 111, which is a device for displaying a score, is basically wirelessly connected to the network 103, and can be, for example, a tablet terminal. In the present embodiment, the connection with the network 103 is a mobile phone line or It is performed by a wireless network such as Wi-fi or BLUETOOTH (registered trademark). Here, the terminal 111 can be a tablet terminal, a smartphone, a mobile personal computer, or the like, but basically, the score data or the like is received from the server to display an image, and a certain input operation is performed using the touch panel, mouse, or keyboard. If you can do this, you can use any device such as a mobile or desktop personal computer or a dedicated terminal. Further, the terminal 111 is not limited to the display of the score, and can be used to execute various processes related to the access to the score providing server of the user and input / output data.

In the present embodiment, the moving image playback terminal 112 reproduces and displays the moving image from the playback starting point of the moving image specified by the terminal 111, and can communicate with the server via the network like the terminal 111. For example, it is not limited to a personal computer, and any terminal device can be used, and a mobile terminal such as a tablet can also be used. Further, as the server of the present embodiment, three servers of the score providing server 101, the communication server 102, and the video distribution server 104 are provided, and the score providing server 101 gives instructions such as displaying an image such as a score and starting playback from the terminal 111. When transmitted to, the score providing server 101 transmits score data, calculates the position of the specified playback position on the video, and instructs the communication server 102 to play the video, and as a result, the video distribution server. 104 transmits and displays the instructed moving image to the instructed display device 112, but is not limited to this, and the score providing server 101 from receiving the instruction to instructing the moving image distribution server 104 to transmit the image data. Or it can be shared by more servers.

Here, the video distribution server 104 may be a posting site such as YouTube, or may simply be a server of a site that distributes video. Further, although described as a video distribution server in the present embodiment, since it is essentially related to the performance of a musical score, only music can be distributed as some content distribution server without images. In this case, it goes without saying that all the following descriptions of the present embodiment also apply to the content distribution server that distributes music without moving images. On the contrary, if it is a moving image, it can be applied to a moving image without sound.

In the present embodiment, the terminal 111 and the moving image playback terminal 112 are described as completely different terminals, but the same terminal can be used as long as each function is achieved. That is, when a moving image playback instruction is transmitted from the same terminal, moving image data for starting playback from a start position designated to the same terminal is transmitted, and the moving image can be viewed together with the score. In this case, the image of the musical score and the moving image can be displayed in combination by any method known in the present art. Specifically, the image of the terminal can be divided into upper and lower parts or left and right to be displayed, and a part of the image can be displayed in an overlapping manner so that the viewer can easily display the image. Of course, even in this case, the display of the score, the instruction for playing the moving image, and the like are processed by the server or the like of the present embodiment, and the moving image is distributed from the moving image distribution server.

FIG. 2 is a diagram showing a series of service flows of the entire system according to the embodiment of the present invention. In the present embodiment, starting from the process of providing an electronic score (arrow 211), a comprehensive service related to the score is provided to the user, and a series of steps thereof is shown in FIG. As shown in FIG. 2, the user 203 performs a performance using the obtained musical score, captures the state with a video camera 202 or the like, uploads it to the video distribution server 104 (arrow 212), and receives the video ID. (Arrow 213). The acquired video ID is transmitted to the communication server 102 (arrow 214), a bar time conversion table or the like is created based on the video and the score obtained by the communication server 102, and the electronic score provided by the score providing server 101 is used. Provide users with services associated with videos.

FIG. 3 is a functional block diagram of a server according to an embodiment of the present invention, and FIG. 4 is a diagram showing an association between a bar on a musical score and a corresponding reproduction position in the present embodiment. The score providing server 101 receives score data from a server (not shown) or the like, stores the score data, manages the score data management module 301, performs necessary processing on the received score data, and transmits the received score data to the terminal. When the specification of is received, the position on the video is calculated, and the position of the currently played image in the entire video is received from the communication server 102 and the position of the score displayed on the terminal is calculated. It also includes a score data transmission module 302 for outputting highlights, and a score-related data reception module 303 for receiving various electronic score-related data used in the present embodiment, such as a bar time conversion table. Further, the communication server 102 is a video management module 304 that manages data such as a URL of a video that plays a musical score displayed on the terminal 111, and a video / identification information acquisition that acquires video data when the video identification information is received. Upon receiving the designation of the start position of the moving image from the module 305 and the score providing server 101, the starting position for instructing the moving image distribution server 104 to play the starting position based on the moving image data managed by the moving image management module 304. A measure time conversion table is created based on the designated module 306, the playback time output module 307 that outputs the current playback time received from the video distribution server 104 to the score providing server 101, and the correspondence between the acquired video and the electronic score. The measure time conversion table creation module 308 is provided.

Hereinafter, the preparation process for the moving image reproduction service from the provision of the electronic musical score according to the embodiment of the present invention will be described with reference to FIGS. 3 to 5. In general, musical score data of electronic musical scores, for example, MusicXML format data, specify note data for each bar, and when the terminal receives the musical score data, a staff notation is displayed on the browser and the note data of each bar is displayed. Draw the score based on it. Therefore, when a certain measure is specified by touching it with a finger or clicking it with a mouse on the browser displaying the score data, it is possible to recognize the number of measures. In the first embodiment described later in the video playback service of the present application, the score is displayed on the browser of the terminal based on the score data in this way, and a predetermined measure on the displayed score is specified to obtain a moving image. The playback start position of the above is specified, and the instruction for playing the moving image is transmitted to the score providing server 101. The score providing server 101 calculates the playback position of the moving image from the designated bar, and in the present embodiment, calculates the playing time from the beginning of the moving image, and instructs the distribution server 102 to play the moving image.

Further, when the score data cannot be displayed as it is on the terminal because there is no plug-in or the like, the score data can be transmitted as image data, for example, PDF format data from the score providing server 101 and displayed on the browser (second embodiment). In the second embodiment, which will be described in detail after the first embodiment, the score data providing server 101 generates PDF score data from the score data, and performs a moving image reproduction start process based on the PDF data.

In order to provide the video playback services of the first and second embodiments described later as described above, an electronic score is first provided to the user, and finally the designated video is associated with the score, specifically, FIG. It is necessary to create a measure time conversion table as shown in. This process will be described with reference to the flowchart shown in FIG.

(First Embodiment)
FIG. 5 is a flowchart showing a process of acquiring the association between the measure on the score and the corresponding reproduction position according to the embodiment of the present invention. In the present embodiment, the process of providing the electronic score is started, and the user is provided with comprehensive services related to the score. First, the user desires the electronic score from the terminal 111, which is not shown, from the purchase screen or the request screen. When instructed by specifying the electronic score, the score providing server 101 transmits the score 201 (step 501). After that, the user 203 displays the obtained score on the terminal 111 or prints it with a printer, performs a performance using the displayed or printed score, and captures the state of the performance such as practice and production with a video camera 202 or the like. (Step 502), the captured video is uploaded to the video distribution server 104, and the video ID is received (step 503). Here, the moving image ID is identification information for identifying the moving image on the net. For example, in YouTube or the like, it is a URL in which the moving image is stored, but the identification information is not limited to this and any identification information can be used.

When the user acquires the moving image ID, the user performs some association with the provided electronic musical score by any method known in the present technology and transmits the moving image ID to the communication server 102 (step 504). The communication server 102 also acquires the information for identifying the provided electronic score by the moving image / identification information acquisition module 305, acquires the moving image data by the moving image ID, and acquires the electronic score from the score providing server 101. (Step 505). The measure time conversion table creation module 308 calculates the correspondence between the measures of the score and the playback time of the moving image based on the acquired moving image and the electronic score (step 506).

In the present embodiment, the bar time conversion table 401 as shown in FIG. 4 is used to associate the playback time 411 of the moving image with the bar number 421 of the score, and by referring to the bar time conversion table 401, for example, the bar number ( 4) Since it can be determined that (measure 4) corresponds to the reproduction time t4 of the moving image, the reproduction time t4 can be calculated from this. That is, the measure time conversion table creation module 308 creates the measure time conversion table 401 by associating the measures of the score with the playback time of the moving image. Here, the bar time conversion table 401 can be automatically detected by any of the methods known in the present embodiment, or can be created by human support. The bar time conversion table 401 created in this way is transmitted to the score providing server 101 for the moving image playback service of the present embodiment (step 507).

As described above, the electronic musical score desired by the user is provided by the musical score providing method of the present embodiment, and after associating with the moving image instructed by the user, the moving image reproduction service of the first embodiment is provided to the user. To do. First, the video playback start process will be described, and then the process of highlighting the video playback position on the score will be described.

(Video playback start processing)
FIG. 6 is a diagram showing an operation of designating a position on the score of the present embodiment.

In the present embodiment, as shown in FIG. 6, the score data transmission module 302 transmits data in MusicXML format to the terminal 111, displays it on the browser to specify the playback position, and informs the server of the specified measure. As a result, the server calculates the playback position of the moving image corresponding to the measure, the playback time in the present embodiment, and instructs the communication server 102 to play the moving image from that position. The communication server 102 identifies the performance video of the musical score displayed on the terminal, instructs the video distribution server 104 to distribute the video by designating the instructed playback time, and the video is played on the display device 112. .. Normally, MusicXML is handled on a terminal using a dedicated display application or playback application known in the present technical field, so details are not described here, but for example, a score is displayed, and a touch with a finger or a click with a mouse, etc. Then, it is possible to embed a script or the like that sends the bar number corresponding to the position on the screen to the server so that the bar number is sent from the terminal to the server, or other present technology. By executing such a process by any method known in the field, the present embodiment can be implemented without particularly changing the software on the terminal side. On the contrary, the present embodiment is implemented by pre-installing or downloading software that executes necessary processing in the present embodiment, such as sending a bar number to the server to the terminal side, so that the software can be executed on the terminal. You can also.

Specifically, first, as shown in FIG. 6, when the user touches and specifies the fourth bar 602 at the top of the score 605 displayed on the terminal 111 with the finger 606, the terminal 111 starts playing the moving image. Get the number. When the bar number is acquired, the terminal 111 transmits the acquired bar number, here, the fourth bar, to the score providing server to notify the score (arrow 602). The score providing server 101 calculates the playback time of the corresponding moving image from the acquired bar number.

In the present embodiment, as shown in FIG. 4 above, the bar time conversion table 401 is created by associating the playback time 411 of the moving image with the bar number 421 of the score in advance. The reproduction time can be calculated by referring to the measure time conversion table 401. The playback time of the moving image is the length of the elapsed playing time based on the first image of the moving image, and the playing time of the last image of the moving image is the length of the entire moving image.

In the present embodiment, the measure time conversion table 401 is created in advance to calculate (convert) the required playback time, but if the time of each measure can be regarded as constant, the playing time and the number of measures are one measure. The playing time can be calculated and the playing time can be calculated from the bar number. Also, since the video usually has a margin that is not played before and after the performance, the performance is started from the position where the sound of a certain volume is played, starting from the first image of the video, and the volume is constant. It is also possible to calculate the total playing time assuming that the performance has ended when the following points are met.

When the score providing server 101 acquires the reproduction time from the bar number, it notifies the communication server 102 of the reproduction time as shown by the arrow 602 in FIG. 6 and instructs the communication server 102 to reproduce the moving image. In the communication server 102, the video management module 304 manages the score data managed by the score providing server 101 in association with the moving image of playing the music of the score, and the score providing server 101 manages the score and the playback start position. When playback is instructed by designating (for example, playback time t4), the video management module 304 identifies the video distribution server 104 that distributes the corresponding video, and the start position designation module 306 indicates by the arrow 604 in FIG. The video distribution server 104 specified in is instructed to distribute the video by designating the playback start position. The video distribution server 104 instructed to distribute the video distributes the designated video from the designated playback position t4 to the video playback terminal 112 as shown by the arrow 603 of FIG. Generally, the video distribution server is provided with an interface for data acquisition and operation according to the type, and although not described in detail in the present embodiment, a playback position is specified using an interface known in the present technical field. To start playback.

(Sheet music highlight display processing)
The score highlighting process of the present embodiment will be described below with reference to FIG. 7. As described above, in the present embodiment, the MusicXML format data is managed by the score providing server 101, transmitted to the terminal 111, and displayed as a score on the browser. At this time, in the present embodiment, the position on the score corresponding to the image being reproduced on the moving image display terminal 112 is highlighted on the terminal 111. As a result, the part of the performance currently being played in the moving image can be known in bar units of the score, which can be useful for checking the performance and practicing.

FIG. 7 is a diagram for explaining the display of the position on the score corresponding to the reproduced image on the moving image of the present embodiment.

More specifically, the score highlight display process will be described. First, as shown by the arrow 703 in FIG. 7, the moving image is played by streaming distribution from the moving image distribution server 104. In the present embodiment, the YouTube URL and, if necessary, the playback time are set using the YouTube API and played. In such a state, the reproduction time output module 307 of the communication server 102 requests the reproduction time from the moving image distribution server 104 as shown by the arrow 705 of FIG. 7, and acquires the reproduction time as shown by the arrow 704 of FIG. Generally, the video distribution server is provided with an interface for data acquisition and operation according to the type of the video distribution server, and although not described in detail in the present embodiment, the current playback time using an interface known in the present technical field is used. To get.

When the reproduction time output module 307 acquires the reproduction time, the reproduction time output module 307 notifies the score providing server 101 of the reproduction time. The score providing server 101 acquires the bar number as the position of the corresponding score by using the bar time conversion table 401 shown in FIG. 4 based on the notified playback time. Although not shown, if the reproduction time t acquired in this example is t2 or more and less than t3, it is determined by the measure time conversion table 401 that it corresponds to the second measure. In this way, the corresponding bar number is determined depending on which time interval range of the playback time in the bar time conversion table 401 the acquired playback time is included.

When the bar number is determined, the score providing server 101 instructs the terminal 111 to highlight the determined bar number, and the terminal 111 highlights the second bar. Regarding the specific processing, the highlight display can be controlled by any method known in the present technology. For example, the score providing server 101 changes the display of the score of the determined bar number to highlight. By sending to the terminal 111, the browser of the terminal 111 can interpret this and display the highlight, or when the server receives an instruction to switch to the highlight display, the specified part is highlighted. By embedding a script to change in the data in advance, it is possible to highlight the specified measure when receiving a highlighting instruction by specifying the measure number from the server, but it is not limited to this. ..

By the above processing, the position on the score corresponding to the performance of the image currently being played is determined in bar units, and the determined bar is highlighted. Here, the communication server 102 executes the above-mentioned acquisition of the playback time at regular intervals and performs the highlight display processing thereafter, so that the corresponding measures are highlighted according to the playback of the moving image, and the score is obtained. It can be displayed as if it is moving on. By making the fixed interval sufficiently smaller than the time between each bar, the highlight can be displayed so as to move smoothly on the score. Alternatively, even if a fixed interval is not used, in the present embodiment, the playback time corresponding to each measure is stored in advance as shown in FIG. 4, so that the notification server 102 actually sets the playback time at each playback time. It is possible to confirm with the video distribution server 104 whether or not there is, and execute a process of changing the bar to be highlighted.

In the present embodiment, the playback position of the moving image is highlighted, but the present invention is not limited to this, and it is possible to indicate it with some kind of marker or display the measure in some way to distinguish it from other measures.

(Second Embodiment)
This embodiment basically has the same configuration as the first embodiment described above, and the functions and configurations of each terminal are also the same. In particular, the moving image display terminal 112, the communication server 102, and the moving image distribution server 104 have the same configuration and processing. In the first embodiment, when displaying the score on the terminal, the score data is transmitted to the terminal (in MusicXML format), and the terminal interprets the score data as MusicXML and displays the score, whereas in the present embodiment, the score is displayed. The difference is that the music score PDF data is generated based on the image data and transmitted to the terminal, and the terminal side displays it as a PDF file on the browser. The main differences between the present embodiment and the first embodiment are as described above. Therefore, since the position on the score is represented by the coordinates instead of the measures, in addition to the bar time conversion table used in the first embodiment. , The coordinate conversion table will also be used.

FIG. 8 is a diagram showing the association between the measures on the score and the corresponding reproduction positions in the present embodiment. In the present embodiment, since the score is transmitted to the terminal as image data, the bar position cannot be specified as in the first embodiment. Therefore, in order to associate the score displayed on the terminal with the playback time, it is necessary to prepare the coordinate measure conversion table 801 in advance in addition to the measure time conversion table 401 as shown in FIG. FIG. 9 is a flowchart showing a process of acquiring the association between the measure on the score and the corresponding reproduction position of the present embodiment.

In the present embodiment, the score data management module 301 receives and manages the score data from a server or the like (not shown) in advance, and manages the score image in PDF format generated from the score data. Module 301 separates the images for each page, stores and manages them (step 901). When an instruction to transmit image data is given from the terminal 111 by designating a score or a moving image to the score providing server 101, the score providing server 101 reads the image of the instructed score and transmits it to the terminal 111. At this time, the position on the score image of each bar is acquired, associated with the bar number, and the coordinate bar conversion table 801 is generated. Specifically, in the present embodiment, the length to the lowest point of the area of the bar is set to the height H, and the length to the boundary with the adjacent bar in the music traveling direction is set based on the coordinates of the upper left of each bar. When the width W includes the coordinates in this area, it is determined to be the measure number. The calculated conversion table is stored.

This process is executed at the time of generating the score image of the PDF, but the present invention is not limited to this, and can be executed at any timing after the PDF is generated as long as it is in time for the reproduction of the moving image.

Next, when the user gives an instruction by specifying a desired electronic score from the purchase screen or request screen of the electronic score of the terminal 111 (not shown), the score providing server 101 transmits the generated score image (step 902).

After that, the user 203 displays the obtained score on the terminal 111 or prints it with a printer, performs a performance using the displayed or printed score, and captures the state of the performance such as practice and production with a video camera 202 or the like. (Step 502), the captured video is uploaded to the video distribution server 104, and the video ID is received (step 503). Here, the moving image ID is identification information for identifying the moving image on the net. For example, in YouTube or the like, the moving image is a URL in which the moving image is stored, but the identification information is not limited to this and any identification information can be used.

When the user acquires the moving image ID, the user performs some association with the provided electronic musical score by any method known in the present technology and transmits the moving image ID to the communication server 102 (step 504). The communication server 102 also acquires the information for identifying the provided electronic score by the moving image / identification information acquisition module 305, acquires the moving image data by the moving image ID, and acquires the electronic score from the score providing server 101. (Step 505). The measure time conversion table creation module 308 calculates the correspondence between the measures of the score and the playback time of the moving image based on the acquired moving image and the electronic score (step 506). The bar time conversion table 401 created in this way is transmitted to the score providing server 101 for the moving image playback service of the present embodiment (step 507).

Hereinafter, the moving image playback start process will be described with reference to FIGS. 8 and 10, and the highlight display process will be described with reference to FIG. 11. However, as described above, the process of the first embodiment is basically used except for the differences. be able to.

FIG. 10 is a diagram for explaining a moving image reproduction start process of the present embodiment. In the present embodiment, as shown in FIG. 10, the score data transmission module 302 transmits a score image in PDF format to the terminal 111 and displays it on the browser to specify the playback position, and the designated position coordinates are set to the server. The server determines which measure the coordinate position corresponds to, calculates the playback time of the corresponding moving image, and instructs the communication server 102 to play the moving image from the playback time. Subsequent processing is the same as in the first embodiment.

Specifically, first, as shown in FIG. 10, when the user touches and specifies the fourth bar at the top of the score 1003 displayed on the terminal 111 with the finger 606, the terminal 111 sets the coordinates at which the moving image starts to be played. get. When the score displayed on the screen of the terminal is touched with a finger or clicked with a mouse, any method known in the present technology can be used as a method of transmitting the touched or clicked coordinates to the server. For example, if you touch with your finger or click with the mouse, you can embed a script in the data that sends the touched or clicked coordinates to the server in advance, and install software that performs such processing on the terminal. Alternatively, it can be downloaded and used sequentially.

When the coordinates are acquired, the terminal 111 transmits the acquired coordinates to the score providing server 101 to notify the score (arrow 1002). The score providing server 101 determines the corresponding bar number from the acquired coordinates using the coordinate bar conversion table 801 and calculates the playback time of the corresponding moving image from the bar time conversion table 401.

In this way, the process after determining the corresponding measure number from the coordinates specified by the user notified from the terminal 111 is the same as that of the first embodiment. That is, by referring to the bar time conversion table 401 created in advance as shown in FIG. 8, for example, it can be determined that the bar numbers (4) (fourth bar) correspond to the playback time t4 of the moving image. The time t4 can be calculated. After that, the video distribution server 104, which has been instructed to distribute the video by performing the same processing as in the first embodiment, transmits the designated video to the video playback terminal 112 from the designated playback position t4 in FIG. Streaming is performed as shown by arrow 603.

(Sheet music highlight display processing)
The score highlighting process of the present embodiment will be described below with reference to FIG. As described above, in the present embodiment, the score image in PDF format is managed by the score providing server 101, transmitted to the terminal 111, and displayed as a score on the browser. At this time, as in the first embodiment, in the present embodiment as well, the position on the score corresponding to the image reproduced by the moving image display terminal 112 is highlighted by the terminal 111. As a result, the part of the performance currently being performed in the moving image can be known in bar units of the score, which can be useful for checking and practicing the performance.

More specifically, the score highlight display process will be described. First, the process from the communication server 102 to notifying the score playing time on the score providing server 101 is the same as that of the first embodiment. The score providing server 101 acquires a bar number as the position of the corresponding score by using the bar time conversion table 401 shown in FIG. 11 based on the notified playback time, and the coordinate measure conversion table 801 is obtained from the acquired bar number. Is used to determine the corresponding region. That is, in the present embodiment, as shown in the coordinate bar conversion table 801 of FIG. 8, the area of each bar when displayed on the terminal in the PDF format from the score data is determined in advance, and the bar number is designated. And the area on the score image of the bar can be specified by the coordinates. Therefore, although not shown, if the reproduction time t acquired in this example is t2 or more and less than t3, it can be determined from the measure time conversion table 401 that it corresponds to the second measure. Next, the area of the second measure is acquired by referring to the coordinate measure conversion table 801.

When the corresponding bar area is acquired, the score providing server 101 instructs the terminal 111 to highlight the display of the determined bar area, and the terminal 111 sets the highlight display. As for the specific processing, the highlight display can be controlled by any method known in the present technology. For example, the score providing server 101 changes the display of the score in the acquired bar area to highlight. By transmitting the score image in PDF format to the terminal 111, the browser of the terminal 111 can interpret this and highlight the area of the bar corresponding to the performance currently being played, or the server. By embedding a script in the data that changes the highlight display of the specified area when receiving an instruction to switch from to highlight display, the area corresponding to the measure is specified by the server, for example, by coordinates and highlighted. When a display instruction is received, the specified area can be highlighted, but the present invention is not limited to this. Further, it is also possible to install or sequentially download and use dedicated software for performing such processing on the terminal. In this case, for example, the data in the area to be highlighted can be managed separately from the image data of the score and sent to the dedicated software together with the image data of the score to be highlighted.

By the above processing, as in the first embodiment, the position on the score corresponding to the performance of the image currently being reproduced is determined in bar units, and the determined bar is highlighted. Since the process of moving the highlight display on the score according to the reproduction of the moving image is the same as that of the first embodiment, it is omitted here.

(Third Embodiment)
This embodiment is a process for introducing a sample of the service before the user obtains the score. The score providing service of the present invention is intended to provide various services other than the score itself to a user who desires the score, but it is possible to set a condition for providing the score to a user who has registered as a member. Here, for example, assuming that a registered user can receive the service of the present invention, in order to guide an unregistered person to registration, the service content is simulated and the user actually experiences the usefulness of the service. Is valid. When providing such a sample experience, it is more effective to have an experience close to this service if possible, but on the other hand, if the sample experience is too close to this service, the motivation to register may be diminished. Therefore, it is important to be able to fully perceive the usefulness of the service, while leaving the content that can only be obtained with this service.

In the present embodiment, any score can be viewed without limiting the number of scores provided by the score providing server in that sense, but by not displaying a part of the display, the performance or the like can be performed. Make it unavailable and effectively guide you to registration.

FIG. 12 is a diagram for explaining a process of managing the display of the score on the terminal screen of another embodiment of the present invention. The sample experience of the present embodiment is provided by displaying the score 1201 and the moving image 1202 playing the score 1201 on one screen as shown in the sample screen 1200 of FIG. In addition, as this service, it will be explained that the user confirmed on the sample screen purchases the score and provides full-screen display and download. Since the score 1201 is displayed together with the moving image 1202, some measures are hidden by the moving image and not all are displayed. However, some measures being played can be viewed by the measure time conversion table used in the above embodiment. .. Therefore, if the moving image 1202 is viewed to the end, the entire score can be checked, but since it is not possible to freely browse any place, it cannot be practically used for performance. In general, the description format of the score is slightly different depending on the editor, so even if it is the same song, the usability often differs. Therefore, as a performer, whether or not to check the entire score before using it. I want to decide. Therefore, even if it is a sample experience, it is necessary to be able to browse the whole. On the other hand, if the score can be browsed arbitrarily, the same content as this service can be provided. Therefore, in the present embodiment, the display is displayed so that a part of the score is displayed according to the playback of the moving image. Provide a sample trial version by managing it. In the case of music instead of video, a sample trial version can be provided by not displaying a part of the score in the cover photo of the music.

Explaining a specific process, in the present embodiment, when a user considering registration gives an instruction by searching for and selecting a desired electronic score from an electronic score search / selection screen (not shown) of a score video display terminal. , The score providing server 101 displays a part 1201 of the score. Then, the server causes the communication server 102 to specify the performance moving image of the displayed musical score, and instructs the moving image distribution server 104 to distribute the moving image to the musical score moving image display terminal. Here, the moving image 1202 can hide the score without discomfort by displaying the non-displayed portion of the score 1201 so as to hide it. Since the user can arbitrarily play the video by instructing the playback time 1203, etc., when the part specified by the user is played back, the corresponding bar is specified by the bar time conversion table, and only before and after that bar is displayed. To do so. At this time, as in the above-described embodiment, the bar being played can be highlighted as in the sample screen 1200.

More specifically, when the video 1202 is streamed from the video distribution server 104 and played back, for example, in the case of YouTube, the playback time is acquired using the YouTube API, and the score providing server 101 acquires the acquired playback. Based on the time, for example, using the bar time conversion table 401, the bar number is acquired as the position of the corresponding score. When the measure number is acquired, the score providing server instructs the score moving image display terminal to display only a predetermined measure before and after the determined measure number. By the above processing, the position on the score corresponding to the performance of the image 1202 currently being reproduced is determined in bar units, and only the score 1201 including a predetermined number of bars before and after the determined bar is displayed. Here, the communication server 102 can display the corresponding measures according to the reproduction of the moving image by executing the above-mentioned acquisition of the reproduction time at regular intervals and performing the display processing thereafter.

On the other hand, when the sample screen 1200 is browsed and, for example, it is decided to purchase the score, the score purchase screen 1210 is displayed by pressing a purchase button (not shown) or the like. On the purchase screen 1210, the input field 1211 of the purchase condition is displayed in the portion where the moving image 1202 is displayed, and the purchase process of the score being viewed can be performed. As a result, when the purchase process is performed, the full screen 1220 is displayed as this service, the score can be freely viewed, and the download can be permitted.

Here, the display management process of the present embodiment has been described as a sample experience for purchasing a musical score, but the present invention is not limited to this, and can be applied to, for example, guiding a free member to a paid member. That is, although free members can watch all the scores, if the use as scores is restricted by restricting the display of this embodiment, it is possible to guide the paid members who can freely obtain the scores. It is valid.

(Other aspects)
The present invention includes the following scope in addition to those described in the claims.

Aspect 1
A sheet music providing system that provides sheet music
A musical score transmission means for transmitting musical scores to terminals via a network,
A moving image identification means for receiving a moving image played in relation to the musical score from a moving image providing server and receiving identification information from the terminal for generating identification information for generating association information between the position on the musical score and the playing time of the moving image. When,
A musical score moving image mapping means for storing the mapping information generated by using the identification information transmitted from the terminal in a database, and
Upon receiving the designation of the start position of the performance, the playback time of the moving image corresponding to the designated position on the score is acquired from the generated association information, and the acquired moving image is sent to the moving image providing server. A score providing system including a playback instruction means for instructing playback of the moving image from the playback time.

Aspect 2
A program that executes a method of providing a score in association with a moving image in a score providing system that provides the score.
The score transmission step to send the score to the terminal via the network,
In order to generate association information between the transmitted position on the score and the playback time of the moving image played in relation to the score, the identification information for receiving the moving image is the moving image identification received from the terminal. Steps and
A musical score moving image mapping step of acquiring the moving image using the identification information transmitted from the terminal and storing the corresponding information generated from the musical score and the moving image in a database.
When the designation of the start position of the performance is received from the terminal, the playback time of the moving image corresponding to the designated position on the score is acquired from the generated association information, and the acquisition is obtained by the moving image providing server. A program characterized by including a playback instruction step for instructing playback of the moving image from the playing time of the moving image.

Claims (5)

It is a method of providing a score in association with a video in a system provided with a score providing system that provides a score and a video providing server that provides a video.
The score transmission step to send the score to the terminal via the network,
In order to generate association information between the transmitted position on the score and the playback time of the moving image played in relation to the score, the identification information for receiving the moving image is the moving image identification received from the terminal. Steps and
A musical score moving image mapping step of acquiring the moving image using the identification information transmitted from the terminal and storing the corresponding information generated from the musical score and the moving image in a database.
When the designation of the start position of the performance is received from the terminal, the playback time of the moving image corresponding to the designated position on the score is acquired from the generated association information, and the acquisition is performed by the moving image providing server. A method comprising a playback instruction step for instructing playback of the moving image from the playing time of the moving image. The first aspect of claim 1, further comprising a moving image posting step of transmitting the moving image played in relation to the musical score from the terminal to the moving image providing server and receiving the identification information of the transmitted moving image. the method of. A claim further comprising a moving image selection step of accessing the moving image providing server from the terminal, selecting the moving image played in relation to the musical score, and receiving identification information of the selected moving image. Item 1. The method according to item 1. It is a method of providing a score in association with music in a system provided with a score providing system for providing the score and a content providing server for providing music.
The score transmission step to send the score to the terminal via the network,
Music identification to receive the identification information for receiving the music from the terminal in order to generate the correspondence information between the transmitted position on the score and the playing time of the music played in relation to the score. Steps and
A musical score music mapping step of acquiring the music using the identification information transmitted from the terminal and storing the matching information generated from the score and the music in a database.
When the designation of the start position of the performance is received from the terminal, the playback time of the music corresponding to the designated position on the score is acquired from the generated association information, and the acquisition is performed by the content providing server. A method comprising: a reproduction instruction step for instructing the reproduction of the music from the reproduction time of the music. It is a method to manage the display of the score on the terminal.
The step of receiving the specified score to be displayed from the terminal via the network,
A video identification step for acquiring identification information of a video played in connection with the specified score, and
A reproduction instruction step of instructing the reproduction of the moving image by the acquired identification information to the moving picture providing server,
When the current playback time is received from the video providing server, the position acquisition step of acquiring the position on the score corresponding to the playback time of the received video and the position acquisition step.
A method characterized in that the terminal is provided with a terminal display step for displaying only a part of a musical score including the acquired position.