JP7755082B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing device and a program .

2. Description of the Related Art Conventionally, there is known a communication system that includes a terminal device and a plurality of external devices, and that displays a playback history of music and a usage history of operation components mounted on the external devices (see, for example, Patent Document 1).
In the communication system described in Patent Document 1, a terminal device assigns IDs to each external device and issues various commands to them. Examples of external devices include a DJ player and a DJ mixer. The DJ player is equipped with a jog dial, and the DJ mixer is equipped with a channel fader. The DJ player and DJ mixer are also equipped with controls such as knobs, buttons, and selector switches.

A function selection screen is displayed on the operation panel of the terminal device, and function buttons including a "play history" button and a "use history" button are displayed on the function selection screen.
The "Playback History" button is a button for obtaining the playback history of songs played on each DJ player from each DJ player. When the "Playback History" button is selected, a playback history display screen is displayed. The playback history display screen displays history information and ID information. The history information displays the playback history in chronological order. The playback history includes the date and time playback started, song title, artist name, label, and playback time.

The "Usage History" button is used to obtain the usage history of each operating component installed on the DJ mixer. The usage history includes the cumulative usage time (operation time or power-on time), the number of uses, operation log information, etc. When the "Usage History" button is selected, a remaining life display screen is displayed. The remaining life display screen displays a mixer image and ID information. The mixer image includes a fader image showing the channel fader and a panel image showing the touch panel, and color-coded markers corresponding to the remaining life are superimposed on the fader image and panel image.

JP 2014-99025 A

Although the communication system described in Patent Document 1 is capable of recording operation log information, the operation log information is only used to determine the lifespan of the DJ mixer, and there is a problem in that the operation log information is not fully utilized.
For this reason, there has been a demand for other ways to utilize the recorded operation log information.

An information processing device according to one aspect of the present invention includes a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music, a modification unit that can modify each of the acquired pieces of audio control information, a playback unit that plays the piece of music, and a reproduction unit that reproduces the plurality of audio control operations based on the modified pieces of audio control information for the played piece of music.

FIG. 1 is a diagram illustrating a configuration of an operation reproduction system according to an embodiment. 1 is a diagram illustrating an outline of functions of an operation reproduction system according to an embodiment. FIG. 1 is a block diagram showing the configuration of an information processing apparatus according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of a control unit according to an embodiment. FIG. 10 is a diagram showing a part of operation log information included in operation reproduction information according to an embodiment. FIG. 10 is a diagram showing an example of an operation screen according to an embodiment. FIG. 10 is a diagram showing an example of an operation display image according to an embodiment. FIG. 4 is a diagram showing a first operation state display section of an operation display image according to an embodiment. FIG. 10 is a diagram showing a second operation state display section of the operation display image according to an embodiment. FIG. 10 is a diagram showing a crossfader state display section of an operation display image according to an embodiment. FIG. 10 is a diagram showing an effect status display section of an operation display image according to an embodiment. FIG. 4 is a diagram showing the display content of a first fader state display section according to an embodiment. FIG. 4 is a diagram showing the display content of a first equalizer state display unit according to an embodiment. 10A and 10B are diagrams showing display contents of an effect status display unit according to an embodiment; FIG. 10 is a diagram showing a jog operation correction image according to an embodiment. FIG. 10 is a diagram showing a template image according to an embodiment. FIG. 10 is a diagram showing an example of a mix content image according to an embodiment. FIG. 10 is a diagram showing an example of an operation reproduction screen according to an embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[Configuration of operation reproduction system]
FIG. 1 is a diagram illustrating the configuration of an operation reproduction system 1A according to this embodiment.
As shown in Figure 1, the operation reproduction system 1A of this embodiment includes an operation device 2, an information processing device 3 that inputs operation information corresponding to input operations to the operation device 2, and a display device 4 connected to the information processing device 3.
The information processing device 3 is configured to be connectable to a network N such as the Internet, and is capable of communicating with servers SV1 and SV2 connected to the network N. The servers SV1 and SV2 are not limited to computers, but also include storage that can be used as a file server. An example of the server SV2 is a server for a music distribution service.
In this specification, DJ operations correspond to sound control operations and include, for example, operations to control the playback position of music, operations to process tone, and operations to control the volume of music. Examples of operations to control the playback position include scratching, variable speed playback, reverse playback, loop playback, and moving the playback position using the controls on the operation device. Examples of operations to process tone include equalizer (EQ) operations and effect (Fx) operations. Examples of operations to control the volume of music include fader operations such as channel faders and crossfaders.

[Overview of operation reproduction system]
FIG. 2 is a diagram illustrating an outline of the functions of the operation reproduction system 1A.
2, in the operation reproduction system 1A, the information processing device 3 records the details of the DJ operations as operation reproduction information based on operation information and song identification information input from the operation device 2 in response to the DJ operations of the user US on the operation device 2. The information processing device 3 is capable of editing the recorded operation reproduction information and is also capable of uploading the operation reproduction information to the server SV1 shown in FIG.
The operation reproduction information includes operation log information indicating the details of the DJ operations performed by the user US on the controls of the operation device 2, and song identification information for identifying the target song that is the target of the DJ operations. The song identified by the song identification information is the song that was the target of the operation when the operation reproduction information was recorded. Hereinafter, the song identified by the song identification information will be referred to as the target song.

The information processing device 3 can reproduce the DJ operation of the user US by acquiring pre-recorded operation reproduction information or operation reproduction information generated by another information processing device 3. Reproducing the DJ operation means reading the operation reproduction information and executing a process similar to the process corresponding to the operation information received from the operation device 2 while the DJ operation is being performed. For example, reproducing the DJ operation means that, when a controller is operated after a predetermined time has elapsed since the start of recording of the operation reproduction information, a process corresponding to the operation of the controller is executed after the same time has elapsed since the start of operation reproduction processing based on the operation reproduction information.
When reproducing the DJ operations, the information processing device 3 generates an image showing the details of the DJ operations based on the operation log information of the operation reproduction information using the second image generation unit 369 (described later) and displays the image on the display device 4. This visualizes the recorded DJ operations of the user. Furthermore, if the information processing device 3 holds a target song identified by the song identification information, the information processing device 3 plays the target song according to the details of the DJ operations based on the operation log information. This reproduces the playback of the target song according to the recorded DJ operations.
In this way, the operation reproduction system 1A can be said to be a DJ operation visualization system that visualizes the DJ operations of the user.
Hereinafter, each component of the operation reproduction system 1A will be described.

[Configuration of operation device]
1 and 2 is a DJ device such as a DJ controller or a DJ mixer, and is an audio control device having controls that allow DJ operations to be performed. Although detailed illustration is omitted, for example, the operation device 2 has various controls such as controls such as a jog dial, a crossfader, and a channel fader, as well as a rotary control that can adjust the volume level of a high frequency band, the volume level of a mid frequency band, the volume level of a low frequency band, the volume level of an input signal, and the application rate of an effect.

[Configuration of display device]
1 is configured by various display panels such as liquid crystal displays, etc. The display device 4 displays an image according to an image signal input from the information processing device 3.
In this embodiment, the information processing device 3 is configured as a notebook PC (Personal Computer), and therefore the display device 4 is integrated with the information processing device 3. However, this is not limiting, and the display device 4 may be a display separate from the information processing device 3.

[Configuration of information processing device]
FIG. 3 is a block diagram showing the configuration of the information processing device 3.
The information processing device 3 is configured, for example, by a PC capable of executing music management software. The information processing device 3 executes the music management software and plays pre-stored music based on operation signals input from the operation device 2. The information processing device 3 generates and stores operation reproduction information that records the details of DJ operations performed by the user US on the operation device 2. Furthermore, the information processing device 3 reproduces the DJ operations performed by the user US based on the operation reproduction information.
As shown in FIG. 3, the information processing device 3 includes an input unit 31, an output unit 32, a communication unit 33, a storage unit 34, a memory 35, and a control unit 36, which are electrically connected by a bus line BL.

The input unit 31 and the output unit 32 have terminals that can be connected to external devices.
The input unit 31 outputs signals input from external devices to the control unit 36. Examples of external devices connected to the input unit 31 include the operation device 2, a pointing device, a keyboard, and an external storage device that stores music data. The external storage device is included in the storage unit of the present invention. An example of a terminal that constitutes the input unit 31 is a USB (Universal Serial Bus) terminal.
The output unit 32 outputs information input from the control unit 36 to an external device. Examples of external devices connected to the output unit 32 include the display device 4, a speaker, and headphones. Examples of terminals constituting the output unit 32 include a USB terminal, a video output terminal, and an audio output terminal.

Under the control of the control unit 36, the communication unit 33 communicates with external devices such as servers SV1 and SV2 on the network N. For example, the communication unit 33 receives operation reproduction information from server SV1 and outputs it to the control unit 36, and also transmits operation reproduction information generated by the control unit 36 to server SV1. The communication unit 33 can also receive music data from a server SV2 that is different from server SV1. The information and data received by the communication unit 33 are stored in the memory unit 34 by the control unit 36. For example, music data received from server SV2 is stored in the memory unit 34.

The storage unit 34 is configured with storage devices such as a hard disk drive (HDD) and a solid state drive (SSD), and stores programs and data for operating the information processing device 3. For example, the storage unit 34 stores an operating system (OS) and music management software that control the information processing device 3, as well as an operation reproduction program for executing the operation reproduction process described below. In other words, the storage unit 34 is a recording medium that stores the operation reproduction program. Furthermore, for example, the storage unit 34 can store a plurality of pieces of music data and operation reproduction information.
The memory 35 is a work memory for the control unit 36 .

[Configuration of control unit]
FIG. 4 is a block diagram showing the functional configuration of the control unit 36.
The control unit 36 is configured with at least one processor such as a CPU (Central Processing Unit), and controls the operation of the information processing device 3. The control unit 36 has a plurality of functional units that are realized when the processor executes a program stored in the storage unit 34. For example, as shown in FIG. 4 , the control unit 36 has an operation acquisition unit 361, a first music acquisition unit 362, a first music playback unit 363, a reproduction information generation unit 364, a first image generation unit 365, a reproduction information acquisition unit 366, an identification information acquisition unit 367, a second music acquisition unit 368, a second image generation unit 369, a modification unit 370, a second music playback unit 371, a reproduction unit 372, and a conversion unit 373.

The operation acquisition unit 361 acquires an operation signal from among signals input from an external device connected to the input unit 31. For example, the operation acquisition unit 361 acquires an operation signal input from the operation device 2 connected to the information processing device 3.
The first music acquisition unit 362 acquires, from the storage unit 34, music data corresponding to the music selected by the user based on the acquired operation signal.

The first music playback unit 363 plays the music data acquired by the first music acquisition unit 362 based on the operation signal acquired by the operation acquisition unit 361. For example, when an operation signal corresponding to an operation on the jog dial of the controller device 2 is acquired, the first music playback unit 363 changes the playback speed and playback direction of the music based on the acquired operation signal. Also, for example, when an operation signal corresponding to an operation on the channel fader of the first channel on the controller device 2 is acquired, the first music playback unit 363 increases or decreases the volume level of the music being played on the first channel based on the acquired operation signal.

The reproduction information generation unit 364 generates operation reproduction information that records the content of the user's DJ operations on the operation device 2. Specifically, the reproduction information generation unit 364 generates operation reproduction information by recording the operation content corresponding to the operation signal input from the operation device 2 in chronological order from the start of recording. In other words, the reproduction information generation unit 364 generates operation reproduction information by recording the user's DJ operations on the operation device 2 in chronological order from the start of recording. The generated operation reproduction information is stored in the memory unit 34.

[Contents of operation reproduction information]
The operation reproduction information includes operation log information and music identification information, as shown in Fig. 2. The operation reproduction information is a file in a format that can be transmitted to the server SV1 or the like.

[Operation log information content]
FIG. 5 is a diagram showing a part of the operation log information LG included in the operation reproduction information.
The operation log information is information that records the operation details of operators operated by the user among the operators of the operation device 2 for each operator along with the elapsed time from the start of recording of the DJ operation. In other words, the operation log information includes a plurality of pieces of sound control information that record each of a plurality of sound control operations.
Specifically, as shown in Fig. 5, the operation log information LG is information in which records RC, each of which includes a record number RN, an elapsed time ET from the start of recording, a control type SC indicating the control that was operated, a channel number CN of the channel on which the operated control is located, and a control position SP indicating the position of the control after it has been operated, are arranged in the order of the record number RN. The control type SC, channel number CN, and control position SP correspond to operation content information.

The record number RN is, for example, an integer value. The record number RN is incremented each time an operation signal is input, and a record RC including the record number RN is generated by the reproduction information generation unit 364. The generated record RC is added to the operation log information LG. In other words, the record number indicates the order of the recorded operations. However, a record number is not necessary.
The channel number CN of the operator shared by each channel is "0." The channel number CN of the operator for the first channel is "1," and the channel number CN of the operator for the second channel is "2." An example of an operator shared by each channel is a crossfader.
The operator position SP of operators other than a jog dial is set to an absolute value within the operable range of the operator. On the other hand, since a jog dial can be continuously rotated clockwise and counterclockwise, the absolute position of the jog dial cannot be recorded. Therefore, if the operated operator is a rotary operator that can be rotated infinitely, such as a jog dial, the operator position SP is set to a relative position based on the position before the operation.

The operation log information includes time adjustment information TA that aligns the playback position of the music with the operation timing of the controls when reproducing the DJ operations. The time adjustment information TA is information for aligning the position on the time axis based on the elapsed time ET of the operation reproduction information with the playback position of the music. In other words, the time adjustment information TA is information that associates the elapsed time ET from the start of recording the DJ operations with the playback position of the music.
Specifically, the time adjustment information TA is at least one record RC including a record number RN, an elapsed time ET, and a playback position PP of the target song corresponding to the elapsed time ET. The playback position PP of the target song is, for example, a playback position from the beginning of the song and is indicated by time.
The time adjustment information TA is recorded every time a predetermined time has elapsed since the start of recording of DJ operations, and also every time an operation to change the playback position of a song is performed. For example, the time adjustment information TA is recorded when a jog dial that changes the playback speed and playback direction of a song is operated, and also when an operation to change the playback state of a song is performed. Examples of operations that change the playback state of a song include starting playback, stopping playback, reverse playback, and changing the tempo.

[Music Identification Information]
The song identification information is information for identifying a song whose playback is controlled by a DJ operation indicated by the operation log information LG. In other words, the song identification information is information that can uniquely identify the song that was being played when the DJ operation was recorded.
An example of the song identification information is song metadata. Song metadata includes the song title, artist name, genre, album name, track order on the album, lyricist name, composer name, arranger name, performance time, production code, product number, etc. In this embodiment, the song identification information is information including the song title, artist name, and album name. In addition, the song identification information may be parameter values based on the analysis results of the song data, or parameter values calculated by a predetermined function from information such as the song title and artist name.
The operation replay information includes song identification information for the songs used when the operation replay information was recorded. Therefore, if multiple songs were used when the operation replay information was recorded, the operation replay information includes song identification information for each of the target songs.

4 generates an image to be displayed on the display device 4. The first image generation unit 365 outputs the generated image to the display device 4 via the output unit 32, thereby causing the generated image to be displayed on the display device 4.
For example, the first image generating unit 365 generates an image of the music management software runtime screen G1 shown in FIG.

FIG. 6 is a diagram showing an example of the operation screen G2.
6, the operation screen G2 includes a record start button G21, a record stop button G22, a reset button G23, and a play button G24, and also has display areas for the elapsed time from the start of recording, the playback time of the song, and the deck currently playing and the BPM of that deck.
The record start button G21 is a button that starts recording DJ operations.
The recording stop button G22 is a button for stopping the recording of DJ operations.
The reset button G23 is a button for erasing the recorded DJ operations, i.e., the reset button G23 is a button for erasing the generated operation reproduction information.
The play button G24 is a button that reads the recorded operation reproduction information and reproduces the DJ operation based on the operation reproduction information. When the play button G24 is pressed and any operation reproduction information is selected by the user from the operation reproduction information stored in the storage unit 34, the operation reproduction process is executed.

4 constitute an operation reproduction processing unit RP. The operation reproduction processing unit RP performs operation reproduction processing to reproduce DJ operations based on operation reproduction information by causing the control unit 36 to execute an operation reproduction program.
The reproduction information acquisition unit 366 corresponds to a control information acquisition unit that acquires a plurality of pieces of acoustic control information included in the operation log information of the operation reproduction information. The reproduction information acquisition unit 366 acquires operation reproduction information selected by the user from the operation reproduction information stored in the storage unit 34. Alternatively, the reproduction information acquisition unit 366 acquires the operation reproduction information from the server SV1 via the communication unit 33.
The identification information acquisition unit 367 acquires the song identification information included in the operation reproduction information acquired by the reproduction information acquisition unit 366 .
The second music acquisition unit 368 acquires music data of a target song identified by the song identification information acquired by the identification information acquisition unit 367 from the music data stored in the storage unit 34. Note that there are cases where music data of the target song is not stored in the storage unit 34. In such cases, the second music acquisition unit 368 may acquire the music data of the target song from the server SV2, or may output a signal indicating that the music data of the target song cannot be acquired to the second music playback unit 371. The music data acquired from the server SV2 by the second music acquisition unit 368 is stored in the storage unit 34. The storage of music data here includes temporary storage by the storage unit 34.

The second image generation unit 369 reads the operation reproduction information acquired by the reproduction information acquisition unit 366 and generates various images, as well as generating various reproduction images of DJ operations based on the operation reproduction information by the reproduction unit 372 described below.

[Operation display image]
FIG. 7 is a diagram showing an example of an operation display image P1 showing the content of DJ operations based on operation reproduction information for each operator provided on the DJ device.
For example, the second image generation unit 369 reads out the operation reproduction information stored in the memory unit 34 and generates an operation display image P1, an example of which is shown in Figure 7, as an image showing the content of the DJ operation included in the operation reproduction information.
The operation display image P1 is an image that shows the operation status of each operator on the DJ equipment, the contents of the target song loaded when the DJ operation is reproduced, and the connection between the target songs. For example, the operation display image P1 is an image that shows the operation status of each operator according to the elapsed time from the start of recording of the operation reproduction information. In other words, the operation display image P1 is an operation content display image that shows the contents of each of the multiple pieces of sound control information included in the operation log information of the operation reproduction information. More specifically, the operation display image P1 is an image that shows the operation status of each operator on the DJ equipment and the playback status of the target song.
7 includes a first song information display area P11, a second song information display area P12, and a details display area P13. In the operation display image P1, a straight line PL indicating the playback position of the target song is set when the DJ operation based on the operation reproduction information is reproduced by the reproduction unit 372 (described later). The straight line PL moves along the horizontal axis, on which the elapsed time of the operation log information in the operation reproduction information is set, to indicate the playback position of the target song.

[Configuration of the first music information display unit and the second music information display unit]
The first song information display section P11 displays information about the first song being played on the first channel. Specifically, the first song information display section P11 displays the name of the first song, the BPM of the first song, the playback position of the first song, the remaining time of the first song, the waveform of the first song, and the song structure of the first song.
The second music information display section P12 displays information about the second music piece being played on the second channel. Specifically, the second music information display section P12 displays the name of the second music piece, the BPM of the second music piece, the playback position of the second music piece, the remaining time of the second music piece, the waveform of the second music piece, the music composition of the second music piece, etc.

Straight lines PL1 and PL2 indicating the playback position of each song are set on the waveform of the first song in the first song information display area P11 and the waveform of the second song in the second song information display area P12 when DJ operations based on the operation reproduction information are reproduced by the reproduction unit 372 (described later). Similar to the straight line PL, the straight lines PL1 and PL2 move on the waveform of the target song to indicate the playback position of the target song.
Note that the information about songs displayed on each song information display section P11, P12 is not limited to the above. For example, each song information display section P11, P12 may display at least the name of the song, and may not display at least one of the other pieces of information. Furthermore, each song information display section P11, P12 may display information about the song other than the above.

[Configuration of detailed display section]
The detail display section P13 displays the operation details of the controls based on the operation log information and the playback status of music on each channel, with the elapsed time included in the operation log information of the operation reproduction information represented on the horizontal axis. The detail display section P13 is located below the first music information display section P11 and the second music information display section P12 in the operation display image P1. The detail display section P13 includes a first operation status display section P14, a second operation status display section P15, a crossfader status display section P16, and an effect status display section P17.
In the detailed display section P13, from top to bottom, there are arranged a first operation status display section P14, a crossfader status display section P16, a second operation status display section P15, and an effect status display section P17, and each of the display sections P14 to P17 extends along the horizontal axis indicating the elapsed time.

[Configuration of first operation status display unit]
FIG. 8 is a diagram showing the first operation status display section P14.
The first operation status display section P14 displays the playback status of the first music piece and the operation status of each operator provided for the first channel along with the elapsed time in the operation reproduction information. As shown in Figures 7 and 8, the first operation status display section P14 includes a time display section TD that indicates the elapsed time included in the operation log information, as well as a first playback status display section P14A, a first fader status display section P14B, and a first equalizer status display section P14C.

The first playback status display area P14A displays the playback status of the first song. As shown in FIG. 15, the first playback status display area P14A displays the playback period of the first channel using a rectangle P141. The timing at which the song was loaded is indicated by the elapsed time corresponding to the left edge of the rectangle P141, which represents the playback period. Characters indicating the track number of the song are also set in the rectangle P141.
The first playback status display area P14A displays the timing at which the CUE button is pressed using a symbol P142, which is a downward-pointing black triangle. The first playback status display area P14A also displays the period during which the CUE button is continuously pressed using a symbol P143, which is a line segment connecting the two symbols P142. Note that if the CUE button is pressed while a song is being played, the playback of that song will be stopped.
The first playback status display section P14A displays the timing of pressing the START/PAUSE button with a downward-facing white triangle symbol P144.

The first fader status display section P14B displays the operating status of the first channel fader. As shown in FIG. 8 , the first fader status display section P14B has an effective range set along the vertical direction, with the upper end of the effective range indicating the position of the first channel fader when the volume level of the first channel is maximum, and the lower end of the effective range indicating the position of the first channel fader when the volume level of the first channel is minimum. The first fader status display section P14B then displays the position of the first channel fader over elapsed time using a line P146. The line P146 indicates changes in the volume level of the first channel and corresponds to a volume level display line along a time series.
In the example of Fig. 8, when the elapsed time indicated by the horizontal axis is from 0 minutes 0 seconds to 2 minutes 40 seconds, the first fader status display section P14B indicates that the first channel fader is positioned at the position where the volume level of the first channel is maximum. Also, in the example of Fig. 15, when the elapsed time is from 3 minutes 25 seconds to 6 minutes 0 seconds, the first fader status display section P14B indicates that the first channel fader is positioned at the position where the volume level of the first channel is minimum.

The first equalizer status display section P14C displays the operational status of the controls for adjusting the volume level of the high frequency band, the mid frequency band, and the low frequency band. These controls are, for example, rotary controls such as a rotary volume control.
8, the first equalizer status display section P14C has an effective range set along the vertical direction, with the upper end of the effective range indicating the position of the control when the volume level of the corresponding band is maximum, and the lower end of the effective range indicating the position of the control when the volume level of the corresponding band is minimum. The first equalizer status display section P14C then displays the position of each control over elapsed time with lines P147, P148, and P149.
Specifically, in the first equalizer state display section P143, the position of the operator for adjusting the volume level of the high frequency band is displayed by a line P147, the position of the operator for adjusting the volume level of the mid frequency band is displayed by a line P148, and the position of the operator for adjusting the volume level of the low frequency band is displayed by a line P149. The colors of the lines P147, P148, and P149 may be different from each other, and the line types of the lines P147, P148, and P149 may be different from each other.

[Configuration of second operation status display unit]
FIG. 9 is a diagram showing the second operation status display section P15.
7 and 9, the second operation status display section P15 displays the playback status of the second music piece and the operation status of each operator provided for the second channel in chronological order from the start of reading of the operation reproduction information. Specifically, the second operation status display section P15 includes a time display section TD, a second playback status display section P15A, a second fader status display section P15B, and a second equalizer status display section P15C.
The second playback status display section P15A displays the playback status of the second music piece. The configuration of the second playback status display section P15A is the same as the configuration of the first playback status display section P14A. That is, as shown in FIG. 16 , the second playback status display section P15A displays the playback period of the second channel using a rectangle P151, the timing when the CUE button was pressed using a symbol P152 similar to the symbol P142, the period during which the CUE button was continuously pressed using a symbol P153 similar to the symbol P143, and the timing when the START/PAUSE button was pressed using a symbol P154 similar to the symbol P144.
9, a white rectangle P155 is set in a rectangle P151 indicating the playback period of the second channel. The rectangle P155 indicates the period during which the jog dial for the second channel was operated during the playback period of the second song. When the user clicks or taps on the rectangle P155, a jog operation correction image P2 (described later) is displayed, allowing the correction unit 370 to correct and edit the jog dial operation information.

The second fader status display section P15B displays the operating status of the second channel fader. Similar to the first fader status display section P14B, the second fader status display section P15B displays the position of the second channel fader over time using a line P156. The line P156 indicates changes in the volume level of the second channel and corresponds to a volume level display line along a time series.
The second equalizer status display section P15C displays the operation status of the operators for adjusting the volume level of the high frequency band, the mid frequency band, and the low frequency band. Similar to the configuration of the first equalizer status display section P14C, the second equalizer status display section P15C is configured such that the position of the operator for adjusting the volume level of the high frequency band is displayed by a line P157, the position of the operator for adjusting the volume level of the mid frequency band is displayed by a line P158, and the position of the operator for adjusting the volume level of the low frequency band is displayed by a line P159.

[Configuration of the crossfader status display section]
FIG. 10 is a diagram showing the crossfader state display section P16.
7, the crossfader status display section P16 is disposed between the first operation status display section P14 and the second operation status display section P15, and extends along the horizontal axis. As shown in FIGS. 7 and 10, the crossfader status display section P16 includes a time display section TD and a position display section P16A that displays the position of the crossfader.
Here, the crossfader is configured to be slidable, for example, in one direction. By sliding the crossfader to one end of the sliding range, the proportion of the first song in the volume being played becomes greater than the proportion of the second song. When the crossfader is positioned at one end of the sliding range, the proportion of the first song becomes the greatest. By sliding the crossfader to the other end, the proportion of the second song in the volume being played becomes greater than the proportion of the first song. When the crossfader is positioned at the other end of the sliding range, the proportion of the second song becomes the greatest.

To display the operational status of the crossfader operated in this manner, the position display section P16A displays the position of the crossfader based on the operation log information LG using a line P161 within an effective range along the vertical direction, as shown in Fig. 10. The position display section P16A indicates that the crossfader is being operated toward one end of the sliding range as it moves upward relative to the crossfader status display section P16, where the first operation status display section P14 is located. Furthermore, the position display section P16A indicates that the crossfader is being operated toward the other end of the sliding range as it moves downward relative to the position display section P16A, where the second operation status display section P15 is located.
In the examples of Figures 7 and 10, the volume ratio between the first song and the second song is adjusted using the first channel fader and the second channel fader, so the position of the crossfader remains set to the intermediate value of the effective range in the position display area P16A and does not change.
In this way, by arranging the position display section P16A and displaying the position of the crossfader as described above, it becomes easier to understand whether the crossfader is being operated to the first channel or the second channel.

[Configuration of effect status display section]
FIG. 11 is a diagram showing the effect status display section P17.
7 and 11, the effect status display section P17 displays the type of effect, the channel to which the effect is applied, the period during which the effect is on, and the application rate of the effect. As shown in Fig. 11, the effect status display section P17 includes a time display section DP, as well as an on status display section P17A, a rate display section P17B, a type display section P17C, and a target display section P17D.

The on-state display section P17A indicates the period during which the effect is on by a rectangle P171.
The ratio display section P17B displays the application ratio, generally expressed as "Dry/Wet." The ratio display section P17B displays the effect application ratio in the effective range along the vertical direction using a line P172. The higher the line P172 is located in the effective range, the greater the application ratio of the effect, and the lower the line P172 is located in the effective range, the smaller the application ratio of the effect.

The type display section P17C displays the type of effect. Specifically, the type display section P17C displays the name of the type of effect that has been set within a rectangle P173 that indicates the setting period for the effect. In the example of Fig. 18, it is shown that "ECHO" was set for the period from 0 minutes 0 seconds to 6 minutes 20 seconds of the elapsed time indicated by the horizontal axis, and "FLANGER" was set for the period from 6 minutes 20 seconds onwards.
The target display section P17D displays the channel to which the effect is to be set. Specifically, the target display section P17D displays the name of the target channel within a rectangle P174 indicating the setting period of the target. In the example of Fig. 11, it is shown that the first channel was set as the target for the period from 0 minutes 0 seconds to 7 minutes 50 seconds of the elapsed time indicated by the horizontal axis, and the master was set as the target for the period from 7 minutes 50 seconds onwards. Note that "master" indicates the master output.

By checking the operation display image P1 described above, you can easily understand the content of the DJ operations recorded in the operation reproduction information. In addition, by checking the operation display image P1, you can easily understand what songs were mixed and how.

[Configuration of the correction section]
The modifying unit 370 modifies the operation reproducibility information stored in the storage unit 34 based on an operation signal input from the input unit 31 when the operation display image P1 is generated and displayed by the second image generating unit 369. In more detail, the modifying unit 370 modifies, as the sound control information to be modified, the sound control information selected in the operation display image P1 from among the multiple pieces of sound control information included in the operation reproducibility information. In other words, the modifying unit 370 can also be said to be an editing unit that can edit the operation reproducibility information including the sound control information.
The correcting section 370 corrects, for example, the first channel fader operation information, the second channel fader operation information, the crossfader operation information, the jog dial operation information, and the effect setting information included in the operation reproduction information.

FIG. 12 is a diagram showing the display contents of the first fader state display section P14B when correcting the first channel fader operation information.
The upper part of Fig. 12 shows a first fader state display section P14B including a line P146 indicating the position of the first channel fader. As shown in the lower part of Fig. 12, when the line P146 is operated by the user, the second image generation section 369 changes the line P146 to a curved or linear shape in accordance with the user's operation. The correction section 370 corrects the first channel fader operation information in the operation log information in accordance with the changed line P146. In this way, the correction section 370 corrects or edits the first channel fader operation information indicating the audio control operation for the first channel fader in accordance with the user's operation on the line P146.
Although not shown, when the user operates a line P156 that is included in the second fader state display section P15B and indicates the position of the second channel fader, the second image generation section 369 changes the line P156 in accordance with the user's operation. Then, the correction section 370 corrects or edits the second channel fader operation information, which is the acoustic control information for the second channel fader in the operation log information, in accordance with the changed line P156.

FIG. 13 is a diagram showing the display contents of the first equalizer state display section P14C when correcting the first equalizer operation information.
The upper part of Fig. 13 shows the first equalizer state display section P14C including the lines P147 to P149, and the lower part of Fig. 13 shows the first equalizer state display section P14C when line P149 is selected as the target of modification or editing among the lines P147 to P149. For example, when a user performs an operation to align lines P147 and P148 with line P149, the second image generation unit 369 aligns lines P147 and P148 with line P149 in accordance with the user's operation. Then, the modification unit 370 modifies the first equalizer operation information, which is acoustic control information for the operator that adjusts the volume level of each frequency band of the first channel in the operation log information, in accordance with the changed lines P147 and P148. That is, the second image generating unit 369 changes the lines P147 to P149 to curved or linear in accordance with the user's operation, and the modifying unit 370 modifies or edits the first equalizer operation information in accordance with the changed lines P147 to P149.
Although not shown, when the user operates lines P157 to P159 that are included in the second equalizer state display unit P15C and indicate the positions of the controls that adjust the volume levels of the respective frequency bands, the second image generation unit 369 changes the lines P157 to P159 in accordance with the user's operation. Then, the correction unit 370 corrects or edits the second equalizer operation information, which is acoustic control information for the controls that adjust the volume levels of the respective frequency bands of the second channel in the operation log information, in accordance with the changed lines P157 to P159.

FIG. 14 shows the display contents of the effect status display section P17 when the effect setting information is corrected.
As described above, the effect status display section P17 includes the time display section DP, as well as an on status display section P17A, a ratio display section P17B, a type display section P17C, and an object display section P17D.
14, when the user selects a rectangle P173 indicating the effect setting period in the type display section P17C, the second image generation section 369 displays a submenu SM1 for selecting the type of effect. In the example of FIG. 14, the submenu SM1 includes options for ECHO, DELAY, SPIRAL, HELIX, and REVERB.
When the user selects an effect from the submenu SM1, the second image generation unit 369 sets the selected effect to the selected effect setting period, and the modification unit 370 modifies or edits the operation log information so that the selected effect is set to the selected effect setting period.
If the length and position of the effect setting period indicated by the rectangle P173 are changed by the user, the second image generation unit 369 displays the type display area P17C reflecting the changed effect setting period. Then, the modification unit 370 modifies or edits the effect setting period in the operation log information in accordance with the changed effect setting period.

14, when a rectangle P174 indicating the setting period is selected by the user in the object display section P17D, the second image generation section 369 displays a submenu SM2 for selecting the setting object. In the example of FIG. 14, the submenu SM2 includes the first channel, the second channel, the third channel, the fourth channel, and the master as options.
When the user selects any setting object in the submenu SM2, the second image generation unit 369 sets the selected setting object to the selected setting period, and the modification unit 370 modifies or edits the operation log information so that the selected setting object is set to the selected setting period.
If the user changes the length and position of the setting period of the setting target indicated by the rectangle P174, the second image generation unit 369 displays the target display area P17D reflecting the changed setting period. Then, the modification unit 370 modifies or edits the setting period of the effect setting target in the operation log information in accordance with the changed setting period.

Although not shown in the drawings, when the user performs an operation to add or delete a rectangle P171 indicating the period during which the effect is on in the on state display section P17A, the second image generation section 369 adds or deletes the rectangle P171 in the on state display section P17A. Also, when the user performs an operation to change the position and length of the rectangle P171, the second image generation section 369 changes the position and length of the rectangle P171 in the on state display section P17A.
Although not shown in the figure, when the user performs an operation to change the line P172 indicating the effect application ratio in the ratio display section P17B, the second image generation section 369 changes the line P172 to a curved or linear shape.
The modifying section 370 then modifies the operation log information in accordance with the user's change operations on the ON state display section P17A and the ratio display section P17B.
In this way, the modifying section 370 modifies the effect setting information including the on-period of the effect, the application rate of the effect, the type of effect, and the channel to which the effect is applied.

FIG. 15 shows a jog operation correction image P2 that is displayed when a period during which the jog dial was operated is selected.
9, the period during which the jog dial for the second channel was operated is set by a rectangle P155 in the second playback state display section P15A of the second operation state display section P15. When the user selects the rectangle P155, the second image generation section 369 displays a jog operation correction image P2 shown in FIG.
The jog operation correction image P2 is included in the operation display image P1. The jog operation correction image P2 includes a song information display section P21 that shows information about the target song operated by the jog dial, a partial waveform display section P22 that shows a partial waveform including a portion of the overall waveform of the target song that changes in response to operation of the jog dial, and a jog dial status display section P23 that shows the operating status of the jog dial.

The song information display section P21 displays the same content as the first song information display section P11 or the second song information display section P12. Specifically, when the jog dial operation period selected by the user is set in the first playback status display section P14A, the song information display section P21 displays information about the first song. On the other hand, when the jog dial operation period selected by the user is set in the second playback status display section P15A, the song information display section P21 displays information about the second song. In other words, the song information display section P21 displays the name, BPM, playback position, remaining time, overall waveform, song structure, etc. of the target song.
The partial waveform display section P22 displays an enlarged waveform of the entire waveform of the target song within a predetermined period that includes the period during which the jog dial is operated.

The jog dial status display section P23 displays the operation status of the jog dial using a representation method known as scratch music notation. Specifically, the jog dial status display section P23 shows the beat position on the horizontal axis and the rotation direction of the jog dial on the vertical axis. More specifically, the jog dial status display section P23 indicates clockwise rotation of the jog dial with an upward line P231 and counterclockwise rotation of the jog dial with a downward line P232. The jog dial status display section P23 also indicates the point at which the sound of the channel controlled by the jog dial in playback mode is cut off by the crossfader with a line P233 along the horizontal axis.

FIG. 16 is a diagram showing a template image P3 displayed together with the jog operation correction image P2.
The user can arbitrarily change the start positions, stop positions, and inclinations of the lines P231 and P232, and the position of the line P233 in the jog dial state display section P23. The second image generation unit 369 changes the start positions, stop positions, and inclinations of the lines P231 and P232, and the position of the line P233, in response to user operations. In other words, the user can arbitrarily change the operation state of the jog dial based on the operation log information.
16, the operation state of the jog dial displayed in the jog dial state display section P23 can be replaced with the operation state of the jog dial displayed by a template TP1, TP2, TP3, ... selected from the template image P3 displayed by the second image generation section 369. In this case, the second image generation section 369 changes the operation state of the jog dial displayed in the jog dial state display section P23 to the operation state of the jog dial displayed by the selected template.
Then, the correction unit 370 corrects or edits the jog dial operation information indicating the operation state of the jog dial to be corrected in the operation log information, according to the operation state of the jog dial changed in the jog dial state display unit P23.

The above-described operation display image P1 is generated by the second image generating unit 369, and the operation display image P1 is displayed on the display device 4, thereby making it possible to visualize the contents of the operation reproduction information.
Furthermore, by the user operating the displayed operation display image P1, the correction unit 370 can correct and edit the operation content of each operator, and ultimately correct and edit multiple pieces of sound control information included in the operation reproduction information.

[Configuration of second music playback unit]
4 corresponds to the playback unit of the present invention, and plays back the music data acquired by the second music acquisition unit 368. At this time, the second music playback unit 371 changes the playback state of the target music in accordance with the user's DJ operation on the operation device 2 reproduced by the reproduction unit 372. Note that if the second music acquisition unit 368 cannot acquire the music data of the target music, the second music playback unit 371 does not play back the music data.

[Configuration of the recapitulation section]
The reproduction unit 372 reproduces the DJ operation based on the operation reproduction information acquired by the reproduction information acquisition unit 366. That is, if the operation reproduction information has been corrected by the correction unit 370, the reproduction unit 372 reproduces the DJ operation, which is an audio control operation, based on the corrected operation reproduction information. Specifically, the reproduction unit 372 analyzes operation log information of the operation reproduction information and controls playback of the target song by the second music playback unit 371 based on the operation log information. More specifically, the reproduction unit 372 reproduces the DJ operation of the user on the operation device 2 at the time the operation reproduction information was recorded, for the target song played by the second music playback unit 371, based on the operation log information of the acquired operation reproduction information.
For example, if the reproduction unit 372 reproduces a DJ operation based on the operation reproduction information when the above-described operation display image P1 is displayed, the reproduction unit 372 reproduces the DJ operation based on the operation reproduction information, and the second music playback unit 371 plays the target music under the control of the reproduction unit 372. At this time, the second image generation unit 369 moves a line PL indicating the elapsed time of the operation log information and lines PL1 and PL2 indicating the playback positions of each music in the operation display image P1 to the right as time passes.

[Configuration of the conversion unit]
The conversion unit 373 converts the operation replay information into operation replay information in a common format. Specifically, the conversion unit 373 converts the operation replay information including a plurality of pieces of acoustic control information into the operation replay information in a format usable by other devices.
The operation reproduction information converted by the conversion unit 373 may be uploaded to the server SV1 or the like on the network N by the communication unit 33, as described above.

[Mix content image]
17 is a diagram showing an example of a mix content image P4 based on operation reproduction information. Specifically, FIG. 17 is an example of a mix content image P4 showing connections between a plurality of target songs based on operation log information LG.
The second image generating unit 369 generates a mix content image P4, an example of which is shown in FIG. 17, as an image showing the content of the DJ operation read out by the reproducing unit 372 from the operation reproduction information.
The operation display image P1 displays the operation status of each control as well as the playback status of music on each channel. In contrast, the mix content image P4 is an image showing the mix content of multiple music pieces based on the operation reproduction information. In other words, the mix content image P4 is an image showing the mix status of music pieces based on the reproduction results of the DJ operations performed by the reproduction unit 372.

The mix content image P4 includes a first deck display portion P4A, a second deck display portion P4B, a third deck display portion P4C, and a fourth deck display portion P4D.
In this embodiment, music loaded onto the first and second decks is played on channel 1, and music loaded onto the third and fourth decks is played on channel 2. Therefore, the first deck display unit P4A and the second deck display unit P4B display the playback status of music played on channel 1, and the third deck display unit P4C and the fourth deck display unit P4D display the playback status of music played on channel 2.

Each deck display unit P4A, P4B, P4C, and P4D displays the playback period of the song using a rectangle P41 that follows the elapsed time. When a song is being played in a loop, each deck display unit P4A, P4B, P4C, and P4D displays a rectangle P411 indicating the loop playback period within the rectangle P41, and also displays a loop playback mark P412 within the rectangle P411. For example, in the example of Figure 17, there is a loop playback period at the end of the song loaded on the second deck, and there is also a loop playback period at the end of the song loaded on the fourth deck, so on the deck display units P4B and P4D, the rectangle P411 and the loop playback mark P412 are set within the rectangle P41 indicating the playback period of the song.

Each deck display section P4A, P4B, P4C, and P4D displays a waveform P42 of a song loaded on the corresponding deck. For example, if a DJ operation is performed to change the playback direction or playback speed of a song, the song's waveform P42 is an edited version of the song's original waveform to match the DJ operation. In other words, the waveform P42 is an edited version of the song's original waveform based on the operation log information.
17, the waveform P42 is arranged above the rectangle P41. However, the present invention is not limited to this, and the waveform P42 may be arranged below the rectangle P41 or displayed overlapping the rectangle P41. Furthermore, as long as the rectangle P411 indicating the loop playback period and the loop playback mark P412 are visibly displayed, only one of the rectangle P41 and the waveform P42 may be displayed.

Similar to the fader status display units P142 and P152 described above, each deck display unit P4A, P4B, P4C, and P4D uses a line P43 to display the position of the channel fader for the channel associated with the deck in accordance with the playback period of the song. Each deck display unit P4A, P4B, P4C, and P4D has an effective range set along the vertical direction in the area above the rectangle P41 that overlaps with the waveform P42. The upper end of the effective range indicates the position of the channel fader when the volume level of the corresponding channel is at its maximum, and the lower end of the effective range indicates the position of the channel fader when the volume level of the corresponding channel is at its minimum. Based on the operation reproduction information, each deck display unit P4A, P4B, P4C, and P4D displays the position of the channel fader when the song loaded on the corresponding deck is being played using a line P43.

By checking this mix content image P4, it is easier to understand how the songs loaded onto each deck were mixed when the operation reproduction information was recorded.

FIG. 18 is a diagram showing an example of the operation reproduction screen G3.
The second image generation unit 369 generates an image of the operation reproduction screen G3 shown in Fig. 18. Specifically, the second image generation unit 369 generates an image of the operation reproduction screen G3 showing the content of the DJ operation performed on the operation device 2 and reproduced by the reproduction unit 372 based on the operation reproduction information, and causes the generated image to be displayed on the display device 4. In other words, the second image generation unit 369 generates an image that visualizes the DJ operation reproduced by the reproduction unit 372.

The operation reproduction screen G3 includes a first music information display section G31, a second music information display section G32, an operation display section G33, a menu button G34, a speed adjustment button G35, a reproduction start button G36, and a reproduction position display bar G37.
The first music information display section G31 and the second music information display section G32 are arranged at the top of the operation reproduction screen G3.
The first song information display section G31 is located on the left side and displays information about the first song among the target songs. Specifically, the first song information display section G31 displays the title, playback time, remaining playback time, and BPM of the first song, as well as waveform information about the first song. The waveform information is a graph of signal strength according to the playback time of the song, with the horizontal axis representing the playback time of the song and the vertical axis representing the signal strength of the song.
The second song information display section G32 is located on the right side and displays information about the second song among the target songs. The display content of the second song information display section G32 is the same as that of the first song information display section G31.

The operation display unit G33 displays a device image G331 in a plan view of the operation device 2. Specifically, the operation display unit G33 displays a plurality of operators that the operation device 2 has at the same positions as the respective positions of the plurality of operators on the operation device 2.
The second image generation unit 369 draws the content of the DJ operation on the operation device 2 reproduced by the reproduction unit 372 and displays it on the operation display unit G33. That is, the second image generation unit 369 visualizes the DJ operation reproduced by the reproduction unit 372 on the operation display unit G33.

For example, the second image generation unit 369 generates an image in which the position of a partial image indicating a controller of the deck indicated by the deck number DN, among the controllers indicated by the controller type SC in the record RC of the operation log information LG that corresponds to the elapsed time since the operation reproduction process was started, is moved to the controller position SP. In other words, the second image generation unit 369 moves the position of a partial image indicating a controller that corresponds to the controller type SC and deck number DN, among the controllers included in the device image G331, to a position corresponding to the controller position SP that corresponds to the elapsed time.
As a specific example, if the operator indicated by the operator type SC of the acquired record RC is a channel fader and the deck number DN indicated by the deck number DN of the record RC is "2", the second image generation unit 369 generates an operation reproduction screen G3 in which a partial image showing the operation knob of the channel fader of the second deck is moved to a position corresponding to the operator position SP of the record RC.

Furthermore, for example, the second image generation unit 369 generates an image showing, by a graphic FG such as an arrow based on the controller position SP, the movement direction or rotation direction of the controller of the deck indicated by the deck number DN, among the controllers indicated by the controller type SC in the record RC of the operation log information LG that corresponds to the elapsed time since the operation reproduction process was started. In other words, the second image generation unit 369 displays, near the controller that corresponds to the controller type SC and deck number DN, among the controllers in the device image G331, a graphic FG such as an arrow that shows the operation direction and rotation direction of the controller, based on the controller position SP according to the elapsed time.
As a specific example, if the operator indicated by the operator type SC of the acquired record RC is a jog dial and the deck number DN indicated by the deck number DN of the record RC is "1", the second image generation unit 369 generates an operation reproduction screen G3 based on the operator position SP of the record RC, in which a figure FG, which is an arrow indicating the operation direction of the rotary operator, is superimposed on a partial image showing the jog dial of the first deck.

Furthermore, for example, the second image generating unit 369 highlights or displays a graphic such as a circle on the operator of the deck indicated by the deck number DN among the operators indicated by the operator type SC in the record RC of the operation log information LG that corresponds to the elapsed time since the operation reproduction process was started. That is, the second image generating unit 369 highlights or displays a graphic such as a circle on the operator that corresponds to the operator type SC and deck number DN among the operators in the device image G331 in accordance with the elapsed time, thereby indicating the operator in a manner that allows it to be distinguished from other operators.
As a specific example, if the operator indicated by the operator type SC of the acquired record RC is the first sampler pad and the deck number DN indicated by the deck number DN of the record RC is "2", the second image generation unit 369 generates an operation reproduction screen G3 in which a partial image showing the first sampler pad of the second deck is highlighted, or an operation reproduction screen G3 in which a figure FG such as a circle is superimposed on the partial image showing the first sampler pad.

The operation display unit G33 can also display information related to the reproduced DJ operation. For example, the second image generation unit 369 displays a speech bubble SB containing information related to the DJ operation on the operation display unit G33. Examples of such information related to the DJ operation include the amount of operation of the controls, the values of parameters changed by the reproduced operation, information that can identify the DJ device used when the operation reproduction information was recorded, and the settings of the DJ application when the operation reproduction information was recorded.
Examples of the operation amount of a control include the movement amount of the control, a numerical value indicating the position of the control, and the rotation angle of the control, etc. In the example of Fig. 7, the speech bubble SB indicates the movement amount of the crossfader in the direction of the arrow.
The parameter value may be the parameter value changed in accordance with the operation of a control that sets a specific parameter value, if the operation of the control is recorded in the operation log information LG. For example, if the operation log information LG records the pressing of a loop button, the parameter value is the "number of bars in the loop" that was set by pressing the loop button.
Displays including this information are sequentially set in the operation display section G33 by the second image generation section 369. As a result, the contents of the reproduced DJ operation and information related to the reproduced DJ operation are set in the operation display section G33, and the operation reproduction screen G3 including the operation display section G33 is sequentially displayed.

The menu button G34 is a button for displaying a menu screen (not shown). The menu screen includes, for example, a selection item for selecting a device image G331 of an operation device displayed on the operation display unit G33.
Here, when reproducing the DJ operations of another user based on the acquired operation reproduction information, for example, a device image G331 that imitates the model of the operation device 2 used by the user is displayed, making it easier to understand the DJ operations of the other user.
Therefore, the second image generation unit 369 displays the device image G331 selected by the user on the operation display unit G33, and visualizes the operations performed on the controls included in the device image G331 based on the reproduced DJ operations, making it easier for the user to understand the content of the DJ operations performed by other users.

The speed adjustment button G35, reproduction start button G36, and reproduction position display bar G37 allow the contents of the DJ operation displayed on the operation display section G33 to be operated like a moving image.
The speed adjustment button G35 is a button that increases or decreases the playback speed of the target music piece by the second music playback unit 371 and the reproduction speed of the DJ operations by the reproduction unit 372 in synchronization with each other. In the example of the operation reproduction screen G3 shown in FIG. 7 , the speed adjustment button G35 includes buttons for adjusting the speed to 0.5x, 0.75x, 1x, 1.25x, and 1.5x. The reproduction unit 372 reproduces the DJ operations at a speed corresponding to the input button and causes the second music playback unit 371 to play the target music piece at the speed corresponding to the input button. When the reproduction speed of the DJ operations is increased or decreased by the speed adjustment button G35, the display update speed of the operation display unit G33 also increases or decreases in synchronization with the reproduction speed of the DJ operations.
In this way, the reproduction unit 372 synchronizes and increases or decreases the speed at which the DJ operation is performed, which is reproduced based on the operation reproduction information, and the playback speed of the target song played by the second music playback unit 371. Note that the reproduction speed of the DJ operation and the playback speed of the target song may be configured so that the user can set any desired speed.

The reproduction start button G36 is a button that starts the operation reproduction process of the DJ operation based on the operation reproduction information. When the reproduction start button G36 is pressed, the operation reproduction process starts, and the second music playback unit 371 starts playing the target music, and the reproduction unit 372 starts reproducing the DJ operation based on the operation reproduction information. Note that when the reproduction start button G36 is pressed again, the operation reproduction process is paused.
The reproduction position display bar G37 is provided with a slider G371 that indicates the current reproduction position of the DJ operation based on the operation reproduction information. The user can adjust the reproduction position by sliding the slider G371 along the reproduction position display bar G37, and based on the position of the slider G371, the reproduction unit 372 changes the reproduction position of the DJ operation and also changes the playback position of the target song by the second music playback unit 371.
The operation reproduction screen G3 may be provided with a button for moving the reproduction position back a predetermined time from the current reproduction position, or a button for moving the reproduction position forward a predetermined time from the current reproduction position.

[Effects of the embodiment]
The operation reproduction system 1A according to the present embodiment described above has the following advantages.
The operation reproduction system 1A includes an operation device 2, an information processing device 3, and a display device 4.
The information processing device 3 includes a reproduction information acquisition unit 366 , a correction unit 370 , a second music playback unit 371 , and a reproduction unit 372 .
The reproduction information acquisition unit 366 corresponds to a control information acquisition unit, and acquires a plurality of pieces of sound control information included in the operation log information of the operation reproduction information. That is, the reproduction information acquisition unit 366 acquires a plurality of pieces of sound control information that record a plurality of sound control operations for audio data.
The modifying unit 370 can modify each of the acquired multiple pieces of acoustic control information.
The second music playback unit 371 corresponds to a playback unit and plays back the target music.
The reproducing unit 372 reproduces a plurality of sound control operations based on operation reproduction information including the plurality of corrected sound control information for the music played by the second music playback unit 371 .

With this configuration, the reproduction unit 372 reproduces multiple sound control operations based on multiple pieces of sound control information for the music played by the second music playback unit 371. Therefore, by acquiring multiple pieces of sound control information, the music can be played in the same way as when the sound control operations were recorded. Therefore, for example, it is possible to automatically play a music piece that involves multiple sound control operations.
Furthermore, even if an incorrect operation is performed when recording sound control operations, the correcting unit 370 can correct the incorrect operation for each of the multiple sound control operations. Therefore, music can be played back with more appropriate sound control operations.

In the information processing device 3, the modifying section 370 modifies the selected piece of sound control information to be modified from among the plurality of pieces of sound control information.
With this configuration, the modifying section 370 modifies the selected piece of sound control information to be modified from among the plurality of pieces of sound control information, so that each piece of sound control information can be modified individually.

The information processing device 3 includes a second image generating unit 369 that generates an operation display image P1 that indicates the content of each of the plurality of pieces of sound control information. The second image generating unit 369 corresponds to an image generating unit. The correcting unit 370 corrects the sound control information selected in the operation display image P1 from the plurality of pieces of sound control information as the sound control information to be corrected.
With this configuration, the user can check the content of each piece of sound control information, i.e., the content of each sound control operation, by checking the operation display image P1 generated by the second image generation unit 369. The correction unit 370 then corrects the sound control information selected by the user in the operation display image P1 as the sound control information to be corrected, allowing the user to easily select the sound control information to be corrected. This makes it easy to check the content of the sound control information and select the sound control information to be corrected.

In the information processing device 3, the plurality of sound control operations are performed on a plurality of different operators.
According to this configuration, the modifying section 370 modifies the acoustic control information, thereby making it possible to modify the operations on the respective controls when recording a plurality of pieces of acoustic control information.

In the information processing device 3, one of the plurality of pieces of acoustic control information is first channel fader operation information indicating an operation on a first channel fader that adjusts the volume level of a first music piece, and another of the plurality of pieces of acoustic control information is second channel fader operation information indicating an operation on a second channel fader that adjusts the volume level of a second music piece. The modifying unit 370 modifies each piece of channel fader operation information.
With this configuration, the recorded operations for each channel fader can be corrected by the correcting section 370.

In the information processing device 3, the operation display image P1 includes a first fader state display section P14B that indicates changes in the volume level of the first channel by a line P146 that follows a time series, and a second fader state display section P15B that indicates changes in the volume level of the second channel by a line P156 that follows a time series. The lines P146 and P156 correspond to the volume level display lines, and the fader state display sections P14B and P15B correspond to the channel fader state display sections.
The modifying unit 370 modifies the first channel fader operation information according to the line P146 adjusted by the user, and modifies the second channel fader operation information according to the line P156 adjusted by the user.
With this configuration, the user adjusts the line P146 in the first fader state display portion P14B of the operation display image P1, causing the correction unit 370 to correct the first channel fader operation information. Also, the user adjusts the line P156 in the second fader state display portion P15B of the operation display image P1, causing the correction unit 370 to correct the second channel fader operation information. This allows the user to easily correct each channel fader operation information.

In the information processing device 3, one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel that can play a target song. The correction unit 370 corrects the crossfader operation information.
With this configuration, the recorded crossfader operation can be corrected by the correcting unit 370.

In the information processing device 3, the operation display image P1 includes a crossfader state display portion P16 that indicates a change in the volume balance between the first channel and the second channel by a line P161 that follows a time series. The line P161 corresponds to a volume balance display line.
The modifying unit 370 modifies the crossfader operation information according to the line P161 adjusted by the user.
With this configuration, the user adjusts the line P161 in the crossfader state display section P16 of the operation display image P1, and the correction section 370 corrects the crossfader operation information. This allows the user to easily correct the crossfader operation information.

In the information processing device 3, one of the plurality of pieces of acoustic control information is jog dial operation information that indicates an operation on a jog dial that adjusts the playback direction and playback speed of the target song. The correction unit 370 corrects the jog dial operation information.
With this configuration, the recorded operation of the jog dial can be corrected by the correcting unit 370.

In the information processing device 3, the operation display image P1 includes a jog operation correction image P2, which includes a jog dial state display portion P23 that indicates the operation state of the jog dial. The correction portion 370 corrects the jog dial operation information in accordance with the operation state of the jog dial adjusted by the user on the jog dial state display portion P23.
With this configuration, the user adjusts the operation state of the jog dial in the jog dial state display portion P23 of the operation display image P1, and the correction portion 370 corrects the jog dial operation information. This allows the user to easily correct the jog dial operation information.

In the information processing device 3, one of the plurality of pieces of sound control information is effect setting information that indicates an operation on a control that sets an effect to be applied to the reproduced target music piece.
The modifying unit 370 modifies the effect setting information including at least one of the type of effect, the on-duration of the effect, the application rate of the effect, and the target channel of the effect. In this embodiment, the effect setting information includes the type of effect, the on-duration of the effect, the application rate of the effect, and the target channel of the effect.
With this configuration, the recorded effect setting operation can be corrected by the correcting section 370.

In the information processing device 3, the operation display image P1 includes an effect status display section P17 that indicates the setting status of the effect. The modifying section 370 modifies the effect setting information in accordance with the setting status of the effect adjusted by the user.
With this configuration, the user adjusts the effect setting state in the effect state display section P17 of the operation display image P1, and the modifying section 370 modifies the effect setting information, thereby enabling the user to easily modify the effect setting information.

The information processing device 3 includes a conversion unit 373 that converts operation reproduction information including a plurality of pieces of sound control information into a format that can be used by other devices.
According to this configuration, by acquiring the operation reproducing information, the other device can, for example, play music based on the operation reproducing information, thereby improving the convenience of the operation reproducing information.

[Modification of the embodiment]
The present invention is not limited to the above-described embodiment, and modifications and improvements within the scope of achieving the object of the present invention are included in the present invention.
In the above embodiment, each of the plurality of pieces of audio control information included in the operation reproduction information is an operation for a corresponding operator and indicates an audio control operation applied to a song. The plurality of pieces of audio control information includes channel fader operation information, crossfader operation information, jog dial operation information, and effect setting information. However, this is not limiting, and the plurality of pieces of audio control information may not include at least one of the channel fader operation information, crossfader operation information, jog dial operation information, and effect setting information. On the other hand, the audio control information may include control information indicating an operation or setting operation for another operator. For example, one of the plurality of pieces of audio control information may be information indicating the operation state of a tempo slider that adjusts the tempo of a song.

In the above embodiment, the operation display image P1, on which the user modifies and edits the sound control information, is an image having the layout shown in FIG. 7. However, this is not a limitation, and the layout and content of the operation display image P1 are not limited to the above. For example, the operation display image P1 may further include a display section showing the operation content of other controls.

In the above embodiment, the modifying unit 370 modifies or edits the corresponding sound control information in response to a user's operation on the operation display image P1. However, this is not limited to this, and the modifying unit 370 may modify or edit the sound control information by replacing the numerical value indicated by the sound control information included in the operation log information of the operation reproduction information with a numerical value input by the user.

In the above embodiment, the conversion unit 373 converts the operation replay information including the recorded and modified sound control information into operation replay information in a format usable by other devices. However, this is not limited to this, and the conversion unit 373 may not be required. Alternatively, the replay information generation unit 364 may generate operation replay information in a format usable by other devices.

[Summary of the present invention]
The present invention will be summarized below.
[1] a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music;
a correcting unit capable of correcting each of the acquired plurality of pieces of acoustic control information;
a playback unit that plays the music;
a reproducing unit that reproduces, for the played music piece, the plurality of sound control operations based on the plurality of pieces of corrected sound control information.

With this configuration, the reproduction unit reproduces multiple sound control operations based on multiple pieces of sound control information for the music played by the playback unit. Therefore, by acquiring multiple pieces of sound control information, the music can be played in the same way as when the sound control operations were recorded. Therefore, for example, it is possible to automatically play a music piece that involves multiple sound control operations.
Furthermore, even if an incorrect operation is performed when recording sound control operations, the correcting unit can correct the incorrect operation for each of the multiple sound control operations, thereby enabling the playback of music with more appropriate sound control operations.

[2] In the information processing device according to [1],
The information processing device, wherein the correction unit corrects a piece of sound control information to be corrected that is selected from the plurality of pieces of sound control information.
According to this configuration, the correction unit corrects the selected piece of sound control information to be corrected from among the plurality of pieces of sound control information, so that each piece of sound control information can be corrected individually.

[3] In the information processing device according to [2],
an image generating unit that generates an operation display image showing the content of each of the plurality of pieces of acoustic control information;
The information processing apparatus, wherein the correction unit corrects, as the sound control information to be corrected, sound control information selected on the operation display image from among the plurality of pieces of sound control information.
With this configuration, the content of each piece of sound control information, i.e., the content of each sound control operation, can be confirmed by checking the operation display image generated by the image generation unit. The correction unit then corrects the sound control information selected in the operation display image as the sound control information to be corrected, allowing the user to easily select the sound control information to be corrected. This makes it easy to check the content of the sound control information and select the sound control information to be corrected.

[4] In the information processing device according to [3],
10. An information processing apparatus, wherein the plurality of sound control operations are operations on a plurality of mutually different operators.
According to this configuration, the modifying unit modifies the acoustic control information, thereby making it possible to modify the operations on the respective controls when recording the plurality of acoustic control information.

[5] In the information processing device according to [4],
one of the plurality of pieces of acoustic control information is channel fader operation information indicating an operation on a channel fader that adjusts the volume level of the music piece,
The information processing device, wherein the correction unit corrects the channel fader operation information.
According to this configuration, the recorded channel fader operations can be corrected by the correcting unit.

[6] In the information processing device according to [5],
the operation display image includes a channel fader state display section that indicates a change in the volume level by a volume level display line along a time series,
The information processing device, wherein the correction unit corrects the channel fader operation information in accordance with the volume level display line adjusted by a user.
With this configuration, the user adjusts the volume level display line in the channel fader state display section of the operation display image, and the correction section corrects the channel fader operation information. This allows the user to easily correct the channel fader operation information.

[7] In the information processing device according to any one of [4] to [6],
one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel that can play the music piece,
The information processing device, wherein the correction unit corrects the crossfader operation information.
According to this configuration, the recorded crossfader operation can be corrected by the correcting unit.

[8] In the information processing device according to [7],
the operation display image includes a crossfader state display section that indicates a change in the volume balance by a volume balance display line along a time series;
The information processing device, wherein the correction unit corrects the crossfader operation information in accordance with the volume balance display line adjusted by a user.
With this configuration, the user adjusts the volume balance display line in the crossfader state display section of the operation display image, and the correction section corrects the crossfader operation information. This allows the user to easily correct the crossfader operation information.

[9] In the information processing device according to any one of [4] to [8],
one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial for adjusting a playback direction and a playback speed of the music piece,
The information processing device, wherein the correction unit corrects the jog dial operation information.
According to this configuration, the recorded operation of the jog dial can be corrected by the correcting unit.

[10] In the information processing device according to [9],
the operation display image includes a jog dial state display portion that indicates an operation state of the jog dial,
The information processing device, wherein the correction unit corrects the jog dial operation information in accordance with an operation state of the jog dial adjusted by a user.
With this configuration, the user adjusts the operation state of the jog dial in the jog dial state display portion of the operation display image, and the correction portion corrects the jog dial operation information. This allows the user to easily correct the jog dial operation information.

[11] In the information processing device according to any one of [4] to [10],
one of the plurality of pieces of sound control information is effect setting information indicating an operation on an operator that sets an effect to be applied to the reproduced music piece,
The information processing device, characterized in that the modification unit modifies the effect setting information including at least one of the type of effect, the on period of the effect, the application rate of the effect, and the target channel of the effect.
According to this configuration, the recorded effect setting operation can be corrected by the correcting unit.

[12] In the information processing device according to [11],
the operation display image includes an effect status display section that indicates a setting status of the effect,
The information processing device, wherein the modifying unit modifies the effect setting information in accordance with a setting state of the effect adjusted by a user.
With this configuration, the modifying unit modifies the effect setting information by the user adjusting the effect setting state in the effect state display section of the operation display image, thereby allowing the user to easily modify the effect setting information.

[13] In the information processing device according to any one of [1] to [12],
An information processing device comprising: a conversion unit that converts the plurality of pieces of acoustic control information into a format that can be used by other devices.
With this configuration, by acquiring multiple pieces of audio control information, other devices can, for example, play music based on the multiple pieces of audio control information, thereby improving the convenience of the audio control information.

3...information processing device, 369...second image generation unit (image generation unit), 370...modification unit, 371...second music playback unit (playback unit), 372...reproduction unit, 373...conversion unit, P1...operation display image, P14B...first fader status display unit (channel fader status display unit), P146...line (volume level display line), P15B...second fader status display unit (channel fader status display unit), P156...line (volume level display line), P16...cross fader status display unit, P161...line (volume balance display line), P17...effect status display unit, P23...jog dial status display unit.

Claims (12)

a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music;
a correcting unit capable of correcting each of the acquired plurality of pieces of acoustic control information;
a playback unit that plays the music;
a reproducing unit that reproduces the plurality of sound control operations based on the plurality of pieces of sound control information corrected for the reproduced music piece,
one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial for adjusting a playback direction and a playback speed of the music piece,
The information processing device, wherein the correction unit corrects the jog dial operation information. 2. The information processing device according to claim 1 ,
The information processing device, wherein the correction unit corrects a piece of sound control information to be corrected that is selected from the plurality of pieces of sound control information. 3. The information processing device according to claim 2 ,
an image generating unit that generates an operation display image showing the content of each of the plurality of pieces of acoustic control information;
The information processing apparatus, wherein the correction unit corrects, as the sound control information to be corrected, sound control information selected on the operation display image from among the plurality of pieces of sound control information. 4. The information processing device according to claim 3 ,
10. An information processing apparatus, wherein the plurality of sound control operations are operations on a plurality of mutually different operators. 5. The information processing device according to claim 3 ,
one of the plurality of pieces of acoustic control information is channel fader operation information indicating an operation on a channel fader that adjusts the volume level of the music piece,
The information processing device, wherein the correction unit corrects the channel fader operation information. 6. The information processing device according to claim 5 ,
the operation display image includes a channel fader state display section that indicates a change in the volume level by a volume level display line along a time series;
The information processing device, wherein the correction unit corrects the channel fader operation information in accordance with the volume level display line adjusted by a user. 5. The information processing device according to claim 3 ,
one of the plurality of pieces of acoustic control information is crossfader operation information indicating an operation on a crossfader that adjusts the volume balance between a first channel and a second channel that can play the music piece,
The information processing device, wherein the correction unit corrects the crossfader operation information. 8. The information processing device according to claim 7 ,
the operation display image includes a crossfader state display section that indicates a change in the volume balance by a volume balance display line along a time series;
The information processing device, wherein the correction unit corrects the crossfader operation information in accordance with the volume balance display line adjusted by a user. 5. The information processing device according to claim 3 ,
the operation display image includes a jog dial state display portion that indicates an operation state of the jog dial,
The information processing device, wherein the correction unit corrects the jog dial operation information in accordance with an operation state of the jog dial adjusted by a user. 5. The information processing device according to claim 3 ,
one of the plurality of pieces of sound control information is effect setting information indicating an operation on an operator that sets an effect to be applied to the reproduced music piece,
The information processing device, characterized in that the modification unit modifies the effect setting information including at least one of the type of effect, the on period of the effect, the application rate of the effect, and the target channel of the effect. 11. The information processing device according to claim 10 ,
the operation display image includes an effect status display section that indicates a setting status of the effect,
The information processing device, wherein the modifying unit modifies the effect setting information in accordance with a setting state of the effect adjusted by a user. A program executed by at least one processor, comprising:
the at least one processor,
a control information acquisition unit that acquires a plurality of pieces of audio control information that record a plurality of audio control operations for a piece of music;
a correcting unit capable of correcting each of the acquired plurality of pieces of acoustic control information;
a playback unit that plays the music;
a reproduction unit that reproduces the plurality of sound control operations based on the plurality of pieces of corrected sound control information for the reproduced music piece;
one of the plurality of pieces of acoustic control information is jog dial operation information indicating an operation on a jog dial for adjusting a playback direction and a playback speed of the music piece,
The correction unit corrects the jog dial operation information.