JP4134624B2 - Playback control information creation device and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus and a program for creating reproduction control information such as beat timing information and tempo information added to music data such as audio data and performance data.
[0002]
[Prior art]
Conventionally, it is known to control the tempo and dynamics (volume, velocity, etc.) of an automatic performance based on an operation such as a user's hand gesture. FIG. 9 shows an example of controlling the tempo and dynamics of the automatic performance by detecting the direction and acceleration of the hand movement by the motion sensor, and this example is described in Japanese Patent No. 314936.
[0003]
In the example of FIG. 9, the performance data is obtained by arranging a plurality of data sets including event data representing an event and delta time data representing a relative time between events in accordance with the progress of the performance. The event data of channel CH1 represents a beat event by a pitch name.₃, C^# ₃And D₃Represents the beat events of the first beat, the second beat and the third beat, respectively. The event data of the channels CH2 to CH6 represents performance events such as sounding or muting. From the motion sensor, for example, C₃, C^# ₃, D₃Are sequentially transmitted as tempo key-on signals, and a dynamics control signal corresponding to the acceleration of the hand movement is also transmitted for each tempo key-on signal.
[0004]
In automatic performance, after the event data of CH3 is read at time t0, the clock signal is counted and when time t1 corresponding to delta time D0 is reached, C₃The event data of CH1 representing (first beat) is read out. At this time, from the motion sensor C₃If the (first beat) tempo key-on signal is not received, the counting of the clock signal is temporarily stopped. C at time t2₃When the (first beat) tempo key-on signal is received, the clock signal counting is resumed. When the clock signal is counted and the time t3 corresponding to the delta time data D1 is reached, the event data of CH2 is read out. Thereafter, the automatic performance is performed by counting the clock signal and reading the event data every time the time corresponding to the delta time is reached.
[0005]
After the event data of CH6 is read at time t7, the time corresponding to the delta time D6 is being measured based on the clock signal.^# ₃When the (second beat) tempo key-on signal is received at time t8, the next CH1 event data is searched to obtain the time interval TIa between the delta times D1 to D7. Since this time interval TIa is longer than the time interval TIm between times t2 and t8 (that is, the tempo of the automatic performance is slower than the hand tempo), the remaining time of the delta time D6 and the delta time D7 so that TIa approaches TIm. Modify to shorten. Thereafter, as described above, the automatic performance is performed by counting the clock signal and reading the event data every time a time corresponding to the delta time is reached.
[0006]
According to the automatic performance process described above with reference to FIG. 9, the tempo of the automatic performance can be controlled to follow the hand tempo. In addition, the dynamics (dynamic feeling) of the automatic performance can be controlled by controlling the volume and velocity (strength for each sound) according to the dynamics control signal received from the motion sensor.
[0007]
FIG. 10 shows another example of conventional automatic performance processing, and this example is described in Japanese Patent Application No. 2000-172617 relating to the application of the same applicant as the present application.
[0008]
Memory track TR₁, TR₂And TR₃Stores performance data, tempo control data and video data relating to the desired music piece. The performance data is obtained by arranging a plurality of data sets including timing data representing timing Ta and event data representing performance events at Ta according to the progress of performance.
[0009]
The tempo control data is obtained by arranging a plurality of data sets including timing data representing the timing Tb and event data representing a tempo control event at Tb according to the progress of performance. The tempo control event is an event for setting the frequency of the clock signal for automatic performance in accordance with the tempo value, and the tempo value is determined based on the detection signal of the motion sensor that responds to the above-described hand gesture motion.
[0010]
The video data is obtained by arranging a plurality of data sets including timing data representing the timing Tc and event data representing a display event at Tc according to the performance progress. The display event is, for example, an event that displays a musical score note corresponding to the performance content.
[0011]
The automatic performance is performed by counting the clock signal and reading a performance event related to Ta in the performance data every time a time corresponding to the timing Ta in the performance data is reached. Tempo control data is generated based on the detection signal of the motion sensor by shaking the motion sensor while listening to the automatic performance. By controlling the frequency of the clock signal in accordance with the tempo control data, the tempo of the automatic performance is controlled so as to follow the tempo of the hand movement operation. The tempo control data based on the hand movement is track TR in the format shown in FIG.₂Is remembered. After this, when performing automatically, track TR₂It is possible to control the tempo of automatic performance based on the tempo control data stored in the above. Since the score display is performed based on the clock signal and the video data, the display score is updated in accordance with the tempo control data.
[0012]
[Problems to be solved by the invention]
According to the prior art described above with reference to FIG. 9, it is necessary to insert beat timing information (CH1 event data and delta time data) in advance in performance data, and there is a problem that it takes time to create performance data. .
[0013]
On the other hand, according to the prior art described above with reference to FIG. 10, tempo control data can be generated and stored in real time by shaking the motion sensor. However, when the stored tempo control data does not satisfy the user, it is necessary for the user to repeat the hand gesture many times until the user is satisfied, and it takes time and effort to create the tempo control data. There is a point.
[0014]
An object of the present invention is to provide a novel reproduction control information creating apparatus capable of efficiently creating reproduction control information such as beat timing information and tempo information with less time and effort.
[0015]
[Means for Solving the Problems]
  This inventionPertaining toThe playback control information creation device
  Playback means for playing music based on music data;
  Display means for displaying a flow of reproduced music based on the music data;
  Input means for inputting a plurality of beat positions and the number of beats between these beat positions with respect to the flow of the reproduced music;
  Storage means capable of storing reproduction control information;
  First determining means for determining one or a plurality of beat positions according to the number of beats related to the input by the input means between the plurality of beat positions related to the input by the input means;
  A beat time is determined based on the music data for each beat position among a plurality of beat positions related to input by the input means and one or a plurality of beat positions related to determination by the first determination means, Second determination means for writing beat time information indicating the beat time related to determination into the storage means as the reproduction control information;
  It is equipped with.
[0016]
  The present inventionAccording to the reproduction control information creating apparatus, when a plurality of beat positions and the number of beats between these beat positions are input with respect to the reproduced music displayed while listening to the music to be reproduced, the input beat number is set between the input beat positions. The beat position is determined accordingly. Then, a beat time is determined for each beat position related to input and determination based on the music data, and beat time information indicating the determined beat time is stored as reproduction control information. As the flow of playback music, a sound waveform can be displayed based on audio data, or a score can be displayed based on performance data, so the user can view such display and listen to the playback music. However, an input operation can be performed easily and in a short time by an instruction from a hand or a mouse (that is, using GUI [Graphical User Interface]).
[0019]
When the interval between two beat positions related to input is constant, the number of beat positions to be determined is small if the number of input beats is small, and the number of beat positions to be determined is large if the number of input beats is large. A small number of determined beat positions means that the tempo is slow, and a large number of determined beat positions means that the tempo is fast. Also, when the number of input beats is constant, a slow tempo can be expressed by widening the interval between input beat positions, or a fast tempo can be expressed by narrowing the interval between input beat positions. it can. Therefore, the tempo change desired by the user can be reflected in the beat time information.
[0025]
  The present inventionIn the reproduction control information creating apparatus, there may be provided drawing means for drawing a tempo curve representing a tempo change regarding the flow of reproduced music and displaying the tempo curve on the display means. In this way, since the beat position, the number of beats, and the like can be input according to the displayed tempo curve, the input or correction work can be performed more easily and appropriately.
[0026]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows a circuit configuration of a music playback apparatus that executes playback control information creation processing according to an embodiment of the present invention. This music playback apparatus is configured to be controlled by a small computer 10 such as a personal computer.
[0027]
The bus 12 includes a CPU (Central Processing Unit) 14, a ROM (Read Only Memory) 16, a RAM (Random Access Memory) 18, an operation unit 20, a display unit 22, an external storage device 24, a CD (Compact Disc). ) A playback unit 26, a timer 28, a DSP (digital signal processor) / sound source unit 30, a MIDI (Musical Instrument Digital Interface) interface 32, a communication interface 34, a reception unit 36, and the like are connected.
[0028]
The CPU 14 executes processes relating to reproduction control information creation, music reproduction, and the like in accordance with a program stored in the ROM 16, and the reproduction control information creation process will be described later with reference to FIGS.
[0029]
The RAM 18 includes various storage units that are used as a working area during various processes of the CPU 14. As a storage unit related to the implementation of the present invention, there is an input data storage area corresponding to the operation unit 20. Details will be described later.
[0030]
The operation unit 20 includes a keyboard capable of inputting characters, numbers, and the like, and a pointing device such as a mouse, and constitutes a GUI together with the display unit 22. The user can input various data using the keyboard and mouse while viewing the display content of the display unit 22 as necessary.
[0031]
The display unit 22 has a display panel such as a CRT type or a liquid crystal type. The display panel can display a sound waveform based on audio data or display a score based on performance data. .
[0032]
The external storage device 24 is detachable from one or more types of recording media among HD (hard disk), FD (flexible disk), CD (compact disk), DVD (digital multipurpose disk), MO (magneto-optical disk) and the like. Is. When a desired recording medium is mounted on the external storage device 24, data can be transferred from the recording medium to the RAM 18. If the mounted recording medium is writable, such as HD, FD, or CD-R (Recordable), the data in the RAM 18 can be transferred to the recording medium.
[0033]
As the program recording means, a recording medium of the external storage device 24 can be used instead of the ROM 16. In this case, the program recorded on the recording medium is transferred from the external storage device 24 to the RAM 18. Then, the CPU 14 is operated according to the program stored in the RAM 18. In this way, it is possible to easily add a program or upgrade a version.
[0034]
The CD playback unit 26 sets audio data in a state where a CD (hereinafter referred to as an audio CD) on which audio data obtained by digitally encoding an audio signal in a format compliant with the CD-DA (Compact Disc-Digital Audio) standard is set. Can be reproduced by playing a digital audio signal.
[0035]
The timer 28 generates a clock signal having a frequency corresponding to the tempo value indicated by the tempo data and supplies it to the CPU 14. The CPU 14 can receive a clock signal as an interrupt command, reproduce the performance based on the performance data by the interrupt process, and can display the score on the display panel of the display unit 22 along with the performance reproduction. As will be described later, the tempo data is created by the process of step 74 in FIG. 3, and can also be created based on the detection signal of the motion sensor 46 responding to the hand movement motion. The performance data can be input from the recording medium of the external storage device 24 and can also be input from the interface 32 or 34.
[0036]
The DSP / sound source unit 30 includes a DSP and a sound source device. The DSP performs a time-axis companding process, a dynamics control process, and the like on the digital audio signal sent from the CD playback unit 26, and these processes are performed under the control of the CPU 14. That is, the CPU 14 controls the time axis companding process and the dynamics control process in the DSP according to the beat timing signal and the dynamics control signal input from the receiving unit 36, respectively. These processes will be described later with reference to FIGS. The DSP supplies the processed digital audio signal to the sound system 38.
[0037]
In the DSP / sound source unit 30, the sound source device has a plurality of music sound generation channels, and digital music sounds according to sound generation control information such as tone color (tone type), pitch, velocity, note on / off, etc. for each channel. Signals (including human voice signals) can be generated. As the sound source method, a known method such as a waveform readout method can be employed. As described above, when the CPU 14 reproduces the performance based on the performance data by the interrupt process, the tone generator generates a digital musical tone signal according to the sound generation control information based on the performance data and supplies it to the sound system 38.
[0038]
The sound system 38 includes a D / A (digital / analog) converter, an amplifier, a speaker, and the like, and converts a digital audio signal or a digital music signal from the DSP / sound source unit 30 into an analog audio signal or an analog music signal, respectively. At the same time, the analog audio signal or analog musical sound signal related to the conversion is converted into sound.
[0039]
The MIDI interface 32 is provided for performing MIDI communication with a MIDI device 40 separate from the small computer 10. In the present invention, in order to receive performance data in the MIDI format from the MIDI device 40. Used.
[0040]
The communication interface 34 is provided to perform information communication with another computer 44 via a communication network 42 (for example, a LAN (local area network), the Internet, a telephone line, etc.). Programs and various data (for example, audio data, performance data, etc.) necessary for the implementation of the present invention are fetched from the computer 44 to the RAM 18 or the external storage device 24 via the communication network 42 and the communication interface 34 in response to a download request. Also good.
[0041]
The receiving unit 36 receives the detection signal of the motion sensor 46 via the antenna 36A, and generates a beat timing signal and a dynamics control signal based on the received detection signal. The generation processing of these signals is performed by the CPU 14. Under the control of
[0042]
As an example, the motion sensor 46 includes a three-dimensional acceleration sensor disposed in a rod-shaped main body having a diameter gradually reduced from the both ends toward the center so that the user can easily grip the motion sensor. The motion and acceleration in the axial direction (left-right direction) and z-axis direction (front-back direction) can be detected. The motion sensor 46 may be in the form of a command stick, or is not limited to one held by hand, but may be one that is attached to a hand or foot using a band or the like, or one that is built into footwear such as shoes. Alternatively, it is not limited to using a three-dimensional acceleration sensor, and a one-dimensional or two-dimensional acceleration sensor, a 1-3 dimensional speed sensor, a tilt sensor, a strain sensor, an impact sensor, or the like may be used. Signal generation processing in the receiving unit 36 based on the detection signal of the motion sensor 46 will be described later with reference to FIG.
[0043]
Next, an example of the reproduction control information creation process will be described with reference to FIGS. In step 50, the music name is displayed on the display panel of the display unit 22 for the audio data of a plurality of songs stored in the external storage device 24, and the keyboard, mouse, etc. of the operation unit 20 are operated from among the displayed song names. To select a song (target song) to be processed by designating a desired song name. What is selected at this time is, for example, a musical piece such as an orchestra and / or a song related to singing. As another example of song selection, an audio CD of a desired song may be set in the CD playback unit 26.
[0044]
In step 52, the audio data of the selected song is read from the external storage device 24 or the CD playback unit 26 into a predetermined area of the RAM 18. FIG. 5 shows the audio data AD written to the RAM 18 at this time. This data AD includes sample data SMP1, 2, 3,... Each representing a sequential sample value of the sound waveform. Next, the sound waveform of the head section is displayed on the display panel of the display unit 22 based on the audio data in the RAM 18 as shown by AW in FIG.
[0045]
In step 54, a section to be processed (processing section) is designated by operating the keyboard, mouse, etc. of the operation unit 20. When the initial display section including the head section where the sound waveform AW is displayed in step 52 is to be processed, the initial display section is designated as the processing section.
[0046]
In step 56, the sound waveform of the processing section is displayed. When the initial display section is set as the processing section in step 54, the sound waveform being displayed is not changed and the sound waveform AW of the head section is kept displayed.
[0047]
In step 58, reproduction of the performance in the processing section is permitted. For example, in response to operating the space bar on the keyboard of the operation unit 20, playback of the performance in the processing section is started. At this time, the display panel shown in FIG. 4 travels so that the cursor CS indicates the reproduction position along the cursor travel line CL parallel to the time axis t of the displayed sound waveform AW. When the performance reproduction reaches the end point of the processing section, it returns to the starting point of the processing section again and repeats the performance reproduction, and the traveling of the cursor CS also returns to the starting point of the processing section and repeats along with the performance reproduction. The user can efficiently perform an input operation described later while listening to such a playback performance and viewing the waveform outline (envelope) of the sound waveform AW and the cursor CS.
[0048]
In step 59, the display panel of FIG.₁It is determined whether or not a beat position line such as is displayed. L₁The process of displaying a beat position line such as is performed in step 72 of FIG. 3 as will be described later. When you first come to step 59 from step 52,₁Since no beat position line is displayed, the determination result in step 59 is negative (N), and the process proceeds to step 62 without performing the processes in steps 60 and 61.
[0049]
In step 62, it is determined whether or not the processing for one song has been completed. When the process for one song is completed, the user instructs the end of the process using the keyboard, mouse, or the like of the operation unit 20. When there is no instruction to end the process, the determination result in step 62 is negative (N), and the process proceeds to step 64 in FIG. The process of step 76 will be described later.
[0050]
In step 64, the first beat position is input for the sound waveform (display waveform) AW displayed on the display panel of FIG. That is, in response to the user pointing a desired beat position with a cursor (not shown) by operating the keyboard, mouse, or the like of the operation unit 20, a round mark S is displayed as a beat mark at the beat position according to the instruction.₁Is displayed, and beat time data corresponding to the beat position according to the instruction is generated based on the audio data AD and written in a predetermined area of the RAM 18. The desired beat position may be designated with the fingers of the hand.
[0051]
In step 66, the process of inputting the second beat position with respect to the display waveform AW is performed in the same manner as in step 64. As a result, the display panel of FIG. 4 has a round mark S as a beat mark at the beat position according to the user's instruction.₂Is displayed and beat time data corresponding to the beat position according to the instruction is written in a predetermined area of the RAM 18.
[0052]
In step 68, processing for inputting the number of beats between the first and second beat positions input in steps 64 and 66 is performed. That is, when the user designates a beat number such as “3” by operating the keyboard, mouse, or the like of the operation unit 20, beat number data indicating the beat number according to the instruction is written in a predetermined area of the RAM 18. The beat number input process may be performed before the beat position input process.
[0053]
In step 70, a beat position between the first and second beat positions is determined based on the beat number data in the RAM 18. For example, when the beat number “3” is instructed in step 68, S is displayed on the display panel of FIG.₁-S₂Two beat positions are determined by dividing the interval into three equal parts. Then, beat time data corresponding to each beat position related to the determination is generated based on the audio data AD and written in a predetermined area of the RAM 18. The generation and writing of the beat time data performed in steps 64 and 66 may be performed in step 70.
[0054]
In step 72, a beat position line is displayed on the display panel of FIG. 4 for each beat position related to input and determination based on the beat time data in the RAM 18. For example, S₁-S₂When the number of beats “3” is instructed between the four beat position lines L as shown in FIG.₁~ L₄Is displayed.
[0055]
In step 74, a tempo value is calculated for each predetermined beat position, and tempo data indicating the obtained tempo value is written in a predetermined area of the RAM 18. S mentioned above₁-S₂In the example between, the line L₁~ L₃The tempo value at the beat position corresponding to any of₁-S₂The time interval between (this is S₁, S₂Obtained from the beat time data of S)₁~ S₂It is obtained by dividing by the number of beats in between. As another method, for example, the line L₁~ L₂The time interval between lines L₁(Or L₂) May be the tempo value at the beat position corresponding to.
[0056]
In step 74, input of control values such as a dynamics value, a follow-up degree, and a voice number may be permitted corresponding to a desired beat position. The user can specify a desired beat position by operating the keyboard, mouse, etc. of the operation unit 20, and can input control values such as a dynamics value, a tracking degree, and a voice number for the specified beat position. The control value is written in a predetermined area of the RAM 18. The dynamics value is a control value for controlling the volume and velocity (the strength of each sound), and the follow-up degree is a control value for instructing the degree of follow-up control with respect to the detection signal of the motion sensor 46. The number is a control value for instructing a tone color (sound type) such as a percussion instrument sound or a shout generated at beat timing.
[0057]
FIG. 5 shows an example of a storage format of the reproduction control information CI stored in the RAM 18 in association with the audio data AD. 5, BN1,2,3,... Are beat number data indicating beat numbers, BT1,2,3,... Are beat time data indicating beat times, and DYN1,2,3,... Are dynamics data indicating dynamics values. , TMP1,2,3... Are tempo data indicating tempo values, FLW1,2,3... Are tracking degree data indicating tracking degrees, and VN1,2,3... Are voice number data indicating voice numbers. For each beat number data, beat time data, dynamics data, tempo data, tracking data, voice number data, and other data are arranged. As the beat time data, data representing the time from the first sample data SMP1 to the beat time in the audio data AD may be used, or data representing the address of the corresponding sample data in the audio data AD is used. Also good.
[0058]
In the example of FIG. 5, the reproduction control information CI is stored independently of the audio data AD. However, a format in which the reproduction control information CI is mixed in the audio data AD may be adopted. A CD-I (Interactive) or the like equipped with a format may be created and read from the external storage device 24 or the CD playback unit 26 into the RAM 18.
[0059]
After step 74, the process returns to step 54, and the processing after step 54 is executed in the same manner as described above.
[0060]
In step 59, the display panel of FIG.₁Since a beat position line such as “E” is displayed, the determination result is affirmative (Y), and a beat timing notification process is performed in step 60. That is, for each displayed beat position line, it is detected that the cursor CS has reached the beat position line (the playback position has reached the beat position), and a click sound is generated from the sound system 38. Change the green color to red color to notify the beat timing. The beat position line may change the display intensity instead of changing the display color.
[0061]
In step 61, the display panel of FIG.₁Since the beat position line such as is displayed, fine adjustment of the beat position is allowed. For example, beat position line L₄When the beat position indicated by is not audible or musically appropriate, the user designates an appropriate beat position with a cursor (not shown) by operating the keyboard, mouse, etc. of the operation unit 20 to indicate the beat position as an arrow. As shown by A, it can be changed appropriately. Corresponding beat time data in the RAM 18 is corrected according to the change of the beat position, and a new beat position line La is displayed for the sound waveform AW based on the beat time data related to the correction.
[0062]
In steps 64 to 72, a circle (beat mark) S₂, S₃Enter the beat position corresponding to₂-S₃Enter the number of beats between S₂-S₃A beat position data is determined, and beat time data is generated based on the audio AD for each beat position related to the input and determination, and written in a predetermined area of the RAM 18. Further, a beat position line is displayed on the display panel of FIG. 4 for each beat position related to input and determination. Thereafter, the same processing is performed on the remaining portion of the head section.
[0063]
As a process before step 64, a tempo curve TC representing a desired tempo change is drawn by operating the keyboard and mouse of the operation unit 20 in parallel with displaying the sound waveform AW on the display panel of FIG. May be displayed. This makes it easy to input the beat position and the number of beats according to the tempo. Further, in the beat number input process of step 68, in order to simplify the user's operation, the same beat number as the previous time may be input in response to operating the return key on the keyboard of the operation unit 20.
[0064]
When the processing of the head section is completed, a new processing section is designated at step 54, and the processing after step 54 is executed in the same manner as described above. Thereafter, when the end of processing for one song is instructed, the determination result in step 62 becomes affirmative (Y), and the routine proceeds to step 76.
[0065]
In step 76, as shown in FIG. 5, the reproduction control information CI for one song stored in the RAM 18 is added to the audio data AD and recorded. As an example, when the CD playback unit 26 can record on a CD-R, the playback control information CI is supplied from the RAM 18 to the CD playback unit 26 and recorded on the CD-R. As another recording example, the audio data AD and the reproduction control information CI may be supplied from the RAM 18 to the external storage device 24 and recorded on the recording medium.
[0066]
In the above example, every time two beat positions are input, the number of beats between the two beat positions is input, and the beat position between the beat positions is determined according to the input beat number. Enter multiple beat positions with the same number of beats in between and input the number of beats, and simultaneously determine the beat positions between adjacent beat positions for many beat positions according to the input beat number. Also good. In addition, the beat position is input / determined and corrected for each processing section related to the designation, but after the beat position for one song is input / determined, the beat position is corrected for each processing section related to the designation. May be performed.
[0067]
In the above example, the beat position between the two beat positions is determined by linear interpolation (equal division). However, a higher-order function such as quadratic or cubic is used for the interpolation calculation, or spline interpolation is performed. You may use it. In addition, a time distribution table may be referred to at the time of interpolation calculation. For example, in Winnawalz, the second beat of the three beats is generally faster, so the time interval of the 2-3 beats is greater than either the time interval of the 1-2 beats or the time interval of the 3-1 beats. It is getting shorter. When interpolation calculation between bar lines is performed with reference to such a time distribution table, it is possible to set the beat time in a Winner-Waltz style. A plurality of types of time distribution tables of this type may be stored so that the user can select an arbitrary one, or the user can set a desired time distribution table in advance.
[0068]
FIG. 6 shows a modification of the processing of FIGS. In this example, steps 80 to 84 are executed instead of steps 64 to 70 in the process of FIGS.
[0069]
After step 62 in FIG. 2, in step 80, an extrapolation function is selected from the menu displayed on the display panel of the display unit 22 by operating the keyboard or mouse of the operation unit 20 (shortcut keys may be set). ).
[0070]
In step 82, the number of measures and the number of beats are input. That is, the number of bars n (n is an integer such as 1, 2, 3...) And the time signature of the music (4 if the time is 4) are input by operating the keyboard, mouse, or the like of the operation unit 20, and the number of bars n. The measure number data indicating the number of beats and the beat number data indicating the number of beats are written in a predetermined area of the RAM 18.
[0071]
In step 84, beat positions corresponding to the number of measures related to the input are determined based on the measure number data and time signature data in the RAM 18 and the reproduction tempo. As the playback tempo, if the music is being played back, the playback tempo may be used, or the user may specify the playback tempo. As an example, if the number of bars n is 2 and the number of beats is 4, 8 beat positions for 2 bars are determined.
[0072]
In step 84, after the beat position is determined, beat time data corresponding to each determined beat position is generated and written in a predetermined area of the RAM 18. Then, the process proceeds to step 72 in FIG. 3 and a beat position line is displayed on the panel in FIG. 4 for each beat position related to the determination based on the beat time data in the RAM 18.
[0073]
In the above-described example, audio data digitally encoded in the CD-DA format is shown as target data for generating playback control information such as beat time data, but digital data is used in MP3 (Moving Picture Experts Group Player-Audio Layer 3) system. Encoded audio data, audio data digitally encoded by ATRAC (Adaptive Transform Acoustic Coding) method, audio data digitally encoded by TwinVQ (Transform-Domain Weighted Interleave Vector Quantization) method, or the like may be targeted. However, the present invention is not limited to this, and may be performance data such as MIDI format, and video music data such as AVI (Audio Visual Interleaved) or MPEG (Motion Picture Expert Group).
[0074]
When performing an automatic performance based on performance data in the MIDI format, the performance data is input from the external storage device 24 or the MIDI interface 32 and stored in the RAM 18. A musical score is displayed on the display panel of the display unit 22 based on the performance data of the RAM 18 and an automatic performance is performed based on the performance data of the RAM 18 by interruption processing based on the clock signal from the timer 28. The beat position can be designated in the same manner as described above while viewing the score and listening to the automatic performance, and the beat time data corresponding to the designated beat position can be stored in the RAM 18. For the video music data, for example, the beat position may be specified while watching the moving image of the conductor.
[0075]
Next, a music playback process using the playback control information created as described above will be described. Assume that the RAM 18 stores audio data AD and reproduction control information CI as shown in FIG. When the audio data AD and the reproduction control information CI are recorded on a recording medium such as a CD-R in step 76 of FIG. 2, the recording medium such as the CD-R is stored in the RAM 18 via the external storage device 24 or the CD reproducing unit 26. Audio data AD and reproduction control information CI can be read.
[0076]
When the start of music reproduction is instructed by operating the keyboard, mouse, etc. of the operation unit 20, the CPU 14 sequentially reads the sample data SMP1, 2, 3,... In the audio data AD and supplies them to the DSP of the DSP / sound source unit 30. To perform music playback. As a result, reproduced music is played from the sound system 38.
[0077]
The user can enjoy the reproduced music by controlling the tempo and dynamics of the reproduced music or adding a percussion sound or a shout by operating the motion sensor 46 in parallel with the music reproduction. The CPU 14 performs motion sensor output analysis processing based on the detection signal of the motion sensor 46 received by the receiving unit 36. In this process, if the accelerations in the x, y, and z-axis directions are αx, αy, and αz, respectively, the absolute value | α | of the acceleration is obtained according to the following equation (1).
[0078]
[Expression 1]
| Α | = (αx²+ Αy²+ Αz²)^1/2
FIG. 7A shows, as an example, a change waveform of the absolute value | α | of the acceleration obtained when a three-beat time hand movement operation is performed. In the motion sensor output analysis process, a peak P exceeding a predetermined threshold value from the change waveform in FIG.₁, P₂, P₃By detecting the timing T as shown in FIG.₁, T₂, T₃At the same time, a beat timing signal that takes H level is generated and peak P₁, P₂, P₃Dynamics control signals representing the absolute values α1, α2 and α3 of the.
[0079]
In the DSP / sound source unit 30, time axis companding processing is performed on the audio signal waveform as shown in FIG. 8 under the control of the CPU 14. FIG. 8A shows the audio signal waveform before processing as an analog waveform for convenience, and beat times t1, t2, and t3 correspond to the beat times indicated by the beat time data BT1, BT2, and BT3, respectively. FIG. 8 (b) shows a beat timing signal similar to that described above with reference to FIG. 7 (b) (based on a hand movement), where the beat times ta and tc are equal to the beat times t1 and t3, respectively. The beat time tb is delayed from the beat time t2.
[0080]
The DSP / sound source unit 30 performs time axis expansion processing so that the time interval ts between t1 and t2 matches the time interval tab between ta and tb, and sets the time interval ts between t2 and t3 between tb and tc. A time axis compression process is performed so as to match the interval tbc. As a result, an audio signal waveform (shown as an analog waveform for convenience) having time intervals tab and tbc as shown in FIG. 8C is obtained. According to such processing, it is possible to control the beat timing or tempo of the audio signal so as to follow the operation tempo such as a hand movement without changing the pitch (pitch) of the audio signal. As the time axis companding process, a known method such as a cut-and-splice method, an overlap addition method by pointer movement amount control, reverb, dither, loop repetition, or the like can be appropriately employed.
[0081]
In the DSP / sound source unit 30, dynamics control processing is performed in accordance with the dynamics control signal input from the receiving unit 36. That is, in response to the dynamics control signal indicating the peak absolute value α1, the audio signal is amplified with an amplification factor corresponding to α1 between beat times ta and tb, and the dynamics control signal sets the peak absolute value α2. In response to the instruction, the audio signal is amplified at an amplification factor corresponding to α2 between beat times tb to tc, and so on. The amplification factor is controlled according to the absolute value of the peak indicated by the dynamics control signal. Amplify the signal.
[0082]
In the above example, the follow-up degree indicated by the follow-up degree data such as FLW1 is set to 100%, but the time axis pressure using the tempo value (time interval) ts indicated by the tempo data TMP1 by setting the follow-up degree to, for example, 50%. Stretching can be performed. In this case, expansion processing is performed so that the time interval is changed from ts to (ts + tab) / 2, and compression processing is performed so that the time interval is changed from ts to (ts + tbc) / 2. In the dynamics control process, if the dynamics value indicated by the dynamics data DYN1 is A1, the audio signal is amplified at an amplification factor corresponding to (A1 + α1) / 2. As a result, tempo control and dynamics control corresponding to the following degree indicated by the following degree data can be performed.
[0083]
When data indicating a percussion instrument sound (or shout) such as a cymbal sound is stored as voice number data such as VN1, an operation sensor when the beat time indicated by the corresponding beat time data such as BT1 arrives. When the hand holding 46 performs a thrusting motion in the z-axis direction, the CPU 14 detects the thrusting motion based on the detection signal of the motion sensor 46 and enables sound generation of percussion instrument sound (or shout). Is supplied to the sound source device of the DSP / sound source unit 30. Since the sound source device supplies a digital musical tone signal representing a percussion instrument sound (or shout) to the sound system 38 according to the sound generation control information, the sound system 38 generates a percussion instrument sound (or shout).
[0084]
When performing an automatic performance based on performance data such as MIDI format, the tempo of the automatic performance can be controlled by controlling the frequency of the clock signal in the timer 28 in accordance with the tempo data such as TMP1. Further, by controlling the frequency of the clock signal in the timer 28 according to the beat timing signal based on the detection signal of the motion sensor 46, the tempo of the automatic performance can be controlled so as to follow the motion tempo such as a hand gesture motion, In that case, tempo control according to the following degree indicated by the following degree data can be performed. In addition, the dynamics of the automatic performance can be controlled by controlling the volume and velocity in accordance with the dynamics control signal based on the detection signal of the motion sensor 46, and in this case, the dynamics corresponding to the following degree indicated by the following degree data. Dynamics control can be performed.
[0085]
【The invention's effect】
As described above, according to the present invention, music is reproduced based on music data, and the flow of reproduced music is displayed as a sound waveform based on the music data, and the beat position is input regarding the flow of reproduced music related to the display. Alternatively, since correction is made, it is possible to perform input or correction work easily and in a short time while watching the display and listening to the reproduced music. Also, the beat position is determined according to the input beat position, beat number, measure number, etc., and the beat time is determined and stored for each beat position based on the music data, so the tempo is reflected. Beat timing information can be obtained accurately and quickly. Therefore, it is possible to obtain an effect of greatly improving the creation efficiency of reproduction control information such as beat timing information and tempo information.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a circuit configuration of a music playback device that executes playback control information creation processing according to an embodiment of the present invention;
FIG. 2 is a flowchart showing a part of a reproduction control information creation process.
FIG. 3 is a flowchart showing the remaining part of the process of FIG. 2;
FIG. 4 is a diagram illustrating a display state of a sound waveform, a beat position line, a tempo curve, and the like on the display unit.
FIG. 5 is a diagram illustrating a storage format of audio data and playback control information.
FIG. 6 is a flowchart showing a modification of the processing of FIGS.
7A and 7B are diagrams for explaining motion sensor output analysis processing, where FIG. 7A is a waveform diagram showing a temporal change in acceleration absolute value | α |, and FIG. 7B is a waveform diagram of a beat timing signal. It is.
8A and 8B are diagrams for explaining time-axis companding processing of an audio signal waveform, where FIG. 8A is a waveform diagram of an audio signal before processing, FIG. 8B is a waveform diagram of a beat timing signal, and FIG. ) Is a waveform diagram of the audio signal after processing.
FIG. 9 is a diagram for explaining an example of conventional automatic performance processing.
FIG. 10 is a diagram for explaining another example of conventional automatic performance processing.
[Explanation of symbols]
10: small computer, 12: bus, 14: CPU, 16: ROM, 18: RAM, 20: operation unit, 22: display unit, 24: external storage device, 26: CD playback unit, 28: timer, 30: DSP Sound source unit, 32: MIDI interface, 34: communication interface, 36: reception unit, 38: sound system, 40: MIDI device, 42: communication network, 44: other computer, 46: motion sensor.

Claims (2)

Playback means for playing music based on music data;
Display means for displaying a flow of reproduced music based on the music data;
Input means for inputting a plurality of beat positions and the number of beats between these beat positions with respect to the flow of the reproduced music;
Storage means capable of storing reproduction control information;
First determining means for determining one or a plurality of beat positions according to the number of beats related to the input by the input means between the plurality of beat positions related to the input by the input means;
A beat time is determined based on the music data for each beat position among a plurality of beat positions related to input by the input means and one or a plurality of beat positions related to determination by the first determination means, Reproduction control information creation apparatus comprising: second determination means for writing beat time information indicating a beat time related to determination into the storage means as the reproduction control information. A program used in a reproduction control information creating apparatus provided with a computer, the computer being
Playback means for playing music based on music data;
Display means for displaying a flow of reproduced music based on the music data;
Input means for inputting a plurality of beat positions and the number of beats between these beat positions with respect to the flow of the reproduced music;
Storage means capable of storing reproduction control information;
First determining means for determining one or a plurality of beat positions according to the number of beats related to the input by the input means between the plurality of beat positions related to the input by the input means;
A beat time is determined based on the music data for each beat position among a plurality of beat positions related to input by the input means and one or a plurality of beat positions related to determination by the first determination means, A program for functioning as second determination means for writing beat time information indicating a beat time related to determination into the storage means as the reproduction control information.

Images