JP3436089B2 - Music control device and storage medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a music control device for controlling a predetermined music element of music that is being automatically played.

[0002]

2. Description of the Related Art A music control device that determines a human motion such as a command motion to control the tempo and dynamics of automatically played music has been known.

Such a conventional music control device determines the beat timing, dynamics and the like designated by the user from the signal output obtained by the user swinging a baton equipped with a predetermined sensor or the like, and the determination result. Music control based on.

[0004]

However, in the conventional music control device described above, the user cannot freely select or adjust the beat timing to be determined.

That is, the beat timing is the time when the user conducts a predetermined command operation, specifically, for example, 2
When controlling the time signature music, the user normally swings the baton up and down alternately, but the beat timing determined at this time is the moment when the baton is switched from top to bottom or bottom to top (or its It is a temporary point located in the vicinity), and it was not possible for the user to freely select or adjust the time point.

[0006] The conducting method is not limited to one type and differs depending on the country or the individual. Therefore, the above-described conventional music control device cannot perform music control comfortably or accurately as the user desires. It was

The present invention has been made in view of this point, and an object of the present invention is to provide a music control device capable of performing music control comfortably or accurately as the user desires.

[0008]

In order to achieve the above object, the music control device of the present invention comprises a motion detecting means for detecting a motion of a person, an automatic playing means for automatically playing music, and an area.
Area information input means for inputting information, change means for changing a predetermined music element of the automatically played music according to the detected motion, and timing for changing the music element by the change means delay means for delaying the delay amount of the delay by the delay means, the local
Set according to the regional information input by the information input means
And setting means for setting.

As a result, even when a human motion is detected, the music element of the music being automatically played is not immediately changed,
Because it is changed after being delayed by the delay amount set in accordance with the input area information, at will of the user, it can be performed comfortably, or exactly musical control.

Here, the motion detecting means may be any device as long as it can detect the motion of a person.
For example, as will be described later in the embodiments of the present invention, it may be detected by a method using a hidden Markov model, but the present invention is not limited to this, and DP matching (dynamic programming match) may be performed.
ing) may be used for detection.

Furthermore, the motion of the person detected in this way is determined based on some information of automatically played music.
As will be described later in the embodiment of the present invention, more accurate determination may be performed using, for example, fuzzy inference.

Examples of the music element include the tempo and dynamics of the music being played.

The setting means for setting the delay amount is not limited to the setting of the delay amount itself by the user directly, but includes the means of setting the delay amount indirectly. For example, as mentioned above,
Since the commanding method differs depending on the country, the user may be allowed to select and set the country, and the delay amount may be automatically set according to the selected and set country. Also,
The user level, that is, the beginner, intermediate, advanced, etc. may be selectively set. The reason why the user can set the delay amount in this way is as follows.

That is, taking the conducting operation as an example, the timing of changing the music element, that is, the beat timing, greatly affects the feeling of the performance.

For example, the point at which the speed of the baton becomes maximum when conducting a command operation is often the time immediately after the command bar is swung down. It feels fast, but for amateurs (beginners) it feels just right.

On the other hand, when the acceleration of the baton is at a maximum, that is, when the baton is swung down, the position becomes the lowest position (the lowest point of the position), the beat timing is set according to the conducting method. It corresponds to the beat timing and is suitable for people who are accustomed to conducting, but generally, it seems that the reaction of the device is slow. In an actual orchestra and the like, the beat timing is often set when a certain amount of time has passed after the lowest point of the position or when the hand is raised.

In general, it is said that the classical orchestra follows the conducting method, and that the German orchestra among the same classical music reacts faster than the Latin orchestra. Further, there are users who want to react when the tempo is slow and want to react at the timing when they start to raise their hands, but when the tempo is fast, they want to react at the lowest point of the position.

The time to start raising the hand and the time to reach the lowest point of the position are different for a very short time at the lowest point in the case of a commanding operation in which the movement of the hand is stagnant. That is, the optimum time for detecting the beat timing is strictly different depending on the user's level, the music genre being played, and the tempo of the song.

Therefore, the beat timings of these beats can be arbitrarily changed.

Further, a motion detecting means for detecting a motion of a person, an automatic playing means for automatically playing music, and area information are inputted.
Area information inputting means for inputting, proficiency level inputting means for inputting proficiency level, changing means for changing a predetermined music element of the automatically played music according to the detected motion, and the changing means The delay means for delaying the timing at which the music element is changed and the delay amount delayed by the delay means are input by the area information input means.
It is characterized in that it has a correction means for correcting in accordance with the applied regional information and the proficiency level input by the proficiency level input means.

As a result, the delay amount is corrected according to the input regional information and the proficiency level, so that the same effect as the above can be obtained.

Further, preferably, the delay means delays the timing in consideration of a time delay until a motion of the person is detected by the motion detecting means after the motion of the person.

As a result, the delay is accurately delayed by the set or corrected delay amount, so that the above effect can be further enhanced.

The storage medium of the present invention includes a motion detection module for detecting a motion of a person, an automatic performance module for automatically playing music, and a regional information input module for inputting regional information.
Module, a change module for changing a predetermined music element of the automatically-played music according to the detected operation, and a delay module for delaying a timing at which the music element is changed by the change module, the delay amount of the delay by the delay module, the local
Depending on the regional information entered by the information entry module
It is characterized in that it includes a setting module for setting by setting.

Furthermore, a motion detection module for detecting a human motion, an automatic performance module for automatically playing music,
A regional information input module for inputting regional information, a proficiency level input module for inputting a proficiency level, and a changing module for changing a predetermined music element of the automatically played music according to the detected motion. A delay module for delaying the timing at which the music element is changed by the change module and a delay amount delayed by the delay module are set by the area information input module.
It is characterized by including a correction module that corrects according to the input regional information and the proficiency level input by the proficiency level input module.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a music control device according to an embodiment of the present invention.

As shown in the figure, the music control apparatus of the present embodiment mainly includes a keyboard 1 for inputting character information, a pointing device 2 such as a mouse for inputting position information, and the like. An acceleration sensor 31 that detects an acceleration in the x-axis (horizontal) direction of a user's motion, that is, a commanding motion performed by waving the baton 30, and an acceleration sensor that detects an acceleration in the y-axis (vertical) direction of this commanding motion. 32, a key detection circuit 3 that detects a pressed state of each key of the keyboard 1, an operation state detection circuit 4 that detects an operation state of the pointing device 2, and an acceleration sensor 31.
A-D conversion circuit (Analog-to-Digi) that converts the electrical analog acceleration signal Ax from the digital acceleration signal Dx
tal Converter) 5, an A / D conversion circuit 6 for converting an electric analog acceleration signal Ay from the acceleration sensor 32 into a digital acceleration signal Dy, and a C for controlling the entire apparatus.
A PU 7, a ROM 8 for storing a control program executed by the CPU 7, table data and the like, a RAM 9 for temporarily storing performance data, various input information including the acceleration signals Dx and Dy, a calculation result, and a timer interrupt process. 10 for measuring interrupt time and various times in
And a display device 11 for displaying various information, such as a large liquid crystal display (LCD) or CRT (Cathode RayTube) display and a light emitting diode (LED), and a floppy disk (F
D) Floppy disk drive that drives 40 (F
DD) 12, a hard disk drive (HDD) 13 that drives a hard disk (not shown) that stores various application programs including the control program and various data, and various application programs including the control program and various data. Compact disk to store-Read only memory (CD-
CD-ROM drive (C
D-ROMD) 14 and external MIDI (Musical)
Instrument Digital Interface) signal input, M
A MIDI interface (I / F) 15 that outputs as an IDI signal to the outside, and a communication network 101
A communication interface (I / F) 16 for transmitting and receiving data to and from the server computer 102, for example.
A tone generator circuit 17 for converting performance data generated by the CPU 7 into a tone signal, an effect circuit 18 for applying various effects to the tone signal from the tone circuit 17, and an effect circuit 18 To convert the sound signal of the sound into sound, for example, DAC (Digital-to-Analog Converte
r) and a sound system 19 such as an amplifier and a speaker.

The above components 3 to 18 are connected to each other via a bus 20, a CPU 10 is connected to a timer 10, a MIDI I / F 15 is connected to another MIDI device 100, and a communication I / F 16 is connected. Communication network 101
The sound source circuit 17 is connected to the effect circuit 18, and the effect circuit 18 is connected to the sound system 19.

Further, the acceleration sensors 31 and 32 are attached to the baton 30 at predetermined positions, and their output sides are respectively connected to the A / D conversion circuit 5 via a cable, for example.
6 is connected to the input side. The acceleration sensor 3
The acceleration signals Ax and Ay detected by 1, 32 are
Instead of transmitting to the A / D conversion circuits 5 and 6 by wire, it may be transmitted wirelessly.

As described above, the hard disk of the HDD 13 can also store the control program executed by the CPU 7. If the control program is not stored in the ROM 8, the control program is stored in this hard disk and stored in the hard disk. ROM by loading in RAM9
The CPU 7 can be caused to perform the same operation as when the control program is stored in 8. By doing so, it is possible to easily add or upgrade the control program.

CD-ROM of the CD-ROM drive 14
The control program and various data read from 41 are
It is stored in the hard disk in the HDD 13. This makes it easy to newly install or upgrade the control program. In addition to the CD-ROM drive 14, a device for utilizing various types of media such as a magneto-optical disk (MO) device may be provided as an external storage device.

As described above, the communication I / F 16 is connected to the communication network 101 such as a LAN (Local Area Network), the Internet, or a telephone line,
It is connected to the server computer 102 via the communication network 101. When the above-mentioned programs and various parameters are not stored in the hard disk in the HDD 13, the communication I / F 16 operates as the server computer 1
02 to download programs and parameters. The computer serving as a client (in the present embodiment, the music control device) transmits a command requesting the download of a program or parameter to the server computer 102 via the communication I / F 16 and the communication network 101. Server computer 10
The HDD 2 receives the command, distributes the requested program and parameters to the computer via the communication network 101, and the computer receives these programs and parameters via the communication I / F 16 to receive the HDD 1
Downloading is completed by accumulating in the hard disk in 3.

In addition, an interface for directly exchanging data with an external computer or the like may be provided.

As can be seen from FIG. 1, the music control apparatus according to the present embodiment is mainly configured by using a general-purpose computer, and the computer includes acceleration sensors 31, 32 of the baton 30 and a sound source section (sound source circuit 7). , The effect circuit 8 and the sound system 19) are attached externally. It should be noted that, of the components constituting the music control device of the present embodiment, which is included in the general-purpose computer depends on the computer used, and there is a computer including a sound source unit as well. Then, the sound source is external hardware.

The control processing executed by the music control device configured as described above will be described below with reference to FIGS.

FIG. 2 is a block diagram for explaining a control process mainly executed by the computer in the music control device according to the present embodiment.

In the figure, each block corresponds to each function when the entire control processing is divided into a plurality of functions, but a part of hardware (A / D conversion circuits 5 and 6, tone generator 61 and sound system) 19) is also mixed.

In the figure, the analog acceleration signal Ax output from the acceleration sensor 31 is input to the A / D conversion circuit 5 and converted into a digital acceleration signal Dx, and a band pass filter (BPF: Band Pass Filter) 51 is supplied. Thus, the noise component included in the digital acceleration signal Dx is removed and then input to the vector quantization unit 53.
Similarly, the analog acceleration signal Ay output from the acceleration sensor 32 is also converted into a digital acceleration signal Dy by the A / D conversion circuit 6, and the bandpass filter 52 is supplied.
Thus, after the noise component included in the digital acceleration signal Dy is removed, the noise component is input to the vector quantization unit 53.

First, the vector quantizer 53 calculates the absolute value of acceleration (hereinafter referred to as "absolute acceleration") from the two digital acceleration signals Dx 'and Dy' from which noise components have been removed, using the following equation (1). D ′ | is calculated and stored in a predetermined area of the RAM 9 (hereinafter referred to as “absolute acceleration storage area”). In this absolute acceleration storage area, absolute acceleration | D '
The time series data of | is stored. Hereinafter, the time waveform represented by this time-series data, that is, the waveform generated when the vertical axis is the absolute acceleration value and the horizontal axis is the time axis, is referred to as the “absolute acceleration time waveform”.

[0041]

[Equation 1] Next, the vector quantization unit 53 causes the acceleration signal Dx,
Dy and the absolute acceleration | D ′ | are extracted, and a vector quantization method is used to classify them into a plurality of types (32 types in the present embodiment) of labels,
Select one of the labels. Here, the feature amount is specifically as follows.

That is, 1) the local strength of the commanding motion and its timing (the local peak of the absolute acceleration time waveform) 2) the direction of the commanding motion (in the xy plane, that is, the x axis is taken as the acceleration value in the horizontal direction, y The quadrant in which the absolute acceleration | D '| is located in the xy plane when the axial direction is taken as the vertical acceleration value, and the x-axis positive direction of the line segment connecting the absolute acceleration | D' | and the origin O Angle) 3) Smoothness of commanding motion (high-frequency component of absolute acceleration time waveform and area of region formed by acceleration trajectory in a predetermined short time in xy plane) 4) Degree of pausing commanding motion (absolute) The depth of the trough of the acceleration time waveform and the time at the bottom of the trough).

For example, when the current state is "the local peak & command motion direction is between angles α and β", the label is "1", and a predetermined set of a plurality of characteristic quantities is set. A label is decided when the condition of matching is satisfied.

In this way, the vector quantizer 53
Is one label label at a predetermined time (for example, 10 msec) (where each of the 32 types of labels is represented by a label l1 to l32, one of the labels l1 to l32).
k) is generated and output to the HMM recognition unit 54.

The HMM recognition unit 54 stores this label label in a predetermined area of the RAM 9 (hereinafter referred to as "label storage area"), as in the vector quantization unit 53. That is, in the label storage area, the label label
The time series data of is stored. Of these, the latest plurality (R) of labels are collected to generate a time series of labels {lr}.

The HMM recognizing unit 54 then uses the hidden Markov model (HMM: HMM: HMM: HMM) of the occurrence pattern of the time series {lr} of this label, that is, the occurrence pattern of each label lr included in the time series of the label. Hidden Marko
v Models) and the probability that a time series {lr} will occur for each model Mk of the plurality of models that make up this HMM (hereinafter referred to as "likelihood"). Pr (L | Mk) is calculated,
Among them, the model M having the maximum likelihood Pr (L | Mk)
k is obtained, and the commanding motion based on this model Mk is recognized as the target commanding motion, that is, the commanding motion currently performed by the user. Here, L indicates a time series {lr}.

The HMM recognition processing performed by the HMM recognition unit 54 will be described in more detail below with reference to FIG.

As shown in FIG. 3, in this embodiment, H
As MM, for example, five discrete states (State)
A so-called LR (Left-to-Right) type 5-state HMM in which any one of Sj (1 ≦ j ≦ 5) is adopted and each state Sj transits only to the adjacent state S on the right side or its own state is adopted. There is.

In the figure, pij indicates the probability of transition from state Si to state Sj, and qij (lk) indicates the state Si to state Sj.
It shows the probability of outputting the label lk when transitioning to j. Here, the “k” of the label lk uses the same character as the “k” of the model Mk, but does not mean the same.

Then, a plurality of types of commanding motions (16 types in this embodiment) are in motion (motions corresponding to each beat when the command rod 30 is swung along each side of the triangle (three swings), and Motions corresponding to each beat when swinging up and down (two motions, etc.), and each divided motion state is associated with each model Mk. For each model Mk, that is, for each motion state First, the probability pij, the probability qij (lk), etc. are learned in advance. This learned value is stored in the HMM database (for example,
This database is stored in a hard disk 55 in the HDD 13). In addition to this, the HMM database 55 also stores information related to the model Mk.

The likelihood Pr (L | Mk) is calculated for each model Mk by the following equation (2) using the setting values learned and set in advance in this way.

[0052]

[Equation 2] However, i _{0 to} i _r indicate the subscript “i” of the state Si, and in the present embodiment, i _{0 to} i _r−1 take the range of 1 to 5, respectively. Ir represents a set of {r: Sr is the final state}, and since the final state is S ₅ in the present embodiment, Ir
= {5}. Therefore, i _r = 5. Further, r _i0 is the probability that the initial state is S _i0 , specifically,
r ₁ = 1 and r _i = 0 (i = 2, ..., 5) are shown.

Likelihood Pr (L | Mk) for each model Mk
When H is calculated, the HMM recognition unit 54 outputs the thus calculated likelihood of each model Mk to the comprehensive determination unit 58.

That is, of the likelihoods of each model Mk, the model Mk that gives the maximum likelihood may be recognized as the commanding motion at that time, but there is a possibility that it will be erroneously recognized only by recognition by the HMM. Therefore, here, only the likelihood of each model Mk is obtained, and the final determination of the motion is left to the comprehensive determination unit 58.

The HMM recognizing unit 54 also recognizes dynamics and articulation based on the characteristic amount of the command movement.

The comprehensive determination unit 58 includes an output from the beat occurrence probability prediction unit 56 and a beat occurrence probability distribution calculation unit 57.
Is also supplied, the comprehensive judgment unit 58 makes a comprehensive judgment literally based on these output states, and music element data such as beat timing and dynamics, that is, the music reproduced by the MIDI reproduction unit 59. And outputs the music element data a for controlling to the MIDI reproducing unit 59.

Of the music element data a, the performance tempo data a1 is supplied to the beat occurrence probability predicting section 56,
The beat occurrence probability prediction unit 56 uses the performance tempo data a.
Based on 1, the beat occurrence probability is predicted by a fuzzy inference (specifically, a fuzzy inference called a direct method).

Here, the direct method is generally a method of performing fuzzy inference according to the following procedure.

1) Describe a fuzzy rule 2) In the fuzzy rule of 1), the language that is a variable is represented by a membership function 3) Obtain the inference result of each fuzzy rule with respect to the input 4) Finalize from the inference result of each rule The beat occurrence probability predicting unit 56 that obtains a realistic inference result specifically calculates the beat occurrence probability by applying the direct method as follows.

For 1), if (the elapsed time from the previous beat timing based on the performance tempo is short) th
Adopt a fuzzy rule called en (probability of beat occurrence).

Regarding 2), the antecedent portion, that is, the portion of the fuzzy rule "the elapsed time from the previous beat timing based on the playing tempo is short" is shown in FIG. 4 (a).
The membership function shown in FIG. 4B is defined, and the membership function shown in FIG. 4B is defined for the consequent part, that is, for the part of the fuzzy rule where "the possibility of beat occurrence is low".

Regarding 3), the elapsed time based on the performance tempo from the previous beat is calculated based on the input performance tempo data a1 (for example, the time when the current tempo is converted into a predetermined reference tempo). Then, using the membership function of FIG. 4A, the grade for the calculated elapsed time is obtained.

When the antecedent part of the fuzzy rule is composed of a plurality of elements connected by "and", the minimum value of the grade obtained from each membership function (this is called "fitness"). ) Is obtained (this calculation is called "" min "calculation").

However, the beat occurrence probability prediction unit 56
Since the antecedent part of the fuzzy rule is composed of one element, the grade obtained with respect to the elapsed time corresponds to the compatibility.

For example, when the elapsed time is t1, FIG.
As shown in (a), since the grade is determined to be g1, the compatibility is g1.

Then, the membership function of the consequent part, that is, the membership function of FIG. 4 (b), which is obtained by cutting off the portion above the above-obtained goodness of fit, is the inference result of the fuzzy rule of 1). Become. For example, if the goodness of fit is g1, the inference result obtained is the portion indicated by the region R1 in FIG.

Regarding 4), when a plurality of types of fuzzy rules in 1) are defined, the inference result in 3) is obtained by the number of each rule. Areas are overlapped (this operation is called "max" operation). The center of gravity of the regions thus superposed is obtained, the perpendicular line is extended from the center of gravity, and the value of the foot is the final inference result.
This is called the center of gravity method.

For example, in FIG. 4B, the center of gravity of the region R1 is obtained as G1, and the occurrence probability corresponding to this center of gravity G1 is obtained as p1. At this time, since the occurrence probability p1 is a numerical value used for control, it is simultaneously defuzzified (Defu
zzify) has also been made.

The beat occurrence probability predicting unit 56 outputs the occurrence probability thus obtained to the comprehensive judging unit 58 as the beat occurrence probability.

The beat occurrence probability distribution calculation unit 57 displays MI
The MIDI data supplied from the DI database (for example, this database is built on the hard disk in the HDD 13) 60 is supplied to the MIDI reproducing unit 59, and the beat occurrence probability distribution calculating unit 57 is as shown in FIG. The membership function is defined, and the beat occurrence probability distribution on the MIDI data is calculated and output by fuzzy inference using the supplied MIDI data and this membership function.

Specifically, as a fuzzy rule, if
(The part being played is close to the beats and notes of MIDI data.) Th
Adopting the rule of en (beat input is likely to be equivalent to that), the membership function of FIG. 5A is defined as the membership function. As shown in FIG. 5A, each membership function is defined so that the grade takes the maximum value at the beat timing of MIDI data. On the other hand, since the MIDI data does not include information indicating the beat position, it is considered that the beat is likely to occur at the beat timing, and the maximum grade is set at the beat timing based on the MIDI data. , Define each membership function. In addition, when it is desired to set the beat position at a position other than the beat timing, as shown in FIG.
The user may manually set the membership function.

When the fuzzy rule and each membership function are defined in this way, the subsequent fuzzy inference is the same as the above-described fuzzy inference in the beat occurrence probability prediction unit 56, and therefore its description is omitted. To do.

The beat occurrence probability distribution calculation unit 57 outputs the calculated beat occurrence probability to the comprehensive determination unit 58.

The comprehensive judging section 58 is the HMM recognizing section 54.
Likelihood of each model Mk from, and beat occurrence probability prediction unit 56
Based on the beat occurrence probability from the beat occurrence probability distribution, and the beat occurrence probability distribution from the beat occurrence probability distribution calculation unit 57, the command action is comprehensively determined by fuzzy inference.

Specifically, if is used as a fuzzy rule, if
(In the HMM recognition unit 54, the likelihood of the HMM corresponding to the nth Nth is high) and (In the beat occurrence probability prediction unit 56, there is a high probability of beat occurrence) and (Calculation of beat occurrence probability distribution) Part 57 adopts the rule that the place being played is near the nth beat of the N beat) then (the beat input is likely to be the Nth beat of the Nth beat), and as a membership function thereof, The membership functions shown in FIGS. 6A to 6D (the illustrated example shows the case where N = 3 and n = 2) are adopted.

Since the subsequent fuzzy inference method is the same as the method of the beat occurrence probability predicting section 56 described above, its explanation is omitted.

The performance tempo can be calculated based on the intervals between the beat timings.

In this way, the comprehensive judgment unit 58 performs fuzzy inference based on the input information to further improve the recognition rate of the command motion recognized by the HMM recognition unit 54, and the judgment result, that is, The music element data a
MIDI playback unit 5 (performance tempo, beat timing, beat type, dynamics, articulation, etc.)
Output to 9.

The MIDI reproducing section 59 determines the timing at which the note-on / off event occurs based on the MIDI data supplied from the MIDI database 60, and at the determined timing, the note-on / off occurs. Information to be transmitted to the sound source unit 61,
That is, the MIDI data is reproduced, and the MI being reproduced is reproduced according to the music element data a from the comprehensive determination unit 58.
Change the corresponding music element of the DI data. Specifically, when the music element data is beat timing, the beat timing or note position of the MIDI data being reproduced is changed so as to match the timing and the beat type, and when the music element data is dynamics,
The dynamics (velocity or volume) of the MIDI data being played back is changed so as to match the dynamics.

When the music element data is the performance tempo, the tempo at which the MIDI data is reproduced is changed. Also, when the music element data is articulation, the gate time length of the MIDI data being reproduced is changed.

The MIDI reproducing unit 59 uses the changed MI.
The DI data is output to the tone generator 61, and in response to this, the tone generator 61 generates a tone signal from the MIDI data and outputs it to the sound system 19.

The sound system 19, as described above,
The generated musical tone signal is converted into sound.

FIG. 7 is a block diagram showing a detailed structure of the MIDI reproducing section 59. In FIG. 7, for convenience of explanation, M is shown.
Elements other than the IDI reproducing unit 59, that is, the area setting unit 80
The skill level setting unit 81 is also illustrated. Note that each block corresponds to each function when the control processing executed by the MIDI reproducing unit 59 is divided into a plurality of functions.

The MIDI reproducing unit 59 includes a beat position correcting unit 71 for correcting the beat position (timing) and a tempo calculating unit 72 for calculating a tempo according to the corrected beat position.
And a MIDI file reproducing unit 73 that reads a MIDI file corresponding to a song designated by the user, sequentially reads out and reproduces MIDI data included in the MIDI file, and the dynamics that the user has instructed to change by the command operation as described above. Accordingly, the dynamics control unit 74 controls the dynamics of the music.

Area information set by the user is input to the beat position correcting section 71 from the area information setting section 80 capable of setting areas such as Germany, Latin and Japan. For example, beginner, intermediate, advanced, etc. The proficiency level setting unit 81 capable of setting the proficiency level of the user inputs the proficiency level information set by the user, the beat determination timing data a2 is input from the comprehensive determination unit 58, and the tempo calculation unit 7
The tempo value calculated in 2 is fed back and input.

Here, the information input by the user in the area information setting unit 80 and the proficiency level setting unit 81 is the same as the information input in a normal computer in the present embodiment.
From the input screen displayed on the display device 11, the keyboard 1
However, the present invention is not limited to this, and hardware for exclusive use of information input such as a panel board of an electronic musical instrument may be externally attached and used.

FIG. 8 is a diagram showing a more detailed structure of the beat position correcting section 71.

In the figure, the beating position correcting section 71 stores a region-delay amount correction coefficient table 91 in which delay amount correction coefficients are stored for each region, and a delay amount correction coefficient for each proficiency level. Skill level-delay amount correction coefficient table 92
A delay amount calculation unit 93 that calculates a delay amount according to the tempo value calculated by the tempo calculation unit 72; a multiplier 94 that multiplies the delay data output from each of the constituent elements 91 to 93; The beat position delay unit 95 delays the beat position based on the delay amount from the multiplier 94.

Area-delay amount correction coefficient table 9
1 has the following data characteristics.

That is, as described above, it is said that the German orchestra responds to the commanding action faster than the Latin orchestra. Therefore, for example,
“0.8” for Germany, “1.
2 "and" 1.0 "are stored corresponding to Japan.

Further, the skill level-delay amount correction coefficient table 9
2 has the following data characteristics.

That is, as described above, the beginner can feel that the response of the device to the commanding action is fast, and that the response is slower as he / she is advanced. For example, "0.9" is stored for the beginner, "1.0" is stored for the intermediate, and "1.1" is stored for the advanced.

The delay amount calculation unit 93 calculates the delay amount del according to the following expression (3) based on the input tempo value tempo.
ay is calculated and output to the multiplier 94.

Delay = coef × tempo + const (3) However, coef represents a proportional coefficient and const represents a constant.

Here, the delay amount delay is calculated in consideration of the delay time from the output of the acceleration signals Ax, Ay from the acceleration sensors 31, 32 to the input of the music element data a to the MIDI reproducing section 59. It Specifically, the constant const is determined in consideration of this delay time.

Since this delay time changes according to the calculation processing capacity of the CPU 7, the calculation capacity of the CPU 7 may be obtained in advance and the delay amount may be corrected according to the calculation capacity. Specifically, when the CPU 7 is high speed, the delay amount is increased, while when the CPU 7 is low speed, the delay amount is decreased.

FIG. 9 illustrates this calculation formula.

As shown in the figure, in the present embodiment, the delay amount delay and the tempo value tempo have a linear (proportional) relationship. In the figure, the slope of the straight line is the proportional coefficient c.
The delay amount when the tempo value tempo is “0” corresponds to the constant ocon.

The delay amount calculation unit 93 stores the proportional coefficient coef and the constant const and stores the delay amount delay by actually performing the calculation shown in the above equation (3) on the input tempo value tempo. However, the characteristics shown by the straight line in FIG. 9 are stored as table data, that is, the delay amount delay for each tempo value tempo is stored as table data, and this table data is stored as the input tempo value. The delay amount delay may be obtained by searching according to tempo.

The delay amount delay and the tempo value tem
As described above, po does not always have to have a linear relationship and may have a non-linear relationship.

Returning to FIG. 8, in the multiplier 94, the delay amount correction coefficient retrieved from the area-delay amount correction coefficient table 91 in accordance with the area information from the area information setting unit 80 and the proficiency level setting unit 81. The delay amount correction coefficient retrieved from the proficiency level-delay amount correction coefficient table 92 and the delay amount delay from the delay amount calculation unit 93 are supplied and multiplied in accordance with the proficiency information from , Beat position delay unit 95
Is supplied to.

Further, the beat position delay unit 95 is also supplied with the beat timing data a1 from the comprehensive judgment unit 58, and the beat position delay unit 95 multiplies the beat position indicated by the beat timing data a2 by the multiplier 94. The information to be delayed according to the delay amount from is output to the tempo calculation unit 72.

In this embodiment, the delay amount calculation unit 9
The delay amount calculated in 3 is multiplied by the delay amount correction coefficient according to the area information and the delay amount correction coefficient according to the proficiency level, and the multiplication result is output to the beat position delay unit 95. The output value to the beat position delay unit 95 is not limited to
It may be obtained by an arithmetic expression or a table.

Returning to FIG. 7, in response to the information for delaying the beat position output from the beat position correcting unit 71 in this way, based on the time interval at which the beat position occurs,
The tempo calculation unit 72 calculates the tempo designated by the user and outputs the tempo value to the MIDI file reproduction unit 73.

The MIDI file reproducing section 73 is also supplied with the dynamics control signal from the dynamics controlling section 74, and the MIDI file reproducing section 73 receives the MIDI file being reproduced.
The tempo of the data is changed based on the information from the tempo calculation unit 72, and the dynamics of the MIDI data is changed based on the information from the dynamics control unit 74.

The dynamics control section 74 is supplied with the dynamics signal a2 from the comprehensive determination section 58, and in response to this signal, the dynamics control section 74 controls to change the dynamics of the MIDI data being reproduced. Output information.

As described above, in the present embodiment, the delay amount for delaying the beat position can be freely set by the user in accordance with the region and the level of proficiency, and the beat position can be delayed according to the set value. Therefore, music control can be performed comfortably or accurately as the user desires.

In this embodiment, the delay amount is automatically changed according to the tempo, but in addition to this, the delay amount is automatically changed according to the genre of the song, the position in the song, the time signature, etc. It may be changed to.

The delay amount may be continuously or discretely adjustable so that the user can arbitrarily adjust it. The adjustment amount may be associated with the proficiency level of the user so that the user can select it according to his or her proficiency level.

Further, M is set so that the beat position delayed by the beat position delay unit 95 matches the beat timing of the performance.
The IDI file reproducing unit 73 may be controlled.

In this embodiment, the command operation is performed by the HM.
Although the recognition is performed by M, the recognition is not limited to this, and may be recognized by a method using the DP matching, for example.

Further, in the present embodiment, the baton 30 is provided with the acceleration sensors 31 and 32 and the commanding motion is detected in accordance with the acceleration obtained therefrom, but the invention is not limited to this, and the angular velocity sensor ( A command operation may be detected according to a signal detected using a gyro sensor), a sensor using magnetism or light, or the sensor may be directly attached to a hand or a finger.

Furthermore, in the present embodiment, the music control device is mainly configured by using a general-purpose computer, but the present invention is not limited to this, and it may be configured by a dedicated device.

A storage medium in which a software program for realizing the functions of the above-described embodiments is recorded is
Needless to say, the object of the present invention can be achieved by supplying the system or device to the computer (or the CPU 7 or MPU) of the system or device to read and execute the program stored in the storage medium.

In this case, the program itself read from the storage medium realizes the novel function of the present invention.
The storage medium storing the program constitutes the present invention.

As a storage medium for supplying the program, for example, the hard disk of the HDD 13 or C
D-ROM 41, MO, MD, floppy disk 4
0, CD-R (CD-Recordable), magnetic tape, non-volatile memory card, ROM 8 and the like can be used. Further, the program may be supplied from the server computer 102 via another MIDI device 100 or the communication network 101.

Further, not only the functions of the above-described respective embodiments are realized by executing the program read by the computer, but also the OS or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that a case where a part or all of the processing of (1) is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function is executed based on the instruction of the program. It goes without saying that the case where the CPU 7 or the like included in the expansion board or the function expansion unit performs some or all of the actual processing and the processing realizes the functions of the above-described embodiments is also included.

[0119]

As described above, according to the present invention,
Even when human movement is detected, the music elements of the music being played automatically are not changed immediately, and the music elements according to the entered regional information are not changed.
Because it is changed after being delayed by the delay amount set Te, at will of the user, it can be performed comfortably, or exactly musical control.

Further, since the delay amount is corrected according to the input regional information and proficiency level, the same effect as the above effect can be obtained.

Further, preferably, the timing is delayed in consideration of a time delay after a person operates and before the operation is detected, so that the timing is delayed exactly by the set or corrected delay amount, Therefore, the above effect can be further enhanced.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a music control device according to an embodiment of the present invention.

FIG. 2 is a block diagram for explaining control processing mainly executed by a computer in the music control device in FIG.

FIG. 3 is a diagram for explaining in detail an HMM recognition process performed by the vector quantization unit in FIG.

FIG. 4 is a diagram showing an example of a membership function used when the beat occurrence probability prediction unit of FIG. 2 performs fuzzy inference.

5 is a diagram showing an example of a membership function used when the beat occurrence probability distribution calculation unit of FIG. 2 performs fuzzy inference.

FIG. 6 is a diagram showing an example of a membership function used when the comprehensive judgment unit of FIG. 2 performs fuzzy inference.

FIG. 7 is a block diagram showing a detailed configuration of a MIDI reproducing unit in FIG.

8 is a diagram showing a more detailed configuration of a beat position correction unit in FIG.

9 is a diagram for explaining a delay amount calculation process performed by a delay amount calculation unit in FIG.

[Explanation of symbols]

1 keyboard (setting means) 2 pointing device (setting means) 7 CPU (motion detection means, automatic performance means, change means,
Setting means, compensation means) 11 display unit (setting unit) 31 acceleration sensor (operation detecting means) 32 acceleration sensor (motion detection means)

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G10H 1/00-7/00

Claims (5)

(57) [Claims] 1. A motion detection means for detecting a motion of a person, an automatic performance means for automatically playing music, a regional information input means for inputting regional information, and the automatic performance according to the detected motion. Changing means for changing a predetermined music element of the music being played, delay means for delaying a timing at which the music element is changed by the changing means, and a delay amount delayed by the delay means for the area information input means. And a setting means for setting according to the regional information inputted by the music control device. 2. A motion detection means for detecting a motion of a person, an automatic performance means for automatically playing music, a regional information input means for inputting regional information, a proficiency level input means for inputting a proficiency level, and the detection. Changing means for changing a predetermined music element of the automatically played music, delay means for delaying a timing at which the music element is changed by the changing means, and delay means for delaying by the delay means. musical control apparatus for an amount of delay, and having a correction means for correcting according to the input learning level by the entered local information and the learning level input means by said area information input means. 3. The delay means delays the timing in consideration of a time delay until a motion of the person is detected by the motion detecting means after the motion of the person. The music control device according to any one of 1. 4. A motion detection module for detecting a motion of a person, an automatic performance module for automatically playing music, a regional information input module for inputting regional information, and the automatic performance according to the detected motion. A change module for changing a predetermined music element of the music being played, a delay module for delaying a timing at which the music element is changed by the change module, and a delay amount delayed by the delay module for the area information input module. A storage medium storing a program that can be realized by a computer, including a setting module that sets according to the regional information input by. 5. A motion detection module for detecting a motion of a person, an automatic performance module for automatically playing music, a region information input module for inputting region information, a proficiency level input module for inputting a proficiency level, and the detection. A change module for changing a predetermined music element of the automatically-played music according to the performed operation, a delay module for delaying a timing at which the music element is changed by the change module, and a delay module for delaying the change element. the amount of delay, the
The regional information entered by the regional information input module
And a correction module that corrects according to the proficiency level input by the proficiency level input module.