JP3376866B2 - Music control device and storage medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a music control device for controlling a predetermined music element of music that is being automatically played.

[0002]

2. Description of the Related Art A music control device for determining a human motion such as a command motion to control the tempo and dynamics of automatically played music has been known.

Such a conventional music control device determines the beat timing, dynamics and the like designated by the user from the signal output obtained by the user swinging a baton equipped with a predetermined sensor or the like, and the determination result. Music control based on.

[0004]

However, in the above-mentioned conventional music control device, although the recognition rate of the motion is high when the recognition target is small, for example, the timing of the beat, the type of the beat (the second beat of three beats). Etc.), when the number of objects to be recognized such as dynamics is increased and various elements are recognized to try to improve the expressive power of music, the user may not be recognized as intended.

This is because even if the person's movement has the same intention, it varies depending on the rhythm before and after that and the person's individuality.
This is because the above conventional music apparatus does not take that point into consideration.

The present invention has been made in view of this point, and it is possible to improve the recognition rate even when there are many objects to be recognized, thereby further improving the expressive power of music. The purpose is to provide.

[0007]

In order to achieve the above object, the music control device of the present invention is generated based on human motion.
From signal, a motion recognition means for recognizing the motion candidates of human, and automatic performance means for automatically playing music, and music element extracting means for extracting a predetermined musical element from the music to be the playing issued extract According to the music element and the motion recognition means
That based on the recognition result by the fuzzy inference, and having a fuzzy determining means for determining the behavior of the person.

[0008] Thus, based on the operation candidate who is the musically elements and recognition extracted from the music to be automatically played, the fuzzy inference, because the recognized human motion candidates <br/> is determined The recognition rate is improved even when there are many recognition targets, and thus the expressiveness of music can be further improved.

Here, the motion recognition means recognizes a human motion candidate .
Anything can be used as long as it can be recognized . For example, as will be described later according to the embodiment of the present invention, it may be detected by a method using a hidden Markov model, and the present invention is not limited to this, and DP matching (dynamic programi
ng matching) may be used for detection.

Examples of the music element include the tempo, beat timing, and note position of the music to be played.

Any kind of fuzzy inference may be used, and in the embodiment of the present invention, a method called a direct method is adopted for explanation. Further, the membership function defined at that time is not limited to a specific function.

Further, in order to achieve the above-mentioned object, the music control device of the present invention is a signal generated based on a human motion.
, A motion to recognize a beat motion candidate in a human command motion
Based on the recognition means and the performance tempo of the automatic performance , the beat
A fuzzy decision means including a beat striking occurrence possibility estimating means for estimating the occurrence probability of beats hitting at the time of hitting motion candidates is recognized, the recognition result by the motion recognizing means, and
Based on the predicted beat occurrence probability, fuzzy estimation
The logical, it characterized <br/> having a ones determining the Personality beat striking operation.

[0013] Thus, based on the performance tempo of the automatic performance, is expected to occur the possibility of beats hitting at the time of the beat striking motion candidates recognized by operation recognition means, the predicted beat striking occurrence likelihood and the operation recognition Recognition by means
Since the motion of the person is determined by fuzzy reasoning based on the result, the recognition rate can be further improved.

Further, preferably, the motion recognition means
Therefore, the motion candidate of the person recognized is the beat in the command motion.
A striking motion candidates, the predetermined music element is the position of the beat timing or notes of music to be the playing the
Fuzzy determination means, based on the extracted beat timing or note positions, including the occurrence possibility beats hitting occurrence likelihood calculating means for calculating a distribution of beats hitting at the time of the beat striking motion candidates is recognized, the gesture recognition Recognition by means
It is characterized in that the pulsing motion of the person is determined based on the result and the calculated beat occurrence probability distribution.

[0015] Thus, the predetermined music elements as beat timing or note position is extracted, based on the beat timing or note position, occurrence of a beat striking at the time the beat striking motion candidates is recognized by the motion recognizing means Since the probability distribution is calculated and the motion of the person is determined by fuzzy inference based on the calculated beat occurrence probability distribution and the recognition result by the motion recognition means , the recognition rate can be further improved. .

Further, the storage medium of the present invention includes a motion recognition module for recognizing a motion candidate of a person from a signal generated based on a motion of a person, an automatic performance module for automatically playing music, and the performance. A music element extraction module that extracts a predetermined music element from music, and a fuzzy determination module that determines the motion of the person by fuzzy inference based on the extracted music element and the recognition result by the motion recognition means A program, a computer
It is characterized by storing a program that can be implemented by a computer .

This makes it possible to obtain the same effects as those described above.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a music control device according to an embodiment of the present invention.

As shown in the figure, the music control apparatus of this embodiment mainly includes a keyboard 1 for inputting character information, a pointing device 2 such as a mouse for inputting position information, An acceleration sensor 31 that detects an acceleration in the x-axis (horizontal) direction of a user's motion, that is, a commanding motion performed by waving the baton 30, and an acceleration sensor that detects an acceleration in the y-axis (vertical) direction of this commanding motion. 32, a key detection circuit 3 that detects a pressed state of each key of the keyboard 1, an operation state detection circuit 4 that detects an operation state of the pointing device 2, and an acceleration sensor 31.
A-D conversion circuit (Analog-to-Digi) that converts the electrical analog acceleration signal Ax from the digital acceleration signal Dx
tal Converter) 5, an A / D conversion circuit 6 for converting an electric analog acceleration signal Ay from the acceleration sensor 32 into a digital acceleration signal Dy, and a C for controlling the entire apparatus.
A PU 7, a ROM 8 for storing a control program executed by the CPU 7, table data and the like, a RAM 9 for temporarily storing performance data, various input information including the acceleration signals Dx and Dy, a calculation result, and a timer interrupt process. 10 for measuring interrupt time and various times in
And a display device 11 for displaying various information, such as a large liquid crystal display (LCD) or CRT (Cathode RayTube) display and a light emitting diode (LED), and a floppy disk (F
D) Floppy disk drive that drives 40 (F
DD) 12, a hard disk drive (HDD) 13 that drives a hard disk (not shown) that stores various application programs including the control program and various data, and various application programs including the control program and various data. Compact disk to store-Read only memory (CD-
CD-ROM drive (C
D-ROMD) 14 and external MIDI (Musical)
Instrument Digital Interface) signal input, M
A MIDI interface (I / F) 15 that outputs as an IDI signal to the outside, and a communication network 101
A communication interface (I / F) 16 for transmitting and receiving data to and from the server computer 102, for example.
A tone generator circuit 17 for converting performance data generated by the CPU 7 into a tone signal, an effect circuit 18 for applying various effects to the tone signal from the tone circuit 17, and an effect circuit 18 To convert the sound signal of the sound into sound, for example, DAC (Digital-to-Analog Converte
r) and a sound system 19 such as an amplifier and a speaker.

The above components 3 to 18 are mutually connected via a bus 20, a CPU 10 is connected to a timer 10, a MIDI I / F 15 is connected to another MIDI device 100, and a communication I / F 16 is connected. Communication network 101
The sound source circuit 17 is connected to the effect circuit 18, and the effect circuit 18 is connected to the sound system 19.

Further, the acceleration sensors 31 and 32 are attached to the baton 30 at predetermined positions, and the output sides thereof are respectively connected to the A / D conversion circuit 5 via a cable, for example.
6 is connected to the input side. The acceleration sensor 3
The acceleration signals Ax and Ay detected by 1, 32 are
Instead of transmitting to the A / D conversion circuits 5 and 6 by wire, it may be transmitted wirelessly.

As described above, the hard disk of the HDD 13 can also store the control program executed by the CPU 7. If the control program is not stored in the ROM 8, the control program is stored in this hard disk and stored in the hard disk. ROM by loading in RAM9
The CPU 7 can be caused to perform the same operation as when the control program is stored in 8. By doing so, it is possible to easily add or upgrade the control program.

CD-ROM of the CD-ROM drive 14
The control program and various data read from 41 are
It is stored in the hard disk in the HDD 13. This makes it easy to newly install or upgrade the control program. In addition to the CD-ROM drive 14, a device for utilizing various types of media such as a magneto-optical disk (MO) device may be provided as an external storage device.

As described above, the communication I / F 16 is connected to the communication network 101 such as a LAN (Local Area Network), the Internet, or a telephone line,
It is connected to the server computer 102 via the communication network 101. When the above-mentioned programs and various parameters are not stored in the hard disk in the HDD 13, the communication I / F 16 operates as the server computer 1
02 to download programs and parameters. The computer serving as a client (in the present embodiment, the music control device) transmits a command requesting the download of a program or parameter to the server computer 102 via the communication I / F 16 and the communication network 101. Server computer 10
The HDD 2 receives the command, distributes the requested program and parameters to the computer via the communication network 101, and the computer receives these programs and parameters via the communication I / F 16 to receive the HDD 1
Downloading is completed by accumulating in the hard disk in 3.

In addition, an interface for directly exchanging data with an external computer or the like may be provided.

As can be seen from FIG. 1, the music control device of the present embodiment is mainly constituted by using a general-purpose computer, and the acceleration sensors 31, 32 of the baton 30 and the sound source section (sound source circuit 7) are provided in this computer. , The effect circuit 8 and the sound system 19) are attached externally. It should be noted that, of the components constituting the music control device of the present embodiment, which is included in the general-purpose computer depends on the computer used, and there is a computer including a sound source unit as well. Then, the sound source is external hardware.

The control processing executed by the music control device configured as described above will be described below with reference to FIGS.

FIG. 2 is a block diagram for explaining a control process mainly executed by the computer in the music control device according to the present embodiment.

In the figure, each block corresponds to each function when the entire control process is divided into a plurality of functions, but a part of hardware (A / D conversion circuits 5 and 6, tone generator 61 and sound system) 19) is also mixed.

In the figure, the analog acceleration signal Ax output from the acceleration sensor 31 is input to the A / D conversion circuit 5 and converted into a digital acceleration signal Dx, and a band pass filter (BPF: Band Pass Filter) 51 is supplied. Thus, the noise component included in the digital acceleration signal Dx is removed and then input to the vector quantization unit 53.
Similarly, the analog acceleration signal Ay output from the acceleration sensor 32 is also converted into a digital acceleration signal Dy by the A / D conversion circuit 6, and the bandpass filter 52 is supplied.
Thus, after the noise component included in the digital acceleration signal Dy is removed, the noise component is input to the vector quantization unit 53.

First, the vector quantizer 53 calculates the absolute value of the acceleration (hereinafter referred to as "absolute acceleration") from the two digital acceleration signals Dx 'and Dy' from which the noise component is removed, according to the following equation (1). D ′ | is calculated and stored in a predetermined area of the RAM 9 (hereinafter referred to as “absolute acceleration storage area”). In this absolute acceleration storage area, absolute acceleration | D '
The time series data of | is stored. Hereinafter, the time waveform represented by this time-series data, that is, the waveform generated when the vertical axis is the absolute acceleration value and the horizontal axis is the time axis, is referred to as the “absolute acceleration time waveform”.

[0033]

[Equation 1] Next, the vector quantization unit 53 causes the acceleration signal Dx,
Dy and the absolute acceleration | D ′ | are extracted, and a vector quantization method is used to classify them into a plurality of types (32 types in the present embodiment) of labels,
Select one of the labels. Here, the feature amount is specifically as follows.

That is, 1) the local strength of the commanding motion and its timing (local peak of the absolute acceleration time waveform) 2) the direction of the commanding motion (in the xy plane, that is, the x axis is taken as the acceleration value in the horizontal direction, y The quadrant in which the absolute acceleration | D '| is located in the xy plane when the axial direction is taken as the vertical acceleration value, and the x-axis positive direction of the line segment connecting the absolute acceleration | D' | and the origin O Angle) 3) Smoothness of commanding motion (high-frequency component of absolute acceleration time waveform and area of region formed by acceleration trajectory in a predetermined short time in xy plane) 4) Degree of pausing commanding motion (absolute) The depth of the valley of the acceleration time waveform and the time at the bottom of the valley).

For example, if the current state is "local peak & command motion direction is between angles α and β", the label is "1", and a predetermined set of a plurality of characteristic quantities is set. A label is decided when the condition of matching is satisfied.

In this way, the vector quantizer 53
Is one label label at a predetermined time (for example, 10 msec) (where each of the 32 types of labels is represented by a label l1 to l32, one of the labels l1 to l32).
k) is generated and output to the HMM recognition unit 54.

The HMM recognizing unit 54 stores the label label in a predetermined area of the RAM 9 (hereinafter referred to as "label storage area"), as in the vector quantizing unit 53. That is, in the label storage area, the label label
The time series data of is stored. Of these, the latest plurality (R) of labels are collected to generate a time series of labels {lr}.

The HMM recognizing unit 54 then uses the hidden Markov model (HMM: HMM: HMM) for the occurrence pattern of the time series {lr} of this label, that is, the occurrence pattern of each label lr included in the time series of the label. Hidden Marko
v Models) and the probability that a time series {lr} will occur for each model Mk of the plurality of models that make up this HMM (hereinafter referred to as "likelihood"). Pr (L | Mk) is calculated,
Among them, the model M having the maximum likelihood Pr (L | Mk)
k is obtained, and the commanding motion based on this model Mk is recognized as the target commanding motion, that is, the commanding motion currently performed by the user. Here, L indicates a time series {lr}.

The HMM recognition processing performed by the HMM recognition unit 54 will be described in more detail below with reference to FIG.

As shown in FIG. 3, in this embodiment, H
As MM, for example, five discrete states (State)
A so-called LR (Left-to-Right) type 5-state HMM in which any one of Sj (1 ≦ j ≦ 5) is adopted and each state Sj transits only to the adjacent state S on the right side or its own state is adopted. There is.

In the figure, pij indicates the probability of transition from state Si to state Sj, and qij (lk) indicates the state Si to state Sj.
It shows the probability of outputting the label lk when transitioning to j. Here, the “k” of the label lk uses the same character as the “k” of the model Mk, but does not mean the same.

Then, a plurality of types (16 types in the present embodiment) of commanding motions (motions corresponding to each beat when the command rod 30 is swung along each side of the triangle (three swings)), Motions corresponding to each beat when swinging up and down (two motions, etc.), and each divided motion state is associated with each model Mk. For each model Mk, that is, for each motion state First, the probability pij, the probability qij (lk), etc. are learned in advance. This learned value is stored in the HMM database (for example,
This database is stored in a hard disk 55 in the HDD 13). In addition to this, the HMM database 55 also stores information related to the model Mk.

The likelihood Pr (L | Mk) is calculated for each model Mk by the following equation (2) using the setting values learned and set in advance in this way.

[0044]

[Equation 2] However, i _{0 to} i _r indicate the subscript “i” of the state Si, and in the present embodiment, i _{0 to} i _r−1 take the range of 1 to 5, respectively. Ir represents a set of {r: Sr is the final state}, and since the final state is S ₅ in the present embodiment, Ir
= {5}. Therefore, i _r = 5. Further, r _i0 is the probability that the initial state is S _i0 , specifically,
r ₁ = 1 and r _i = 0 (i = 2, ..., 5) are shown.

Likelihood Pr (L | Mk) for each model Mk
When H is calculated, the HMM recognition unit 54 outputs the thus calculated likelihood of each model Mk to the comprehensive determination unit 58.

That is, of the likelihoods of each model Mk, the model Mk that gives the maximum likelihood may be recognized as the commanding motion at that time, but there is a possibility that it will be erroneously recognized only by recognition by the HMM. Therefore, here, only the likelihood of each model Mk is obtained, and the final determination of the motion is left to the comprehensive determination unit 58.

The HMM recognition unit 54 also recognizes dynamics and articulation based on the characteristic amount of the commanding motion.

The comprehensive determination unit 58 includes an output from the beat occurrence probability prediction unit 56 and a beat occurrence probability distribution calculation unit 57.
Is also supplied, the comprehensive judgment unit 58 makes a comprehensive judgment literally based on these output states, and music element data such as beat timing and dynamics, that is, the music reproduced by the MIDI reproduction unit 59. And outputs the music element data a for controlling to the MIDI reproducing unit 59.

Of the music element data a, the performance tempo data a1 is supplied to the beat occurrence probability predicting section 56,
The beat occurrence probability prediction unit 56 uses the performance tempo data a.
Based on 1, the beat occurrence probability is predicted by a fuzzy inference (specifically, a fuzzy inference called a direct method).

Here, the direct method is generally a method of performing fuzzy inference according to the following procedure.

1) Describe a fuzzy rule 2) In the fuzzy rule of 1), the language that becomes a variable is represented by a membership function 3) Obtain the inference result of each fuzzy rule with respect to the input 4) Finalize from the inference result of each rule The beat occurrence probability predicting unit 56 that obtains a realistic inference result specifically calculates the beat occurrence probability by applying the direct method as follows.

That is, for 1), if (the elapsed time from the previous beat timing based on the performance tempo is short) th
Adopt a fuzzy rule called en (probability of beat occurrence).

Regarding 2), the antecedent portion, that is, the portion of the fuzzy rule "the elapsed time based on the performance tempo from the previous beat timing is short" is shown in FIG. 4 (a).
The membership function shown in FIG. 4B is defined, and the membership function shown in FIG. 4B is defined for the consequent part, that is, for the part of the fuzzy rule where "the possibility of beat occurrence is low".

Regarding 3), the elapsed time based on the performance tempo from the previous beat is calculated based on the input performance tempo data a1 (for example, the time when the current tempo is converted into a predetermined reference tempo). Then, using the membership function of FIG. 4A, the grade for the calculated elapsed time is obtained.

When the antecedent part of the fuzzy rule consists of a plurality of elements connected by "and", the minimum value of the grade obtained from each membership function (this is called "fitness"). ) Is obtained (this calculation is called "" min "calculation").

However, the beat occurrence probability prediction unit 56
Since the antecedent part of the fuzzy rule is composed of one element, the grade obtained with respect to the elapsed time corresponds to the compatibility.

For example, when the elapsed time is t1, FIG.
As shown in (a), since the grade is determined to be g1, the compatibility is g1.

Then, the membership function of the consequent part, that is, the membership function of FIG. 4 (b), which is obtained by cutting off the portion above the above-mentioned goodness of fit, is the inference result of the fuzzy rule of 1). Become. For example, if the goodness of fit is g1, the inference result obtained is the portion indicated by the region R1 in FIG.

Regarding 4), when a plurality of types of fuzzy rules in 1) are defined, the inference result in 3) can be obtained by the number of each rule. Therefore, the inference result, that is, a plurality of regions such as the region R1 is obtained. Areas are overlapped (this operation is called "max" operation). The center of gravity of the regions thus superposed is obtained, the perpendicular line is extended from the center of gravity, and the value of the foot is the final inference result.
This is called the center of gravity method.

For example, in FIG. 4B, the center of gravity of the region R1 is obtained as G1, and the occurrence probability corresponding to this center of gravity G1 is obtained as p1. At this time, since the occurrence probability p1 is a numerical value used for control, it is simultaneously defuzzified (Defu
zzify) has also been made.

The beat occurrence probability predicting unit 56 outputs the occurrence probability thus obtained to the comprehensive judging unit 58 as the beat occurrence probability.

The beat occurrence probability distribution calculation unit 57 displays MI
The MIDI data supplied from the DI database (for example, this database is built on the hard disk in the HDD 13) 60 is supplied to the MIDI reproducing unit 59, and the beat occurrence probability distribution calculating unit 57 is as shown in FIG. The membership function is defined, and the beat occurrence probability distribution on the MIDI data is calculated and output by fuzzy inference using the supplied MIDI data and this membership function.

Specifically, as a fuzzy rule, if
(The part being played is close to the beats and notes of MIDI data.) Th
Adopting the rule of en (beat input is likely to be equivalent to that), the membership function of FIG. 5A is defined as the membership function. As shown in FIG. 5A, each membership function is defined so that the grade takes the maximum value at the beat timing of MIDI data. On the other hand, since the MIDI data does not include information indicating the beat position, it is considered that the beat is likely to occur at the beat timing, and the maximum grade is set at the beat timing based on the MIDI data. , Define each membership function. In addition, when it is desired to set the beat position at a position other than the beat timing, as shown in FIG.
The user may manually set the membership function.

When the fuzzy rule and each membership function are defined in this way, the subsequent fuzzy inference is the same as the fuzzy inference in the beat occurrence probability prediction unit 56 described above, and therefore its description is omitted. To do.

The beat occurrence probability distribution calculation unit 57 outputs the calculated beat occurrence probability to the comprehensive determination unit 58.

The overall determination unit 58 is the HMM recognition unit 54.
Likelihood of each model Mk from, and beat occurrence probability prediction unit 56
Based on the beat occurrence probability from the beat occurrence probability distribution, and the beat occurrence probability distribution from the beat occurrence probability distribution calculation unit 57, the command action is comprehensively determined by fuzzy inference.

Specifically, the fuzzy rule is if
(In the HMM recognition unit 54, the likelihood of the HMM corresponding to the nth Nth is high) and (In the beat occurrence probability prediction unit 56, there is a high probability of beat occurrence) and (Calculation of beat occurrence probability distribution) Part 57 adopts the rule that the place being played is near the nth beat of the N beat) then (the beat input is likely to be the Nth beat of the Nth beat), and as a membership function thereof, The membership functions shown in FIGS. 6A to 6D (the illustrated example shows the case where N = 3 and n = 2) are adopted.

The subsequent fuzzy inference method is the same as the method of the beat occurrence probability predicting section 56 described above, and therefore its explanation is omitted.

The performance tempo can be calculated based on the intervals between the beat timings.

In this way, the comprehensive judgment unit 58 performs fuzzy inference based on the input information to further improve the recognition rate of the commanding motion recognized by the HMM recognition unit 54, and the judgment result, that is, The music element data a
MIDI playback unit 5 (performance tempo, beat timing, beat type, dynamics, articulation, etc.)
Output to 9.

The MIDI reproducing unit 59 determines the timing at which a note-on / off event occurs based on the MIDI data supplied from the MIDI database 60, and the note-on / off occurs at the determined timing. Information to be transmitted to the sound source unit 61,
That is, the MIDI data is reproduced, and the MI being reproduced is reproduced according to the music element data a from the comprehensive determination unit 58.
Change the corresponding music element of the DI data. Specifically, when the music element data is beat timing, the beat timing or note position of the MIDI data being reproduced is changed so as to match the timing and the beat type, and when the music element data is dynamics,
The dynamics (velocity or volume) of the MIDI data being played back is changed so as to match the dynamics.

When the music element data is the performance tempo, the tempo at which the MIDI data is reproduced is changed. Also, when the music element data is articulation, the gate time length of the MIDI data being reproduced is changed.

The MIDI reproducing unit 59 uses the changed MI.
The DI data is output to the tone generator 61, and in response to this, the tone generator 61 generates a tone signal from the MIDI data and outputs it to the sound system 19.

The sound system 19 is, as described above,
The generated musical tone signal is converted into sound.

As described above, in this embodiment, the command motion recognized by the HMM recognition unit 54 is processed by the beat occurrence probability and beat occurrence probability distribution calculation unit 57 predicted by the beat occurrence probability prediction unit 56. Based on the calculated beat occurrence probability distribution, fuzzy inference is used to make a comprehensive determination to determine the final commanding motion, so that the recognition rate of the commanding motion can be further improved.

In this embodiment, the comprehensive judgment unit 58
Uses the both of the beat occurrence probability predicted by the beat occurrence probability prediction unit 56 and the beat occurrence probability distribution calculated by the beat occurrence probability distribution calculation unit 57 so as to comprehensively determine the command action. However, the determination may be made using only one of them. That is, there is at least one piece of information for improving the recognition rate of the commanding motion recognized by the HMM recognizing unit 54, and it is sufficient if the fuzzy inference can be performed using the information.

Further, it goes without saying that the fuzzy rules and membership functions described in the above embodiments are merely examples, and the present invention is not limited to these. For example, fuzzy rules for recognition of dynamics and articulation may be defined, and fuzzy inference may be applied to these recognitions.

In this embodiment, the command operation is performed by the HM.
The recognition is made by M, but the recognition is not limited to this.

Further, in the present embodiment, the baton 30 is provided with the acceleration sensors 31 and 32 and the commanding motion is detected according to the acceleration obtained from this, but the invention is not limited to this, and the angular velocity sensor ( A command operation may be detected according to a signal detected using a gyro sensor), a sensor using magnetism or light, or the sensor may be directly attached to a hand or a finger.

Further, in the present embodiment, the music control device is mainly composed of a general-purpose computer, but the present invention is not limited to this, and it may be composed of a dedicated device.

A storage medium in which a software program for realizing the functions of the above-described embodiments is recorded is
Needless to say, the object of the present invention can be achieved by supplying the system or device to the computer (or the CPU 7 or MPU) of the system or device to read and execute the program stored in the storage medium.

In this case, the program itself read from the storage medium realizes the novel function of the present invention,
The storage medium storing the program constitutes the present invention.

As a storage medium for supplying the program, for example, a hard disk of the HDD 13 or C
D-ROM 41, MO, MD, floppy disk 4
0, CD-R (CD-Recordable), magnetic tape, non-volatile memory card, ROM, etc. can be used.
In addition, another MIDI device 100 or communication network 10
Alternatively, the program may be supplied from the server computer 102 via 1.

Further, by executing the program read by the computer, not only the functions of the respective embodiments described above are realized, but also the OS and the like running on the computer are actually executed based on the instructions of the program. It goes without saying that a case where a part or all of the processing of (1) is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program read from the storage medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function is executed based on the instruction of the program. It goes without saying that the case where the CPU 7 or the like included in the expansion board or the function expansion unit performs some or all of the actual processing and the processing realizes the functions of the above-described embodiments is also included.

[0086]

As described above, according to the present invention,
Based on music elements extracted from automatically played music and recognized human motion candidates , by fuzzy reasoning,
Since the recognized motion candidates of the person are determined, the recognition rate is improved even when there are many recognition targets, and thus the expressiveness of music can be further improved.

[0087] Further, according to the present invention, based on the play Te <br/> down port of the automatic performance, it beats punching motion candidates by operation recognition means sure
Occurrence possibility of beats hitting the identified point in time is predicted, to the predicted beat striking occurrence likelihood and the motion recognizing means
Since the person's motion is determined by fuzzy inference based on the recognition result by the recognition result , the recognition rate can be further improved.

Further, preferably, the beat timing or the position of the note is extracted as the predetermined music element, and the operation recognition is performed based on the beat timing or the position of the note.
Occurrence possibility distribution of beats hitting at the time of the beat striking motion candidates is recognized by identification means is calculated, based on the recognition result by the computed beat striking occurrence possibility distribution and the motion recognizing means, the operation of the people fuzzy reasoning Since it is determined by, the recognition rate can be further improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a music control device according to an embodiment of the present invention.

FIG. 2 is a block diagram for explaining control processing mainly executed by a computer in the music control device in FIG.

FIG. 3 is a diagram for explaining in detail an HMM recognition process performed by the vector quantization unit in FIG.

FIG. 4 is a diagram showing an example of a membership function used when the beat occurrence probability prediction unit of FIG. 2 performs fuzzy inference.

5 is a diagram showing an example of a membership function used when the beat occurrence probability distribution calculation unit of FIG. 2 performs fuzzy inference.

FIG. 6 is a diagram showing an example of a membership function used when the comprehensive judgment unit of FIG. 2 performs fuzzy inference.

[Explanation of symbols]

7 CPU (motion recognition means, automatic performance means, music element extraction means) 31 Acceleration sensor (motion recognition means) 32 Acceleration sensor (motion recognition means) 53 Vector quantization section (motion recognition means) 54 HMM recognition section (motion recognition means) ) 55 HMM database (motion recognition means) 56 beat occurrence probability prediction section (music element extraction means) 57 beat occurrence probability distribution calculation section (music element extraction means) 58 comprehensive determination section (fuzzy determination means) 59 MIDI playback section ( Automatic playing means) 60 MIDI database (Automatic playing means)

Claims (4)

(57) [Claims] 1. A motion recognition means for recognizing a motion candidate of a person from a signal generated based on a motion of a person, an automatic performance means for automatically playing music, and a predetermined music element from the music to be played. Music control comprising: music element extraction means for extracting; and fuzzy judgment means for judging the motion of the person by fuzzy inference based on the extracted music elements and the recognition result by the motion recognition means. apparatus. 2. A beat recognizing means for recognizing a beat motion candidate in a human conducting motion from a signal generated based on a human motion, and the beat motion candidate being recognized based on a performance tempo of an automatic performance. A fuzzy determination means including a beat occurrence possibility prediction means for predicting a beat occurrence probability at a time point, based on a recognition result by the motion recognition means and the predicted beat occurrence probability, by fuzzy inference,
A music control device comprising: a person who determines a beat motion of the person. 3. The motion candidate of the person recognized by the motion recognition means is a beat motion candidate in a conducting motion, and the predetermined music element is a beat timing or a note position of the music to be played, The fuzzy determination unit includes a beat occurrence probability calculation unit that calculates a beat occurrence probability distribution at a time point when the beat operation candidate is recognized, based on the extracted beat timing or the position of the note. The music control device according to claim 1, wherein the beat motion of the person is determined based on the recognition result by the recognition means and the calculated beat occurrence probability distribution. 4. A motion recognition module for recognizing a human motion candidate from a signal generated based on a human motion, an automatic performance module for automatically playing music, and a predetermined music element from the played music. A program including a music element extraction module for extracting, and a fuzzy judgment module for judging the motion of the person by fuzzy inference based on the extracted music element and the recognition result by the motion recognition means.
A computer-readable storage medium that stores a program that can be realized by a computer .