JP7621139B2 - Performance control device, performance control method, and performance control program - Google Patents

Description

The present invention relates to a performance control device, a performance control method, and a performance control program for causing a robot to play a musical instrument.

Patent document 1 discloses a performance control device for making a robot play a musical instrument (see patent document 1). The performance control device is made up of a computer with a performance control program installed.

The performance control device executes a performance control program to sequentially send a large number of motion (posture) commands to the robot according to the music. The robot sequentially moves its hands and fingers based on the large number of motion (posture) commands sent from the performance control device. Specifically, the robot moves its hands to a predetermined position on the musical instrument (piano keys) and presses the instrument (piano keys) with its fingers. This causes the robot to play the musical instrument.

Japanese Patent Application Publication No. 8-1562

However, with conventional performance control devices, the user must pre-register the robot's motions (postures) that correspond to each musical scale of the instrument, and then create numerous motion (posture) commands that correspond to the music.

For this reason, for example, to make a robot with three fingers play an instrument with a two-octave (24-note) scale, 72 different motions (postures) must be registered, posing the problem that managing all of the postures is tedious. Another problem is that only users who are familiar with robot operation inputs can create motion (posture) commands.

The present invention was made in consideration of the above problems, and aims to provide a performance control device that can make a robot play a musical instrument through simple user input operations.

In order to solve the above-mentioned problems, one aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit , wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger among a plurality of fingers of the robot to press the musical instrument .
Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes a pre-processing time from when at least one finger of the robot starts to press the musical instrument to when the at least one finger of the robot actually presses the musical instrument, before the timing at which the at least one finger of the robot presses the musical instrument.
Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the performance motion includes the timing at which the robot's hands move.
Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for moving the robot based on information about the performance motion edited by the editing unit, wherein the performance motion includes a choreography motion for moving a part of the robot that is unrelated to the performance.
Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the display control unit displays on the display device the scale of the finger currently being pressed based on the information about the performance motion edited by the editing unit.

Another aspect of the present invention is a performance control method for causing a robot to play a musical instrument, comprising: a display step of displaying on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing step of editing the performance motion in response to a user's operation input of the graphic; and a job generation step of generating a job program for operating the robot based on information of the performance motion edited by the editing step , wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger among a plurality of fingers of the robot to press the musical instrument .

Yet another aspect of the present invention is a performance control program for causing a robot to play a musical instrument, which has a computer realize a display function for displaying on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument, an editing function for editing the performance motion in response to a user's operation input of the graphic, and a job generation function for generating a job program for operating the robot based on information on the performance motion edited by the editing function , wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger among a plurality of fingers of the robot that presses the musical instrument .

According to the present invention, a performance motion is created using a user interface that focuses on the robot's hands, making it easy to understand the position of the robot's hands and the timing of the hand manipulation of the instrument, and allowing the user to make the robot play an instrument through simple input operations.

1 is a perspective view of a robot controlled by the performance control device of the present embodiment. FIG. 2 is a detailed view of the hand of the robot. 1 is a block diagram showing an example of the hardware configuration of a performance control device according to an embodiment of the present invention; 2 is a block diagram showing an example of the functional configuration of the performance control device of the present embodiment. FIG. FIG. 13 is a diagram showing an example of a graphic representation of a performance motion. FIG. 13 is a diagram showing an example of a graphic representation of a performance motion. FIG. 7 shows the correspondence between a graphic representing a performance motion and the movements of the robot's fingers (FIG. 7(a) shows a graphic representing a performance motion, and FIGS. 7(b1) to (b3) show the movements of the robot's fingers). FIG. 8A shows the correspondence between a graphic representing a performance motion and the movements of the robot's fingers (FIG. 8A shows a graphic representing a performance motion, and FIGS. 8B1 and 8B2 show the movements of the robot's fingers). FIG. 9 shows the correspondence between figures representing performance motions and robot movements (FIGS. 9(a) and (b) show figures representing performance motions, and FIGS. 9(c1) and (c2) show the movements of the robot's fingers). FIG. 10A shows an example of a graphic representing a musical score, and FIG. 10B shows an example of a graphic representing a performance motion. FIG. 13 is a diagram showing an example of an initial setting screen. FIG. 2 is a diagram showing a moving range of a hand of a robot. FIG. 13(a) shows an example in which the robot's left hand holds a fingerboard, and FIG. 13(b) shows an example in which the robot's right hand holds a fingerboard. FIG. 13 is a diagram showing another example of the initial setting screen. FIG. 2 is a diagram showing a moving range of a hand of a robot.

Hereinafter, a performance control device, a performance control method, and a performance control program according to an embodiment of the present invention will be described with reference to the accompanying drawings. However, the present invention can be embodied in various forms and is not limited to the embodiments described in this specification. The present embodiment is provided with the intention of enabling those skilled in the art to fully understand the invention by fully disclosing the specification.
<Robot configuration>

Figure 1 is a perspective view of a robot 1 controlled by a performance control device 21 of this embodiment. The robot 1 is, for example, a humanoid robot, and includes a waist 2, a torso 4 connected to the waist 2 via a waist joint 3, two arms 6 (left and right as seen from the robot) connected to the torso 4 via shoulder joints 5, and a head 11 connected to the torso 4 via a neck joint 10. Each arm 6 includes an upper arm 7 and a lower arm 8 connected to the upper arm 7 via an elbow joint 9. A hand 14 is connected to the lower arm 8 via a wrist joint 12. The hand 14 has a palm 13 (see Figure 2) and fingers 15 connected to the palm 13 via finger joints 16. Each joint of the robot 1 is driven by a servo motor. Note that the robot is not limited to a humanoid robot, as long as it has hands 14 and can play a musical instrument.

Figure 2 is a detailed view of the hand 14 of the robot 1. A servo motor 18 that rotates a disk 17 is provided on the palm 13. Each disk 17 and each finger 15 are connected via a rod 19. By driving the servo motor 18, the finger 15 rotates around the knuckle 16, and the finger 15 presses down on the instrument 20 or separates the finger 15 from the instrument 20. 14b is a holder for the instrument. Note that, although multiple (three) finger 15 are configured to be operable in Figure 2, only one finger 15 may be operable. The hand 14 may be configured appropriately to suit the instrument.

As shown in FIG. 1, the musical instrument 20 has a fingerboard 20a. The musical instrument 20 is, for example, an electronic musical instrument. With this musical instrument 20, sound is produced simply by pressing the fingerboard 20a with the fingers 15 of the hand 14 of the robot 1, and the scale changes by changing the position at which the fingers 15 press the fingerboard 20a. Note that the musical instrument is not limited to an electronic musical instrument, and may be a keyboard instrument such as a piano, a wind instrument such as a recorder, a string instrument such as a guitar, or a percussion instrument such as a drum.

The robot 1 is controlled by a performance control device 21. The performance control device 21 controls the servo motors 18 of each joint of the robot 1 so that the robot 1 plays the musical instrument in accordance with music in the same way as a human being. That is, the performance control device 21 controls the servo motors 18 of each joint of the robot 1 so that the hand 14 of the robot 1 moves along the musical instrument 20 (in this embodiment, the hand 14 moves along the fingerboard 20a) and operates the musical instrument 20 (in this embodiment, the finger 15 presses the fingerboard 20a). Note that the operation of the musical instrument 20 by the hand 14 of the robot 1 is not limited to the above embodiment. For example, in the case of a keyboard instrument, the finger 15 of the robot 1 may press the keyboard, in the case of a wind instrument, the finger 15 of the robot 1 may cover the tone hole, in the case of a string instrument, the hand 14 of the robot 1 may pluck the strings directly or using a bow, etc., and in the case of a percussion instrument, the hand 14 of the robot 1 may strike the percussion instrument directly or using a drumstick, etc.
<Hardware configuration of the performance control device>

Figure 3 is a block diagram showing an example of the hardware configuration of the performance control device 21 of this embodiment. The hardware of the performance control device 21 is, for example, a general-purpose information processing device such as a personal computer.

As shown in FIG. 3, the performance control device 21 includes a CPU 22, a ROM 23, a RAM 24, a display device 25, a keyboard 26, a mouse 27, a storage device 28, a communication interface 29, a VRAM 30, a key scan circuit 31, a mouse scan circuit 32, a sound source unit 33, a sound system 34, and a bus 35. The display device 25 is connected to the bus 35 via the VRAM 30. The keyboard 26 is connected to the bus 35 via the key scan circuit 31. The mouse 27 is connected to the bus 35 via the mouse scan circuit 32. The mouse 27 may be a tablet, a touch screen, a touch panel, etc.

The display device 25 is a liquid crystal display, an organic electroluminescence display, or the like. Graphics representing performance motions, which will be described later, are displayed on the display device 25. A user can move a cursor displayed on the display device 25 and give instructions by operating the mouse 27. The mouse scan circuit 32 detects the movement and operation of the mouse 27, and outputs a detection signal to the CPU 22 via the bus 35. The keyboard 26 has keys for inputting letters, numbers, symbols, and the like. The key scan circuit 31 detects whether the keyboard 26 is on or off, and outputs a detection signal to the CPU 22.

The storage device 28 is, for example, a hard disk, a CD-ROM, a DVD, or a memory card. The storage device 28 stores a performance control program (described later), drawing information of figures representing performance motions (described later), information on performance motions edited by an editing unit (described later), and information on the posture of the robot 1 (described later).

The CPU 22 expands the performance control program stored in the storage device 28 into the RAM 24, and executes the performance control program expanded into the RAM 24, thereby controlling the entire performance control device 21. The RAM 24 temporarily stores information used in the processing of the CPU 22.

The communication interface 29 is an interface that is connected to a communication network such as a USB, a LAN, or the Internet. The robot 1 is connected to the communication interface 29 via a wired or wireless connection. The communication interface 29 transmits signals to the robot 1 and receives signals from the robot 1.

The sound source unit 33 includes software as a sound source and a DSP for adding effects. The sound source unit 33 generates a sound signal based on information about a performance motion edited by an editing unit (to be described later), and outputs the sound signal to a sound system 34. The sound system 34 has a D/A converter, an amplifier, and a speaker, and produces a sound for simulation based on the sound signal.
<Functional configuration of the performance control device>

Figure 4 is a block diagram showing an example of the functional configuration of the performance control device 21 of this embodiment. The performance control device 21 includes a display control unit 41, an editing unit 42, a job generation unit 43, and a sound control unit 44. These functional configurations are realized by the CPU 22 executing a performance control program.

The display control unit 41 displays on the display device 25 a graphic representing a performance motion including the position of the hands 14 of the robot 1 and the timing at which the hands 14 of the robot 1 operate the musical instrument 20. The editing unit 42 edits information about the performance motion in response to a user's input of an operation on the graphic. The job generation unit 43 generates a job program for operating the robot 1 based on the information about the performance motion edited by the editing unit 42. The job program generated by the job generation unit 43 is transmitted to the robot 1 via the communication interface 29. The sound control unit 44 controls the sound source unit 33 and the sound system 34 to play sounds for simulation based on the information about the performance motion edited by the editing unit 42.
<Specific processing of the performance control device>

The specific processing and user input operations in the performance control device 21 configured as described above will be described with reference to Figures 5 to 9.

First, the display control unit 41 of the performance control device 21 displays figures representing performance motions on the display device 25. Figure 5 shows examples of figures 50a to 54 representing performance motions displayed on the display device 25. As shown in Figure 6, the figures 50a to 54 representing performance motions are displayed in the performance motion display area 101 of the display device 25.

The figure 50a in FIG. 5 represents the fingerboard 20a of the musical instrument 20. The three circles 51a to 51c represent the three finger portions 15 of the robot 1. The vertical position of the top circle 51a, which represents the three circles 51a to 51c, represents the position of the hand 14 of the robot 1 on the fingerboard 20a. ● 51c represents the finger portion 15 (the finger portion 15 that presses the musical instrument 20) selected by the user's operation input among the three finger portions 15. ● 51c represents the timing at which the finger portion 15 of the robot 1 presses the musical instrument 20. ● The diagonal line band 52 extending to the right from 51c represents the duration of the state in which the finger portion 15 of the robot 1 presses the musical instrument 20. ● The diagonal line band 53 extending to the left from 51c represents the pre-processing time from when the finger portion 15 of the robot 1 starts the action of pressing the musical instrument 20 to when it actually presses the musical instrument 20. The dotted rectangular zone 54 indicates the timing for moving the hand 14 of the robot 1 (moving the arm 6 of the robot 1). The horizontal axis of FIG. 5 indicates time, and the vertical axis of FIG. 5 indicates the position of the hand 14 on the fingerboard 20a. The figure 55 is a timing indicator that matches the speed of the music.

The editing unit 42 then edits the performance motion according to the user's input of the operation of the figures 50a, 51-54. For example, when the user clicks one of the circles 51a, 51b, 51c with the mouse 27, the editing unit 42 selects the bottommost finger 15 of the robot 1, represented by the circle 51c, as the finger 15 that will press the instrument 20. The display control unit 41 changes the circle 51c selected by the user to a ● 51c.

For example, when the user drags and drops the figure 50a and ●51c with the mouse 27, the editing unit 42 determines the timing at which the finger unit 15 of the robot 1, represented by ●51c, presses the instrument 20. The display control unit 41 changes the left-right positions of the figures 50a and ●51c and displays them.

Furthermore, for example, when the user drags and drops the right end of band 52 with mouse 27, editing unit 42 determines the duration of the state in which band 52 indicates that finger unit 15 of robot 1 is pressing instrument 20. Display control unit 41 changes the length of band 52 and displays it.

Furthermore, for example, when the user drags and drops the left end of band 53 with mouse 27, editing unit 42 determines the pre-processing time represented by the length of band 53, from when finger unit 15 of robot 1 starts the action of pressing instrument 20 to when instrument 20 is actually pressed. Display control unit 41 changes the length of band 53 and displays it.

Furthermore, for example, when the user drags and drops the rectangular zone 54 with the mouse 27, the editing unit 42 determines the timing at which the hand 14 of the robot 1 represented by the rectangular zone 54 moves. The display control unit 41 changes and displays the rectangular zone 54.

Figure 7 shows the correspondence between ●51b, band 52, and band 53, which are figures representing playing motions, and the movements of the fingers 15 of the robot 1. Band 53 in Figure 7(a) represents the pre-processing time (movement time of the fingers 15) from when the fingers 15 of the robot 1 start to press the instrument 20 until they actually press the instrument 20, as shown in Figures 7(b1) and (b2). By adjusting the length of band 53, it is possible to reliably produce sound at the point of ●51b. Also, by adjusting the length of band 53, the speed of the fingers 15 can be adjusted (for example, if the length of band 53 is short, the fingers 15 will move quickly), allowing the dynamics of the sound to be adjusted in keyboard instruments and percussion instruments.

●51b in Fig. 7(a) represents the timing at which the fingers 15 of the robot 1 press the instrument 20, as shown in Fig. 7(b2). Band 52 in Fig. 7(a) represents the duration for which the fingers 15 of the robot 1 press the instrument 20. Area 102 in Fig. 7(a) (a section without band 52) represents the time that the fingers 15 are off the instrument 20, as shown in Fig. 7(b3). The user can adjust the position of ●51b and the length of band 52 through operational input to adjust the timing at which the sound is produced and the duration of the sound.

The vertical position of ●51b represents the position of the finger portion 15 on the fingerboard 20a of the instrument 20, so the user can adjust the scale by adjusting the vertical position of ●51b through operational input.

Figure 8 shows the correspondence between a rectangular zone 54, which is a figure representing a performance motion, and the movement of the fingers 15 of the robot 1. The rectangular zone 54 in Figure 8(a) represents the timing for moving the hand 14 of the robot 1. If the width of the rectangular zone 54 is wide, the hand 14 of the robot 1 moves slowly, and if the width of the rectangular zone 54 is narrow, the hand 14 of the robot 1 moves quickly.

As shown in FIG. 8(a), when the band 52 and the rectangular zone 54 overlap, the robot 1's hand 14 can be moved while the robot 1's finger 15 presses the instrument 20, as shown in FIG. 8(b1)(b2). This enables slide performance, which smoothly changes the scale while producing sound. As shown in area 103 in FIG. 8(a), by setting the length of the band 53 to the left of ●51b to zero and moving the band 52 to the right of the previous ●51b closer to ●51b, the finger 15 represented by ●51b can continue to press the instrument 20.

Figure 9 shows the correspondence between the figures ●51b, ●51c, band 52, and band 53 that represent playing motions, and the movements of the robot 1. As shown in Figure 9(a), if adjacent ●51b are the same finger 15, band 52 and band 53 cannot be positioned so that they overlap. This is because the same finger 15 cannot simultaneously press the instrument 20 and be separated from the instrument 20.

As shown in Figure 9 (b), if adjacent ●51b and ●51c are different finger parts 15, band 52 and band 53 can be arranged to overlap. In this case, as shown in Figure 9 (c1), while finger part 15 represented by ●51b (finger 2) is pressing down, finger part 15 represented by ●51c (finger 3) prepares to press down, and as shown in Figure 9 (c2), finger 3 presses instrument 20 and finger 2 simultaneously leaves the instrument. In this way, by arranging band 52 and band 53 so that they overlap, quick switching of finger parts 15 is possible.

Next, the job generating unit 43 generates a job program for operating the robot 1 based on the information of the performance motion edited by the editing unit 42. The performance motion includes the position of the hand 14 of the robot 1 and the timing at which the finger 15 of the robot 1 presses the musical instrument 20, and so on, and therefore it is possible to make the robot 1 play the musical instrument 20.
<Displaying musical scales and generating sounds for simulation>

As shown in area 104 of FIG. 6, the display control unit 41 displays the note of the finger 15 being pressed by the robot 1, for example C4, on the display device 25 based on the information of the playing motion edited by the editing unit 42 (the vertical position of the hand 14, the distance between the fingers, which will be described later, and the relationship between the position of the hand 14 and the note), and also displays the position of the hand 14 on the fingerboard 20a, for example 0.110, on the display device 25.

The sound control unit 44 controls the sound source unit 33 and the sound system 34 to generate sounds for the simulation based on the information of the performance motion edited by the editing unit 42. This allows the edited performance motion to be played back and checked in the simulation without moving the actual robot 1. When the playback process is started, the display control unit 41 moves the figure 55 representing the timing indicator to the right.
<Choreography Motion>

The performance motion includes choreography motion that moves parts of the robot 1 (head 11, torso 4, etc.) that are not related to the performance. As shown in FIG. 6, the display control unit 41 displays figures 56a and 56b representing the choreography motion on the display device 25. The figures 56a and 56b representing the choreography motion are displayed in the choreography motion display area 105.

For example, the figure 56a represents the action of rotating the torso 4 of the robot 1 to the left, and the figure 56b represents the action of rotating the torso 4 of the robot 1 to the right. The choreography motion is registered in advance, i.e., stored in the storage device 28.

The editing unit 42 edits the choreography motion in response to user input, for example dragging and dropping figures 56a, 56b with the mouse 27. After editing the performance motion, the editing unit 42 adds and edits the choreography motion. Note that the movement choreography motion of the arm 6 of the robot 1 is overwritten by the performance motion of the arm 6 of the robot 1. The editing unit 42 can match the performance motion even if the choreography motion is time-series data of the joint angles of the robot 1, etc.

As a choreography motion, the hand 14 on the opposite side to the hand 14 holding the fingerboard 20a can also be moved. In this case, the editing unit 42 creates a choreography motion in which the fingers 15 of the hand 14 on the opposite side close when the figure 50 appears, in response to a user operation input, for example, when the user uses the mouse 27 to click on the figure 50 representing the holding part 20b (see FIG. 1) of the instrument 20.

The job generating unit 43 operates the robot 1 based on the information of the choreography motion edited by the editing unit 42. In this way, the robot 1 can be given a choreography motion that matches the music performance.
<User interface showing musical score>

As shown in FIG. 10(a), the display control unit 41 may display a graphic 61 representing a musical score with notes arranged on a five-line staff on the display device 25. The editing unit 42 edits the musical score through operational input by the user. Since the graphic 61 representing the musical score corresponds to a general musical score, the user can create music more easily.

The display control unit 41 converts the figure 61 representing the score shown in Fig. 10(a) into a figure representing a performance motion shown in Fig. 10(b) based on the score information edited by the editing unit 42. Fine adjustment of the position of the hand 14, the timing of moving the hand 14, and the selection of the finger 15 pressing the instrument 20 are performed using the figure representing the performance motion shown in Fig. 10(b).
<Initial settings>

Figure 11 shows an example of an initial setting screen displayed by the display control unit 41 on the display device 25. A dialogue 62 for setting the reference posture of the robot 1 is displayed on the display device 25. The user inputs a posture of the robot 1 registered in advance into the dialogue 62 as the reference posture. The editing unit 42 takes into account the set reference posture of the robot 1 when creating a performance motion. This is the only reference posture of the robot 1 that the user sets, and there is no need to set multiple postures according to music.

The display device 25 displays a dialogue 63 for setting the length of the fingerboard 20a. As shown in FIG. 12, the length of the fingerboard 20a represents the range of movement of the hand 14 of the robot 1 (the range of movement of the arm 6). The user inputs the range of movement of the hand 14 in this dialogue 63. When creating a performance motion that moves the hand 14 of the robot 1, the editing unit 42 creates a performance motion that moves the hand 14 holding the fingerboard 20a of the robot 1 parallel to the back of the hand. The editing unit 42 may also create a performance motion that moves the hand 14 holding the fingerboard 20a closer to the hand 14 that holds the instrument 20.

The display device 25 displays dialogues 66a and 66b for setting the hand 14 holding the fingerboard 20a of the musical instrument 20 and the hand 14 fixing the musical instrument 20. When the user selects the dialogue 66a, the display control unit 41 displays a choice of the hand 14 holding the fingerboard 20a (hand 14a or hand 14b (see Figs. 13(a) and 13(b))) in the dialogue 66a. When the user selects the dialogue 66b, the display control unit 41 displays a choice of the hand 14 fixing the fingerboard 20a (hand 14a or hand 14b) in the dialogue 66a. In response to the user's operation input, the editing unit 42 creates a performance motion so that the left hand 14a of the robot 1 holds the fingerboard 20a and the right hand 14b fixes the musical instrument 20, as shown in Fig. 13(a), or creates a performance motion so that the robot 1 moves in the opposite direction, as shown in Fig. 13(b). This allows you to set the hand 14 of the robot 1 that plays the instrument 20 and the hand 14 that is fixed. Even after creating a performance motion, you can have the robot 1 play the instrument 20 with the hand 14 on the opposite side by switching to the opposite hand 14 in the dialogues 66a and 66b and then switching the reference posture.

As shown in FIG. 11, the display device 25 displays a dialogue 67 for setting the distance between the fingers 15 of the robot 1. The display device 25 also displays a dialogue 68 for setting the relationship between the position of the hand 14 (hand position) and the musical scale (the note name of the top finger). C3, D3, E3, etc. in the dialogue 68 represent musical scales such as Do, Re, Mi, etc. Based on this setting, the display control unit 41 displays the musical scale of the finger 15 that is being pressed on the display device 25.

As shown in Fig. 14, instead of setting the length of the fingerboard 20a on the initial setting screen, a reference posture 1 (reference symbol 64) and a reference posture 2 (reference symbol 65) of the robot 1 may be set. As shown in Fig. 15, the reference posture 1 represents the upper end position of the hand 14 of the robot 1, and the reference posture 2 represents the lower end position of the hand 14 of the robot 1. In this case, the editing unit 42 creates a performance motion that moves the hand 14 by apportioning between the reference posture 1 and the reference posture 2.
<Effects of the performance control device of this embodiment>

The above describes an example of the configuration, functions, and specific processing of the performance control device 21 of this embodiment. The performance control device 21 of this embodiment provides the following effects.

The performance motion is created using a user interface that focuses on the hands 14 of the robot 1, so the position of the hands 14 of the robot 1 and the timing of the hand 14 operating the instrument are easy to understand, and the user can make the robot 1 play the instrument 20 with simple input operations.

Since there is no need to register multiple postures of the robot 1, it is easy to manage the postures of the robot 1.

The timing at which the robot's 1 fingers 15 press the instrument 20 can be set.

In a robot 1 with multiple fingers 15, it is possible to select which finger 15 to use to press the fingerboard 20a, and quick performance is possible by switching between the fingers 15.

The figure representing the performance motion includes a band 52, so the duration that the fingers 15 of the robot 1 press the instrument 20 can be set.

The figure representing the performance motion includes a band 53, so the pre-processing time can be set, making it possible to play sounds at optimal timing in accordance with the musical score.

The figure representing the performance motion includes a rectangular zone 54, making it possible to perform slide performances and other performances that take advantage of the characteristics of the instrument 20 that has a fingerboard 20a.

It is also possible to match choreography motions to performance motions.

The display device 25 displays the musical scale of the finger 15 of the robot 1 while it is being pressed, so it is possible to check the performance without actually moving the robot 1.

The sound control unit 44 produces simulation sounds, making it possible to check the performance without actually moving the robot 1.

1...Robot, 14...Hand, 15...Finger, 20...Musical instrument, 21...Performance control device, 34...Sound system, 41...Display control unit, 42...Editing unit, 43...Job generation unit, 44...Sound control unit, 50a-54...Figures representing performance motions

Claims (9)

A performance control device for causing a robot to play a musical instrument, comprising:
a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing unit that edits the performance motion in response to a user's operation input of the figure;
a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit ,
the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument;
The performance control device , wherein the performance motion includes selection of at least one finger portion of the robot to press the musical instrument . 2. The performance control device according to claim 1 , wherein the performance motion includes a duration of time that at least one finger of the robot presses the musical instrument. A performance control device for causing a robot to play a musical instrument, comprising:
a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing unit that edits the performance motion in response to a user's operation input of the figure;
a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit,
the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument;
The performance control device, wherein the performance motion includes a pre-processing time from when at least one finger of the robot starts to press the instrument until when the at least one finger of the robot actually presses the instrument, before the timing when the at least one finger of the robot presses the instrument. A performance control device for causing a robot to play a musical instrument, comprising:
a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing unit that edits the performance motion in response to a user's operation input of the figure;
a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit,
The performance control device, wherein the performance motion includes a timing for moving the robot's hands. A performance control device for causing a robot to play a musical instrument, comprising:
a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing unit that edits the performance motion in response to a user's operation input of the figure;
a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit,
The performance control device, wherein the performance motion includes a choreography motion that moves a part of the robot that is unrelated to the performance. A performance control device for causing a robot to play a musical instrument, comprising:
a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing unit that edits the performance motion in response to a user's operation input of the figure;
a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit,
the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument;
The display control unit is a performance control device that displays the scale of the finger currently being pressed on the display device based on the information of the performance motion edited by the editing unit. The performance control device according to any one of claims 1 to 6, characterized in that the performance control device further comprises a sound control unit that controls a sound system to produce sounds for simulation based on the information of the performance motion edited by the editing unit. A performance control method for causing a robot to play a musical instrument, comprising the steps of:
a display step of displaying on a display device a graphic representing a playing motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument;
an editing step of editing the performance motion in response to an operation input of the figure by a user;
a job generating step of generating a job program for operating the robot based on the information of the performance motion edited by the editing step ,
the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument;
The performance control method , wherein the performance motion includes selecting at least one finger portion of the robot to press the musical instrument . A performance control program for causing a robot to play a musical instrument,
On the computer,
a display function for displaying on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate a musical instrument;
an editing function for editing the performance motion in response to a user's operation input of the figure;
a job generation function that generates a job program for operating the robot based on the information of the performance motion edited by the editing function ;
the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument;
The performance control program , in which the performance motion includes selection of at least one finger portion of the robot to press the musical instrument .

Images