CN112019910A - Musical instrument playing method and device, television and storage medium - Google Patents

Abstract

The embodiment of the invention discloses a musical instrument playing method and device, a television and a storage medium. The method comprises the following steps: acquiring a human body image of a user through a camera, and displaying the human body image and the virtual musical instrument on a screen to form a real-time picture; acquiring the moving position information of the hand of the user in the real-time picture; and playing the simulated sound of the virtual musical instrument according to the moving position information. According to the technical scheme of the embodiment of the invention, the instrument is played by the large-screen equipment with the camera, so that the ambience in the scene can be created, everyone can feel the fun of striking or playing in the simplest mode, and meanwhile, the cost for purchasing the instrument and the cost for going to a training institution for training can be saved.

Description

A musical instrument performance method, device, television and storage medium

technical field

Embodiments of the present invention relate to the technical field of smart TVs, and in particular, to a method, a device, a TV, and a storage medium for playing a musical instrument.

Background technique

Most people have dreamed of becoming a musician, but often due to many practical reasons, they cannot buy musical instruments such as drums or pianos. There are also most beginners who need to go to the corresponding training institutions or music practice if they want to practice musical instruments. For families with musical instruments, there are also problems, such as practicing drums or piano at home late at night or during breaks, because the volume cannot be controlled, which will cause varying degrees of disturbance to neighbors.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a musical instrument playing method, device, television, and storage medium, so as to save the cost of purchasing musical instruments and the cost of going to a training institution for training.

In a first aspect, an embodiment of the present invention provides a method for playing a musical instrument, comprising:

Obtain the user's human body image through the camera, and display the human body image and the virtual musical instrument on the screen to form a real-time image;

acquiring the mobile position information of the user's hand in the real-time screen;

The simulated sound of the virtual musical instrument is played according to the moving position information.

Optionally, before acquiring the moving position information of the user's hand in the real-time picture, the method further includes:

Get the practice repertoire entered by the user;

A moving guide prompt is displayed in the real-time screen according to the practice repertoire.

Optionally, after acquiring the moving position information of the user's hand in the real-time screen, the method further includes:

A prompt indicating whether the operation is correct or not is displayed according to the moving guide prompt and the moving position information of the user's hand.

Optionally, the method further includes:

After the user has practiced the practice piece, a score is generated according to the number of correct operations.

Optionally, the number of the users is two;

Obtaining the user's human body image through the camera, and displaying the human body image and the virtual musical instrument on the screen to form a real-time image, including:

Obtain the human body images of two users through the camera, and display the two human body images and the same virtual musical instrument on the left and right sides of the screen to form a real-time image;

After the user exercises the practice repertoire, a score is generated according to the number of correct operations, including:

After the two users have finished practicing the practice pieces, each generates a score to compare the scores.

Optionally, before the screen displays the human body image and the virtual musical instrument, the method further includes:

Get the practice instrument entered by the user on the candidate instrument;

The virtual musical instrument is generated from the practice musical instrument.

Optionally, the practice musical instrument is a drum set.

In a second aspect, an embodiment of the present invention also provides a musical instrument performance device, comprising:

a picture generation unit, used for obtaining the user's human body image through the camera, and displaying the human body image and the virtual musical instrument on the screen to form a real-time picture;

a hand acquisition unit, configured to acquire the movement position information of the user's hand in the real-time screen;

A sound playing unit, configured to play the simulated sound of the virtual musical instrument according to the moving position information.

In a third aspect, an embodiment of the present invention further provides a television, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the foregoing embodiments when the processor executes the computer program The musical instrument playing method described in any one of.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements any of the musical instrument performance methods described in the foregoing embodiments.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Description of drawings

1 is a schematic flowchart of a musical instrument performance method in Embodiment 1 of the present invention;

Fig. 2 is the schematic flow chart of a kind of musical instrument performance method in the second embodiment of the present invention;

Fig. 3 is the schematic flow chart of a kind of musical instrument performance method in the third embodiment of the present invention;

Fig. 4 is the structural representation of a kind of musical instrument performance device in the fourth embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a television in Embodiment 5 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowchart depicts the steps as a sequential process, many of the steps may be performed in parallel, concurrently, or concurrently. Furthermore, the order of the steps can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Furthermore, the terms "first," "second," etc. may be used herein to describe various directions, acts, steps or elements, etc., but are not limited by these terms. These terms are only used to distinguish a first direction, act, step or element from another direction, act, step or element. For example, a first speed difference may be referred to as a second speed difference, and similarly, a second speed difference may be referred to as a first speed difference, without departing from the scope of this application. Both the first speed difference and the second speed difference are speed differences, but they are not the same speed difference. The terms "first", "second", etc. should not be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as "first" or "second" may expressly or implicitly include one or more of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

Example 1

FIG. 1 is a schematic flowchart of a method for playing a musical instrument according to Embodiment 1 of the present invention, and the embodiment of the present invention is applicable to the situation of musical instrument playing. The method of the embodiment of the present invention may be performed by a musical instrument performance device, which may be implemented in software and/or hardware, and may generally be integrated into a TV with a camera or a large-screen device. 1, a musical instrument performance method according to an embodiment of the present invention specifically includes the following steps:

Step S110 , acquiring a human body image of the user through a camera, and displaying the human body image and the virtual musical instrument on the screen to form a real-time image.

Specifically, the method for playing a musical instrument in this embodiment of the present invention can be performed by an electronic device with a camera, for example, a TV with a camera, where the camera is located above the TV screen, and can capture images and display images on the screen at the same time. Of course, other electronic devices, such as mobile phones, tablet computers, laptop computers, personal digital assistants, mobile Internet devices, wearable devices, etc., may also be used, which are not limited in this embodiment of the present invention. In this embodiment, when the TV enters the function of playing musical instruments, the camera is turned on, and the user's body image and surrounding scenes are obtained through the camera and displayed on the screen. form a real-time picture. In this embodiment of the present invention, the virtual musical instrument may be a virtual drum kit, a virtual piano, a virtual guitar, etc. Of course, it may also be some other virtual musical instruments, which is not limited in this embodiment of the present invention.

As an optional embodiment, before the screen displays the human body image and the virtual musical instrument, the method further includes:

Get the practice instrument entered by the user on the candidate instrument;

The virtual musical instrument is generated from the practice musical instrument.

In this embodiment, the alternative musical instruments refer to a variety of musical instruments provided by the TV, which can be freely selected by the user to play. For example, the TV provides three musical instruments: drums, piano, and guitar, and the user can choose one of the three musical instruments. , such as a drum kit as a practice instrument. After the user selects a musical instrument from the candidate musical instruments as a practice musical instrument, the virtual musical instrument is generated according to the practice musical instrument, and the virtual musical instrument is displayed on the screen, floating on the content of the camera, to form a real-time picture.

Step S120: Acquire movement position information of the user's hand in the real-time screen.

Specifically, the movement position information of the user's hand may be the position information of the user's palm movement, or may also be the position information of the user's entire arm and the palm moving together, or may be the user's hand and the musical instrument device. Location information for device movement. For example, when the virtual musical instrument is a virtual drum kit, the movement position information is the position information of a drum hammer bound to the user's hand.

Step S130: Play the simulated sound of the virtual musical instrument according to the moving position information.

Specifically, in the real-time picture displayed on the screen, for each virtual musical instrument, each corresponding area corresponds to a different simulated sound, and the camera acquires the moving area of the user's hand, and then emits the corresponding simulated sound corresponding to the area. . For example, taking the virtual musical instrument as a drum kit as an example, after the camera recognizes the user's hand bones, drumsticks will appear and bind to the hands. Each drum is distributed on both sides and the bottom of the screen, and regional layout is performed in the background , each corresponding area corresponds to a different drum sound, when the hand moves to hit each area, different drum sounds will be emitted respectively, and the user can master the rhythm by himself. Of course, in the process of playing, the electronic device can also display some visual effects, such as beating notes, sound wave effects and so on. As an optional embodiment, the musical instrument performance method further includes: receiving volume adjustment information input by a user; and adjusting the volume of the simulated sound of the virtual musical instrument according to the volume adjustment information. Specifically, the volume adjustment information input by the user may be input through physical keys or voice adjustment, such as volume adjustment through a TV remote control.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Embodiment 2

FIG. 2 is a schematic flowchart of a method for playing a musical instrument according to Embodiment 2 of the present invention, and the embodiment of the present invention can be applied to a situation in which musical instruments are played. The method of the embodiment of the present invention may be performed by a musical instrument performance device, which may be implemented in software and/or hardware, and may generally be integrated into a TV with a camera or a large-screen device. 2, a musical instrument performance method according to an embodiment of the present invention specifically includes the following steps:

Step S210 , acquiring a human body image of the user through a camera, and displaying the human body image and the virtual musical instrument on the screen to form a real-time image.

Step S220, acquiring the practice piece input by the user.

Specifically, a plurality of practice songs can be pre-stored in the electronic device, and the user can select his favorite practice song according to his own preferences, and the user can input it through physical buttons or voice. Alternatively, the electronic device can be connected to a network, and the user can remotely download a favorite track for practice by searching. Users can follow the on-screen prompts, select the corresponding level and track, and start playing a song.

Step S230, displaying a moving guide prompt on the real-time screen according to the practice piece.

Specifically, the movement guide prompt is to guide the user how to operate the hand movement position in front of the camera to practice playing according to the practice piece and the virtual musical instrument. For example, when a user selects their favorite music, a simple drum prompt will appear on the top of the screen for guidance. Through the above convenient operations, they can complete the practice of a song independently.

Step S240: Obtain the movement position information of the user's hand in the real-time screen.

Step S250: Play the simulated sound of the virtual musical instrument according to the moving position information, and simultaneously display a prompt indicating whether the operation is correct or not according to the moving guide prompt and the moving position information of the user's hand.

Specifically, if the movement position of the user's hand matches the movement guide information of the practice piece, the music of the piece will be played as usual without any prompt. If the movement position of the user's hand does not match the movement guide information of the practice piece, you can The screen displays an error prompt, such as a red circle prompt, which can either play the analog sound of the wrong moving position as usual, or a warning sound, or no sound is played, and so on.

Step S260, after the user has practiced the practice piece, generate a score according to the number of times the operation is correct or not.

Specifically, after the performance, the system automatically generates a score according to the number of correct operations, and proposes a strike suggestion, and the user can choose to repeat the song or the next song.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Embodiment 3

FIG. 3 is a schematic flowchart of a method for playing a musical instrument according to Embodiment 3 of the present invention, and the embodiment of the present invention is applicable to the situation of musical instrument playing. The method of the embodiment of the present invention may be performed by a musical instrument performance device, which may be implemented in software and/or hardware, and may generally be integrated into a TV with a camera or a large-screen device. 3, a musical instrument performance method according to an embodiment of the present invention specifically includes the following steps:

Step S310 , acquiring the human body images of the two users through the camera, and displaying the two human body images and the same virtual musical instrument on the left and right sides of the screen to form a real-time image.

Specifically, in the embodiment of the present invention, the TV screen can be divided into left and right scenes, and each scene has a user playing the same musical instrument, playing the same song, and playing the game.

Step S320, acquiring the practice piece input by the user.

Step S330 , displaying a moving guide prompt in the real-time screen according to the practice piece.

Step S340: Obtain the movement position information of the hands of the two users in the real-time screen.

Step S350: Play the simulated sound of the virtual musical instrument according to the moving position information, and simultaneously display a prompt whether the operation is correct or not according to the moving guide prompt and the moving position information of the user's hand.

Step S360 , after the two users have finished practicing the practice pieces, each generate a score to compare the score.

Specifically, after the two users practice the practice repertoire, the system automatically generates scores according to the number of correct operations, and compares the scores of the two users.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Embodiment 4

A musical instrument performance device provided by the embodiment of the present invention can execute a musical instrument performance method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. The device can be composed of software and/or hardware (integrated circuit), and can generally be integrated into a TV with a camera or a large-screen device. FIG. 4 is a schematic structural diagram of a musical instrument performance device 400 in Embodiment 4 of the present invention. 4 , the musical instrument performance device 400 according to the embodiment of the present invention may specifically include:

The picture generation unit 410 is used for obtaining the human body image of the user through the camera, and displaying the human body image and the virtual musical instrument on the screen to form a real-time picture;

a hand obtaining unit 420, configured to obtain the moving position information of the user's hand in the real-time screen;

The sound playing unit 430 is configured to play the simulated sound of the virtual musical instrument according to the moving position information.

Optionally, the apparatus 400 further includes:

A repertoire input unit, used to obtain the practice repertoire input by the user;

A movement guide unit, configured to display a movement guide prompt in the real-time screen according to the practice piece.

Optionally, the apparatus 400 further includes:

The operation judging unit is configured to display a prompt indicating whether the operation is correct or not according to the moving guide prompt and the moving position information of the user's hand.

Optionally, the apparatus 400 further includes:

The score generating unit is configured to generate a score according to the number of times the operation is correct or not after the user has practiced the practice piece.

Optionally, the number of the users is two;

The picture generation unit 410 is further configured to acquire the human body images of the two users through the camera, and display the two human body images and the same virtual musical instrument on the left and right side of the screen to form a real-time picture;

The score generating unit is further configured to generate scores respectively to compare the scores after the two users have finished practicing the practice pieces.

Optionally, the picture generation unit 410 is further configured to acquire the practice musical instrument input by the user on the candidate musical instrument; and generate the virtual musical instrument according to the practice musical instrument.

Optionally, the practice musical instrument is a drum set.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Embodiment 5

FIG. 5 is a schematic structural diagram of a TV according to Embodiment 5 of the present invention. As shown in FIG. 5 , the TV includes a processor 510, a memory 520, an input device 530 and an output device 540; the number of processors 510 in the TV can be One or more, one processor 510 is taken as an example in FIG. 5; the processor 510, the memory 520, the input device 530 and the output device 540 in the TV can be connected through a bus or other means, and the connection through a bus is taken as an example in FIG. 5 .

As a computer-readable storage medium, the memory 520 can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the musical instrument performance method in the embodiment of the present invention (for example, the screen generation in the musical instrument performance device). unit 410, hand acquisition unit 420, and sound playback unit 430). The processor 510 executes various functional applications and data processing of the television by running the software programs, instructions and modules stored in the memory 520, ie, implements the above-mentioned musical instrument performance method.

That is:

Obtain the user's human body image through the camera, and display the human body image and the virtual musical instrument on the screen to form a real-time image;

acquiring the mobile position information of the user's hand in the real-time screen;

The simulated sound of the virtual musical instrument is played according to the moving position information.

Of course, in the television provided by the embodiment of the present invention, the processor of the processor is not limited to performing the above method operations, and can also perform related operations in the musical instrument performance method provided by any embodiment of the present invention.

The memory 520 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Additionally, memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, memory 520 may further include memory located remotely from processor 510, which may be connected to the television through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The input device 530 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the television. The output device 540 may include a display device such as a display screen.

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Embodiment 6

Embodiment 6 of the present invention also provides a storage medium containing computer-executable instructions, where the computer-executable instructions are used to perform a musical instrument performance method when executed by a computer processor, and the method includes:

Obtain the user's human body image through the camera, and display the human body image and the virtual musical instrument on the screen to form a real-time image;

acquiring the mobile position information of the user's hand in the real-time screen;

The simulated sound of the virtual musical instrument is played according to the moving position information.

Of course, a storage medium containing computer-executable instructions provided by an embodiment of the present invention, the computer-executable instructions of which are not limited to the above-mentioned method operations, and can also perform any of the musical instrument performance methods provided by any embodiment of the present invention. related operations.

The computer-readable storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .

Program code embodied on a storage medium may be transmitted using any suitable medium, including - but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

According to the technical solution of the embodiment of the present invention, playing a musical instrument through a large-screen device with a camera can not only create an immersive atmosphere, but also make everyone feel the fun of hitting or playing in the simplest way, and also It can save the cost of purchasing musical instruments and the cost of going to training institutions for training.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (10)

1. A musical instrument playing method, comprising:

acquiring a human body image of a user through a camera, and displaying the human body image and the virtual musical instrument on a screen to form a real-time picture;

acquiring the moving position information of the hand of the user in the real-time picture;

and playing the simulated sound of the virtual musical instrument according to the moving position information.

2. The musical instrument playing method according to claim 1, further comprising, before acquiring the movement position information of the hand of the user in the real-time screen:

acquiring an exercise song input by a user;

and displaying a movement guide prompt in the real-time picture according to the exercise track.

3. The musical instrument playing method according to claim 2, further comprising, after acquiring the movement position information of the hand of the user in the real-time screen:

and displaying a prompt of whether the operation is correct or not according to the movement guide prompt and the movement position information of the hand of the user.

4. A musical instrument playing method according to claim 3, characterized in that the method further comprises:

and after the user exercises the exercise tracks, generating scores according to the times of operating correctness or not.

5. The musical instrument playing method according to claim 4, wherein the number of the users is two;

the human body image of user is acquireed through the camera to at screen display human body image and virtual musical instrument form real-time picture, include:

acquiring human body images of two users through a camera, and displaying the two human body images and the same virtual musical instrument on the left and right of a screen in a split mode to form a real-time picture;

after the user has practiced the exercise track, generating a score according to the number of times of operating correctness or not, comprising:

after the two users finish practicing the exercise tracks, the scores are generated respectively.

6. The musical instrument playing method according to claim 1, further comprising, before the screen displaying the human body image and the virtual musical instrument:

acquiring an exercise instrument input by a user in the alternative instruments;

generating the virtual instrument from the practice instrument.

7. The musical instrument playing method according to claim 6, wherein the practice musical instrument is a drum kit.

8. An instrument playing apparatus, comprising:

the picture generating unit is used for acquiring a human body image of a user through a camera, and displaying the human body image and the virtual musical instrument on a screen to form a real-time picture;

a hand acquisition unit for acquiring movement position information of a user's hand in the real-time picture;

and the sound playing unit is used for playing the simulation sound of the virtual musical instrument according to the moving position information.

9. A television comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the musical instrument playing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a musical instrument playing method according to any one of claims 1 to 7.

Images