NZ791398B2

NZ791398B2 - Method and device for decomposing, recombining and playing audio data

Info

Publication number: NZ791398B2
Application number: NZ791398A
Authority: NZ
Inventors: Kariem Morsy; Christoph Teschner; Federico Tessmann
Original assignee: Algoriddim Gmbh
Filing date: 2020-03-17
Publication date: 2024-09-03

Abstract

Besides the creative aspects of disc jockey (DJ) work, it is a major task of a DJ to seamlessly blend between two songs. For this purpose, conventional DJ equipment provides features for changing tempo and key of a song such as to match tempo and key of the other song, respectively, and provides controls for cross-fading volumes or parameters of audio effects, for example equalizer effects, of the two songs. In order to achieve a smooth transition, it is an aim of the DJ to avoid clashing of the vocals of the two songs. Embodiments of the present disclosure may address this by providing a method for processing and playing audio data, comprising the steps of: receiving mixed input data, the mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track; decomposing the mixed input data to obtain at least a first decomposed track resembling the at least one first source track; generating output data based on the first decomposed track; wherein the mixed input data are first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track and wherein the method further comprises the steps of: receiving second mixed input data, the second mixed input data being a sum signal obtained from mixing at least one third source track with at least one fourth source track; decomposing the second mixed input data to obtain a third decomposed track resembling the at least one third source track, and a fourth decomposed track resembling the at least one fourth source track, wherein a control input represents a desired setting of a first volume level of the first decomposed track, a second volume level of a second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track; and wherein recombined output data is generated by recombining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level; and playing the recombined output data through an audio output.

Description

Besides the creative aspects of disc jockey (DJ) work, it is a major task of a DJ to seamlessly blend between two songs. For this purpose, conventional DJ equipment provides features for changing tempo and key of a song such as to match tempo and key of the other song, respectively, and provides controls for cross-fading volumes or parameters of audio effects, for example equalizer effects, of the two songs. In order to achieve a smooth transition, it is an aim of the DJ to avoid clashing of the vocals of the two songs. Embodiments of the present disclosure may address this by providing a method for processing and g audio data, sing the steps of: receiving mixed input data, the mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track; decomposing the mixed input data to obtain at least a first decomposed track resembling the at least one first source track; generating output data based on the first decomposed track; wherein the mixed input data are first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track and wherein the method further comprises the steps of: receiving second mixed input data, the second mixed input data being a sum signal obtained from mixing at least one third source track with at least one fourth source track; decomposing the second mixed input data to obtain a third decomposed track ling the at least one third source track, and a fourth osed track resembling the at least one fourth source track, wherein a l input ents a d setting of a first volume level of the first decomposed track, a second volume level of a second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track; and wherein recombined output data is generated by recombining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level; and playing the recombined output data through an audio output.

NZ 791398 [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Method and device for decomposing, recombining and playing audio ption The present ion relates to a method for processing and playing audio data comprising the steps of receiving mixed input data and playing recombined output data. Furthermore, the ion relates to a device for sing and playing audio data, preferably DJ equipment, sing an audio input unit for receiving a mixed input signal, a recombination unit and a playing unit for playing output data. In on, the present invention relates to a method and a device for representing audio data, i.e. on a display.

Audio equipment of the above-mentioned characteristics is used in various applications in which audio data is played, processed, transmitted or recorded. In particular, methods and devices of such type are used in the field of music entertainment and may be implemented in sound s for live entertainment and public s systems (PA systems). One example is DJ equipment which is conventionally d to receive at least two different input signals and comprises a recombination unit for mixing the two input signals at desired volume levels. DJ devices usually include a l element called crossfader for increasing the volume level of the first input signal while at the same time decreasing the volume level of the second input signal such as to continuously blend between the input signals.

In recent years, the creative and artistic contribution of disc jockeys during live shows has had a significant impact not only on the overall entertainment quality at a particular venue, but also on the development of modern music as such. Certain audio effects, timing and pitch ions developed by influential DJs have been used in modern arrangements of conventionally bduced recordings. Accordingly, there is a strong desire in the field to ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman explore new techniques for live modification of existing audio signals in order to further increase the creative freedom for the DJ during the show.

Besides the creative aspects of DJ work, it is a major task of a DJ to seamlessly blend between two songs. For this purpose, conventional DJ equipment provides features for changing tempo and key of a song such as to match tempo and key of the other song, respectively, and provides controls for fading volumes or parameters of audio s, for example equalizer effects, of the two songs. In order to achieve a smooth transition, it is an aim of the DJ to avoid clashing of the vocals of the two songs.

Therefore, the transition is y made during a time interval in which at least one of the two songs has a pause in the vocal track, for example during an instrumental solo part, an outro part or during a break between chorus and verse or at similar positions. r, this is a considerable restraint for the DJ and requires good knowledge or analysis of the song al during mixing. Furthermore, in many songs, especially in the genre of Pop or Hip Hop, often there is virtually no break in the vocal track that would be large enough for the DJ to allow a smooth transition to the other song. For such songs, a clashing of two vocals playing together or some interruption in the flow can mes not be avoided with the conventional techniques.

Similarly, in the case electronic music that sometimes does not contain any vocals at all, it is an aim of the DJ to avoid clashing of bass lines, synths, etc. of the two songs. The transition is then usually made during a time interval in which at least one of the two songs has a pause in one or more of the tonal/melodic tracks, e.g. toward the end/start of the song, or during a break or percussive parts of the song for example.

As regards the audio sources, in particular the input data available to the DJ, conventional methods and devices are usually confined to processing mixed input signals such as mixed stereo audio files obtained from online digital Hicic stores or streaming services. Audio files are usually ed in astudio by mixing a plurality of source tracks, for example a plurality of [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman vocal and mental tracks, applying audio effects and mastering the project to obtain a stereo audio file. As the ed audio file is a sum signal of all source tracks and effects etc., information about individual source tracks is usually lost and not available any more from a direct inspection of the audio file alone.

In recent years, there have been several approaches based on artificial intelligence and deep neural networks in order to decompose mixed audio signals to separate a vocal part of the signal. Some Al systems usually implement a convolutional neural network (CNN), which has been trained by a plurality of data sets for example including a vocal track, an instrumental track and a mix of the vocal track and the instrumental track. Examples for such conventional Al systems capable of separating source tracks such as a g voice track from a mixed audio signal include: Prétet, "Singing Voice Separation: A study on training data", Acoustics, Speech and Signal sing (ICASSP), 2019, pages 506-510; "spleeter" — an open-source tool provided by the music streaming y Deezer based on the teaching of Prétet above, "PhonicMind" (https://phonicmind.com) — a voice and source separator based on deep neural networks, "Open-Unmix" — a music source separator based on deep neural networks in the frequency domain, or "Demucs" by Facebook Al Research — a music source separator based on deep neural ks in the rm domain. These tools accept music files in standard formats (for example MP3, WAV, AlFF) and decompose the whole song to provide osed/separated tracks of the song, for example a vocal track, a bass track, a drum track, an accompaniment track or any mixture thereof. The tracks may then be stored by the user for audio production, analysis purposes or playback. For example, the conventional Al systems may be used to set up a karaoke e by removing vocal tracks from commercially available songs.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Although one could think about g source tracks as input files into a DJ system to allow recombining these tracks during a live show, such approaches did not become popular among DJs for several reasons.

First, dual source tracks, such as original voice tracks, of commercially released music are usually not available separately, as record labels or producers usually only provide complete mixed audio files. Second, in the case of streaming music where audio data needs to be processed and played on the fly, it would usually not be possible to download the entire audio content upfront and eprocess it prior to playback with a software module such as one of the tools mentioned above.

Third, decomposition of audio files having a typical playback duration of a couple of minutes or more involves complex calculations which are relatively time consuming, especially when using a neural network, such that extensive time and work is required to e the setup before a show, i.e. to upload, decompose, download, store and organize all the tracks that are possibly desired to be used during the show. In general, decomposition using neural networks is known to achieve good quality and precise separation of tracks but it takes relatively long time to decompose an audio file of usual size (playback duration of a few minutes). On the other hand decomposition without neural networks, for example based on simple digital signal processing such as cting the left from the right channel in a stereo audio file, also known as phase cancellation ded on the tion that many vocal source tracks are recorded in mono and are equally applied on both stereo channels), is known to be quick and with regards to sing time may be suitable for use under live stances, however these approaches provide low quality and y do not yield the expected results. Fourth, DJ systems allowing playback of separated tracks require multichannel mixing units having separate faders for ing the volume of each track. Such onal faders are difficult to operate aneously for bDJ and increase the complexity of the system.

In view of the above background, it is an object of the present invention to provide a method and a device for processing and playing audio data which increase the artistic and creative freedom of a user to l the playback of audio data, preferably under the circumstances of a live show.

According to an aspect of the invention, there is provided a method for processing and playing audio data, comprising the steps of: receiving mixed input data, the mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track; decomposing the mixed input data to obtain at least a first decomposed track resembling the at least one first source track; generating output data based on the first decomposed track; n the mixed input data are first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track and wherein the method further comprises the steps of: receiving second mixed input data, the second mixed input data being a sum signal obtained from mixing at least one third source track with at least one fourth source track; decomposing the second mixed input data to obtain a third osed track resembling the at least one third source track, and a fourth decomposed track resembling the at least one fourth source track, wherein a control input represents a desired setting of a first volume level of the first decomposed track, a second volume level of a second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth osed track; and wherein recombined output data is generated by recombining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level; and playing the ined output data through an audio output.

According to another aspect of the invention, there is provided a device for processing and playing audio data, comprising: an audio input unit for ing mixed input data, the mixed input data being a sum signal obtained - 5A - from mixing at least a first source track with at least a second source track, wherein the audio input unit is a first audio input unit for receiving first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track, a first decomposing unit connected to the audio input unit for osing the mixed input data to obtain at least a first decomposed track resembling the first source track; a second audio input unit for receiving second mixed input data, the second mixed input data being a sum signal ed from mixing at least a third source track with a fourth source track; a second decomposing unit connected to the second audio input unit for decomposing the second mixed input data to obtain a third decomposed track resembling the third source track and a fourth decomposed track resembling the fourth source track; a recompose lling section adapted to be controlled by a user to generate a control input representing a desired setting of a first volume level of the first decomposed track, a second volume level of a second track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track and a playing unit for playing recombined output data; a recombination unit for recombining at least the first decomposed track with the second track to generate recombined output data for the g unit, wherein the recombination unit is adapted to generate the recombined output data by recombining the first decomposed track at the first volume level, the second track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level.

In order to solve the above object, according to a first aspect of the present invention there is provided a method for processing and g audio data, sing the steps of: providing mixed input data, said mixed input data being obtained from mixing a plurality of source tracks; processing the mixed input data by an AI system comprising a neural network trained to separate audio data of predetermined timbres from mixed audio data, wherein the mixed input data are processed by the AI system to obtain a group of - 5B - decomposed tracks comprising at least a first decomposed track representing audio signals of a first predetermined timbre, a second decomposed track representing audio s of a second predetermined timbre different from said first predetermined timbre, and a third decomposed track representing audio signals of a third predetermined timbre different from said first and second predetermined timbres; reading a control input from a user, said control input representing a desired setting of a first volume level and a second volume level; recombining at least first and second selected tracks selected from the group of decomposed tracks to generate a first recombined track; recombining the first recombined track at the first volume level with at least a third track selected from the group of decomposed tracks, at the second volume level, to obtain a second ined track; playing the audio data based on the second recombined track. ing to an important feature of the first aspect of the present invention, mixed input data are processed by an AI system to generate at least three decomposed tracks of different timbres. For this e, the AI system may comprise one neural network trained to separate two or three decomposed tracks as an output. Alternatively, two or more neural networks may be used in the AI system, which are suitably ured to operate independently from [The next page will be page 6] [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] a.Chapman ed set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman one another, wherein different neural networks of the Al system are configured to generate decomposed tracks of different timbres. Such neural networks or layers of a neural network may interact with one r to exchange data during the separation process in order to achieve istic effects and improve the separation result.

The Al system has been trained by training data which comprise both mixed input data, as well as tracks of a n timbre which are components of the mixed input data, i.e. tracks of a certain timbre which timbre is included in the perceived sound of the mixed input data when played. As mentioned above, Al systems being able to separate from mixed audio data, tracks of a specific timbre are known in the art for other purposes.

In the context of the t disclosure and in particular all aspects and embodiments of the present invention, different timbres pond to components of the audio signal that ate from different sound sources such as different musical instruments, different software instruments or samples, different voices etc. In particular, a certain timbre may refer to at least one of: - a recorded sound of a certain musical instrument (such as bass, piano, drums (including classical drum set sounds, electronic drum set sounds, percussion sounds), guitar, flute, organ etc.) or any group of such instruments; - a synthesized sound which has been synthesized by an analog or digital synthesizer, for example to resemble the sound of a certain musical instrument (such as bass, piano, drums (including classical drum set sounds, electronic drum set , sion sounds), guitar, flute, organ etc.) or any group of such instruments; - a sound of a vocalist (such as a singing or rapping vocalist) or a group of such vocalists; D- any combination thereof.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman These timbres relate to ic frequency components and distributions of frequency components within the spectrum of the audio data as well as temporal distributions of frequency components within the audio data, and they may be separated through an Al system specifically trained with training data containing these timbres.

Another important feature of the first aspect of the present ion relates to recombining the at least three decomposed tracks in such a manner that a user does not need to select dual volume levels for each of the three or more decomposed tracks, but instead is able to control the ination result and thus the playback of the osed tracks by only setting first and second volume levels. Controlling first and second volume levels may be easily achieved by using two separate control elements (such as buttons or faders). Preferably, controlling is realized by single control element.

Specifically, in a first recombination step, at least two decomposed tracks are selected, which are recombined to generate a first recombined track. The user may then choose a volume setting just for the first recombined track and any third track, and the method may then recombine these tracks according to the set volume levels in order to obtain a second recombined track, which is rooted then to a playing unit for playback.

For example, an Al system may be used which is configured and trained to generate four , Le. a drum track, a bass track, a vocal track, and a complement track, said complement track including all remaining / timbres such that a mixture of the complement track with the drum track, the bass track and the vocal track will result in an audio signal ntially equal to the mixed input signal. In such an Al system, when recombining bass, vocal and complement tracks, at first, to obtain a first recombined track, a single control element may be used by the user in order to control the volumes of the decomposed drum track versus the recombined remainder of bmix. This allows a DJ to easily blend between two important sound [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman components, i.e. the drum component and the tonal nic or melodic) component of a song.

By simply changing the selection of decomposed tracks in the first recombination step, the user may use the same Al system in an alternative configuration, for example for a different type of music. In the above example, if the drum track, the bass track and the complement track are selected for recombination in the first recombination step to generate the first recombined track, the user may control the volume of the vocals versus the volume of the remainder of the mix (the mental part) by a simple control input, in particular a single control t such as to easily blend between vocal (acapella) and instrumental. r to the example given above, it is in general preferred in a method of the first aspect of the invention that at least one of the first, second and third predetermined timbres is selected from the group consisting of a drum timbre, a vocal timbre, and a tonal timbre ng a harmony, key or melody of the mixed input data. These are the essential sound components to be influenced in DJ equipment in order to create certain audible s. These timbres relate to specific frequency components and distributions of frequency components within the um of the audio data as well as temporal distributions of frequency components within the audio data, and they may be separated through an Al system specifically trained with ng data containing these timbres.

Furthermore, preferably at least one of the first, second and third predetermined timbres is a complement timbre, Le. a complement track is generated during decomposition as well, such that a mix of all decomposed tracks (including the complement track) substantially equals the mixed input data. Thus, all audio ents are still ented in the sum of all ?omposed tracks such that a DJ may control the audible sound such as to [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman W0 75456 approach the sound of the original mix/original song or deviate therefrom to a desired amount. ing to a second aspect of the present invention, there is provided a device for processing and playing audio data, preferably DJ equipment, comprising: an audio input unit for providing mixed input data, said mixed input data being obtained from mixing a plurality of source tracks; an Al system comprising a neural network d to separate audio data of predetermined timbres from mixed audio data, wherein the Al system is ured to receive and process the mixed input data and to generate a group of decomposed tracks, comprising at least a first decomposed track representing audio signals of a first predetermined timbre, a second decomposed track representing audio s of a second predetermined timbre different from said first predetermined timbre, and a third decomposed track representing audio signals of a third ermined timbre different from said first and second predetermined timbres; a controlling section adapted to be lled by a user to generate a l input representing a desired setting of a first volume level and a second volume level; a recombination unit configured to recombine at least first and second selected tracks selected from the group of decomposed tracks to generate a first recombined track, wherein the recombination unit is further configured to recombine the first recombined track at the first volume level with at least a third track selected from the group of decomposed tracks, at the second volume level, to obtain a second recombined track; a playing unit configured to play audio data based on the second ined track.

A device of the second aspect of the invention is specifically configured to carry out the method of the first aspect of the invention and their embodiments as bed above such as to achieve the respective effects and advantages.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman In a preferred ment of the second aspect of the invention, the device comprises a mode control unit configured to change an operational mode of the device at least between a first operational mode and a second operational mode, wherein in the first operational mode, the recombination unit is configured to recombine a first set of selected tracks selected from the group of decomposed tracks to generate the first recombined track, and wherein in the second operational mode, the recombination unit is configured to recombine a second set of selected tracks ed from the group of decomposed tracks to generate the first recombined track, said second set of selected tracks being different from said first set of ed tracks. In this embodiment, the same Al system may be used to l different groups of sound components by simply changing the selection of decomposed tracks to be ined, wherein the user input still affects a first recombined track, Le. a group of tracks, such as to ensure a fast and a simple control.

The mode control unit may se a mode control element operable by a user to selectively set the device to the first operation mode or the second operation mode. In this ment, the user still has an influence on the selection and may thus change the selection of decomposed tracks depending on the type of music to be played, but on the other hand has the advantage that not all individual decomposed tracks need to be controlled during the actual playback.

A method of the first aspect of the invention and a device of the second aspect of the invention have particular advantages when implemented in DJ equipment. In particular, the simplified control of a recombination of decomposed tracks as well as the features allowing to adapt the grouping of decomposed tracks to a certain type of music make the creative work of a DJ more intuitive and fast while maintaining the necessary flexibility. Therefore, in a device of the second aspect of the invention the audio input unit Hiterably comprises a first input section configured to e first mixeddata and a second input section configured to receive second mixed ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman input data different from said first mixed input data, wherein the recombination unit is configured to recombine audio data originating from the first mixed input data with audio data originating from the second mixed input data. The device is therefore configured to receive mixed input data of two different songs, for example. By controlling the volume levels of certain osed tracks of both songs, the DJ may more flexibly and more smoothly blend between the two songs and may achieve certain new audio creations and effects such as through recombining decomposed tracks of different songs. For example, the DJ may play vocals of one song over the instrumental of another song. This will be discussed in more detail below with respect to other s and embodiments of the t invention.

According to a third aspect of the present invention, the above object is achieved by a method for processing and playing audio data, sing the steps of (a) receiving mixed input data, said mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track, (b) decomposing the mixed input data to obtain at least a first decomposed track ling the at least one first source track, (c) generating output data based on the first decomposed track, and (d) playing the output data through an audio output.

In the context of the present disclosure, mixed input data are representative of an audio signal obtained from mixing a plurality of source tracks, in particular during music production. Thus, mixed input data are obtained from a previous mixing process that has been ted before the start of the processing according to the method of the present ion. In other words, the methods of the invention is using input data from a previous mix down process, separate from the processing of the present invention. In particular, the mixed input data may be audio files, for example audio files ning a piece of music that has been produced in a recording studio by mixing a Hkality of source tracks. For example, a first source track may be a vocalobtained from ing a vocalist via a microphone, while a second [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman source track may be an instrumental track obtained from recording an instrumentalist via a microphone or a direct line signal from the instrument.

Usually, a plurality of vocal tracks and/or a plurality of instrumental tracks are recorded at the same time or one after another. The plurality of source tracks are then transferred to a mixing station in which the source tracks are individually edited, various sound effects are applied to the source tracks, individual volume levels are ed and preferably one or more mastering effects are eventually applied to the sum of all . At the end of the production process, the final audio mix is stored on a suitable recording medium, for example in an audio file on the hard drive of a computer. Such audio files preferably have a conventional audio file format such as MP3, WAV, AIFF or other, in order to be readable by standard playback devices, such as computers or smartphones running an Apple mobile operating system (iOS), an Apple Macintosh operating system (macOS), a Microsoft Windows ing system or a Google Android operating system etc.

In ments of the present ion, mixed input data are preferably audio files, especially audio files containing music. Methods or devices of the present invention are preferably configured to store such audio files onto local storage means of the device, and/or to receive, ally to , such audio files from a remote server, for e via internet. Devices of the present invention may therefore comprise network connectivity means, such as a WlFl interface or a LAN interface, ured to receive the audio files. Alternatively or in addition, the devices may have any other tivity, such as a Bluetooth interface or a USB port, adapted to receive audio files.

The device may have a hard drive to store audio data.

According to an embodiment of the present invention, the steps of receiving mixed input data, decomposing the mixed input data, generating and playing output data are carried out in a continuous process. This means that Ringessing of the audio data from input (receiving mixed input data) to outputoutput data) is carried out continuously, or on the fly, i.e. without [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ntial time delay. For example playback of decomposed audio data can be started within a time period smaller than 2 seconds, preferably smaller than 150 milliseconds, most preferably smaller than 50 milliseconds, from the receipt of the mixed input data. In particular, it is not necessary to upload audio files to a remote server of a decomposition service provider, wait for the decomposition to be finished on the remote server and download the decomposed track from the server to a local device, store the decomposed track on a local memory and then play the decomposed track. The uous processing may be realized by carrying out all steps of the process, i.e. receiving mixed input data, decomposing the mixed input data, generating and playing output data within a single device or within a plurality of local devices all connected to one another by cables and/or in a local network and/or via near-field-wireless connections (WlFl, Bluetooth, IR etc.).

In addition or alternatively, the continuous sing including the steps of ing mixed input data, decomposing the mixed input data, generating and playing output data may be implemented within a single software application (single software program or app), adapted to be run on an electronic control unit (ECU) such as a computer, a tablet, a smartphone, a standalone DJ re console etc.

According to the method of the present invention, mixed input data of the above-described type are received and decomposed such as to obtain a first osed track that les a first source track or a sum of first source tracks. Optionally a second decomposed track may be obtained during decomposition that les a second source track or a sum of second source tracks. For example, the first decomposed track may resemble an al voice track or the sum signal of a plurality of voice tracks, for example the sum of original voice tracks from each vocalist of a choir or the sum signal of two or more voice tracks from a doubled vocal track. se, the second decomposed track may resemble an original instrumental track Htipleh as a single recorded or produced instrumental track or a sum signal ofinstrumental tracks such as the sum signal of all instrumental tracks.

[Annotation] a.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman In a preferred ment, the first decomposed track resembles the main vocal track or the sum of all main vocal tracks in isolation, whereas the second decomposed track les the remainder of the mix, i.e. the sum of all tracks except for the main vocal track/tracks.

The quality of decomposition results, i.e. how y a decomposed track resembles a corresponding source track or a ponding mixture of source tracks, can be assessed by using for example the MUSDB18 dataset (Zafar Rafii, Antoine Liutkus, Fabian-Robert Stéter, nos loannis Mimilakis, and Rachel Bittner. The musdb18 corpus for music separation, 2017) which serves as the standard benchmark and reference database for the evaluation of decomposition algorithms. Experiments on the MUSDB18 dataset typically measure the signal to distortion ratio (SDR) among other measures as well as human evaluations as assessed by mean opinion score (MOS) on perceived quality of the decomposition results.

State-of—the-art SDR scores are greater than 5.0, some are even greater than 7.0 (Défossez, A., Usunier, N., Bottou, L., & Bach, F. (2019). Music Source Separation in the Waveform Domain. arXiv preprint arXiv:1911.13254.) As regards human evaluation for example 2 ratings may be provided on a scale from 1 to 5: first the quality and absence of cts (1: many artifacts and distortion, content is hardly recognizable, 5: perfect y, no artifacts) and second the contamination by other source tracks (1: contamination if frequent and loud, 5: no contamination). State-of—the-art human evaluation ratings are greater than 3.0, or even r than 4.0. Preferably, in embodiments of the present invention, in a step of decomposing a mixed track ed from mixing at least a first source track with at least a second source, to achieve a decomposed track, the decomposed track les the first source track with a MOS score of greater than 2.0, preferably greater g 4.0, and/or with a SDR score of greater than 3.0 dB, preferably greater5.0 dB.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman W0 75456 In an ment of the invention, there may be provided a second track and the method may further comprise the following steps, preferably carried out in the continuous process: reading a control input from a user, said control input representing a desired setting of a first volume level of the first decomposed track and a second volume level of the second track, recombining at least the first decomposed track at the first volume level with the second track at the second volume level to generate recombined output data, and playing the recombined output data. The second track may be an independent track obtained from second mixed input data (for example a second song) or may be obtained in the step of decomposing the mixed input data and thus may form a second decomposed track resembling the at least one second source track of the mixed input data.

In the context of the present sure, ining a first track with a second track can be achieved in any manner known as such, for example by a recombination processing comprising a first step of g the tracks based on their respective volume levels (for example multiplying the signal values of the first and second tracks by their respective first and second volume levels, or using any suitable amplifying unit which amplifies the first and second tracks ing to their first and second volume levels), and a second step of mixing the scaled/amplified tracks in a software or hardware mixer (for example by summing the signal values of the scaled/amplified tracks at equal or corresponding time ).

In particular, according to an embodiment of the t invention there is provided a method for processing and playing audio data, comprising the steps of receiving mixed input data, said mixed input data being a sum signal obtained from mixing at least one first source track, for example a vocal track, and at least one second source track, for example an instrumental track, decomposing the mixed input data to obtain at least a first ?endomposed track resembling the at least one first source track, and adecomposed track resembling the at least one second source track, [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman reading a control input from a user, said control input representing a desired setting of a first volume level of the first decomposed track and a second volume level of the second decomposed track, recombining at least the first decomposed track at the first volume level with the second osed track at the second volume level to generate recombined output data, and playing the recombined output data.

Using a second track according to any of the described embodiments is particularly suitable for live applications in order to create and immediately play novel recombinations, e.g. remixes or mashups, based on the first decomposed track and r track. Preferably, in the present invention the method allows a user to control the volume levels of the first and second decomposed tracks and recombine the first and second osed tracks at the specified volume levels such as to play the recombined output signal obtained from recombining of the first and second osed tracks. ically, this allows a DJ to make seamless tions between songs, in particular to avoid overlapping vocal playback during a transition. Even during a time interval of the playback where both songs contain vocal parts, the user may seamlessly blend n the songs, for example by simply fading out the volume of one of the vocals such as to provide space to fade in the vocals of the other song. At any time before, after or during the vocal transition, the user may perform a transition from the instrumental track of the first song to the instrumental track of the second song.

Another advantage achieved by a method of the invention is that a user is able to access individual components of an audio mix in order to be able to recombine these components in a modified manner to create a so-called mashup or remix. This es a number of new options for the creative or artistic work of the user when playing audio such as during a live show. For example, the user can use the control input to manipulate a volume ratio m:een a decomposed vocal track and a decomposed instrumental trackplaying the recombined output data, for example swipe between vocal [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman and instrumental as desired, swap vocals of two songs, while maintaining instrumentals or vice versa as will be discussed in more detail later.

In a preferred embodiment of the invention, osing the mixed input data is carried out segment-wise, wherein decomposing and, if applicable, recombining are carried out based on a first segment of the mixed input data such as to obtain a first segment of output data, and wherein osing of a second t of the mixed input data is d out while playing the first segment of output data.

A segment-wise and parallel osition of mixed input data according to this embodiment drastically decreases the time required for performing the decomposition calculations and thus allows to start playback of the output data significantly earlier, preferably immediately, i.e. with no noticeable latency. In particular, it is not necessary to decompose the complete input data such as the complete audio file in order to obtain playable decomposed track data. Instead, it will be sufficient to finish osition of only one segment, or only some but not all of the ts of an audio file in order to be able to start playback, because parts of the decomposition calculations, in particular decomposition of other segments of the audio file, will be carried out during playback of previous segments.

Another technical effect of segment-wise decomposition is that it is more memory efficient and it will not be necessary to store all mixed input data, in particular the whole input audio file, at once within a local memory of the equipment (which may not even be possible nor desired for sources providing audio material intended for streaming usage, but not for download and/or permanent storage of the entire input audio file on a hard drive, for example). Instead, decomposition and recombining of the audio data can be carried out on the fly on the basis of a continuous stream of mixed input data, xample a continuous audio stream from a remote server, such as amusic service (Spotify, Apple Music etc.). Thus, the mixed input [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman data may be ed via streaming from a remote sewer, preferably through the internet.

Another advantage of segment-wise decomposition is that playback of the output data can be started at any desired position (at any desired g time). In particular, the first segment, which is decomposed first, does not arily have to be the starting segment at the beginning of the audio file.

In particular, it is not necessary to process and decompose the entire audio file, but instead decomposition can be started at the very segment that contains the desired playing position. Thus, it is possible to quickly and preciselyjump forward and backward to arbitrary positions within an audio file with low or even without any recognizable delay, independent of the size and playback duration of the entire audio file.

According to a further embodiment of the present invention, a method of the described type can be provided, wherein in step (a) an input audio file having a predetermined file size and a predetermined playback duration is received, which contains audio data to play the mixed input data, and a first segment is extracted from the input audio file, which contains audio data to play the mixed input data within a first time interval smaller than the predetermined ck duration, n in step (b) the first segment of the input audio file is decomposed to obtain a first t of the first osed track and ally a first segment of the second decomposed track, wherein in step (c) a first segment of the output data is generated from the first segment of the first decomposed track, preferably by recombining at least the first segment of the first decomposed track at the first volume level with the first segment of the second decomposed track at the second volume level, and wherein the method further comprises the steps of: (a2) extracting a second segment from the input audio file, which is different from the first segment and which contains audio data to play the mixed input data within a Hit0nd time interval smaller than the predetermined playback duration of theaudio file and shifted in time with t to the first time interval, (b2) [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman osing the second segment of the input audio file to obtain a second segment of the first decomposed track and optionally a second segment of the second decomposed track, optionally (02) recombining at least the second t of the first osed track at the first volume level with the second t of the second decomposed track at the second volume level to generate a second segment of the output data, n at least one of the steps (a2), (b2) and (c2) is med while playing the first segment of the output data, and wherein generation of the second segment of the output data is completed before playback of the first segment of the output data is completed.

In the present disclosure, the file size or size of audio data refer to the total number of frames of decoded and/or uncompressed data. Depending on the specific sampling rate of the audio data, a certain number of frames ponds to a certain playback duration.

According to this embodiment, the mixed input data is an input audio file of a predetermined file size and predetermined playback duration. Such input audio file can be retrieved from a local storage device or can be streamed, for example from a remote server via internet etc. As described above, the input audio file (or an image/copy thereof), ably a decoded version of the input audio file in case of using a compressed format as input, is virtually partitioned into at least two segments and further processing, in particular decomposing, is performed based on the segments. As soon as a segment is decomposed, the segments of the decomposed tracks can immediately be further processed, in particular recombined and played, while decomposing of the second segment can be commenced or continued. Since the size (in ) and playback duration of the segments is usually smaller (preferably much smaller, for example less than 20 seconds playback duration) than the size and playback duration of the input audio file (usually several minutes), abacktime required for decomposing a segment is significantly smaller andof the corresponding segment of the output signal can be started [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman earlier. Furthermore, during playback of a first segment of the output data, sing of a second segment of the input audio file, in particular decomposing the second segment to obtain a second segment of the first decomposed track and optionally the second decomposed track, can be d out in el. Preferably, all segments have fixed equal size.

Preferably, the size of the individual segments into which the input audio file is partitioned is adapted to the processing time required for decomposing the individual segments such that generation of the second segment of the output data is completed before ck of the first segment of the output data is completed. As a result, playback of the whole output track, Le. a continuous ck of all consecutive segments of the whole output track or of a part to be played, can be performed as soon as the processing/decomposition of the first segment is completed.

Preferably, the length of the first time interval is set such that the time required for decomposing the first segment is smaller than 2 seconds, such that the method can be used in a live situation for example by a DJ who may spontaneously decide to play one or more decomposed tracks to achieve specific effects. Furthermore, if the time required for decomposing the first segment is set to be smaller than 150 milliseconds, a playback of decomposed tracks can be triggered more precisely in time with a given beat such as to allow lly real-time playback of any part of the audio file. Most preferably, the time required for decomposing the first segment is smaller than 50 milliseconds such that playback and beat/timing synchronization and position shifting within the audio file can be performed virtually latency-free which means that no recognizable time lag occurs. Under such conditions, the decomposed tracks of the audio file can be handled by the DJ in the same manner as original mixed audio files or conventional effect tracks etc.

In order for a person skilled in the art to find a suitable segment size, he/she gfigurationld measure the time t1 ed for a given hardware and reto decompose audio data of a certain playback duration T1, [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman then choose a desired decomposition time t2 according to the requirements as the time delay/latency that would be acceptable, and then obtain the playback duration T2 of the segment, for e from the equation T2=T1*t2/t1. Partitioning of the audio data may then be carried out into consecutive segments which each have a playback duration T2. If the segment size is chosen to be too small, the quality of decomposition will be lowered. If the t size is chosen to be too large, sing time and thus latency is increased.

By using methods as described above, in particular methods which process segments of the input audio file to decrease processing time to a level suitable for a live performance it would in principle be possible to start playback of the decomposed tracks at any desired position (time on within the output track) by sing a segment of ied size which starts right at the desired playing position. This will in principle achieve acceptable results with regard to y and audio quality and is usable for an application where the userjust wants to jump to a certain position in the track to play the track from this position onwards, for example in a music player ation. However, especially during creative DJ work it is sometimes desired to quickly and precisely perform small positional shifts, switching between forward or backward playback or changes in playing speed. For e during a technique called "scratching", the DJ quickly toggles n rd and backward playback at a certain position of the song to achieve the corresponding scratching audio effect resembling the effect achieved by quick fonNard and backward rotations of a playing vinyl record. The inventors of the present invention have found that ation of such techniques to the output track obtained from decomposed tracks may result in audio artefacts and may not e the results as expected from applying the same technique to a corresponding source track or a corresponding conventional mixed track.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman The inventors have found that this problem can be solved by an embodiment of the present invention, which comprises the steps of receiving an input audio file having a predetermined file size and a predetermined playback duration, which ns audio data to play the mixed input data, partitioning the input audio file into a plurality of segments in succession, which contain audio data to play the mixed input data within a plurality of time intervals following each other, receiving a play position command from a user representing a user’s command to play the input audio file from a certain start play position, identifying a first segment out of the plurality of predetermined segments such that the start play position is within the time interval which corresponds to the first t, decomposing the first segment of the input audio file nt to be processed first, not necessarily starting segment of the input audio file) to obtain a first segment of the first decomposed track and optionally a first segment of the second decomposed track, generating a first segment of the output data based on the first t of the first decomposed track, preferably by recombining at least the first segment of the first decomposed track at the first volume level with the first segment of the second decomposed track at the second volume level, and playing the first segment of the output data starting at the start play position, which is a play position later than or equal to the start of the time interval of the first segment of the output data. For clarity, the first t is not necessarily the starting segment of the audio file, but a segment containing the desired start play position and therefore to be decomposed first in the process.

It is noted the ts in succession refers to segments, in particular fixed segments, chosen such that a starting position of a certain t is later than the starting ons of all preceding segments.

According to this embodiment, decomposing of the input audio file is again performed in segments. However, the starting point of the first segment to be msen by the user as one could have thought to be the most time-effectiveomposed is not necessarily cal to a desired start play position [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman way to define the segment as it contains the data to be played next, but instead fixed partitioning of the whole input audio file is set in the partitioning step, such that in each decomposing cycle for this input audio file the start points and end points of the segments will be maintained fixed. It has turned out that this technique y reduces or even completely avoids any sound cts at nt parts of consecutive segments in the decomposed track and the output data. If an Al system is used in the step of decomposing, one reason for the improvement could be that the decomposed audio signal at a specific audio position in the track depends on the data analyzed by the Al system before and after the specific audio position in the track, i.e. on the audio data between the start and end points of the respective t that contains the specific position and is ed by the Al system. Having predefined segments with fixed start and end points throughout the input audio file ensures that the same osed audio data are always obtained from an is of the same piece of input audio data that lie within the same segments. For example, if a segment starts at position 15:00 (seconds: hundredth of one second) and has a playback duration of 5 seconds, then start play positions 15:30, 17:50 or 18:00 are all based on the identical decomposed segment only using a different position offset within the same decomposed segment.

In another preferred embodiment of the t ion, the mixed input data is first mixed input data being a sum signal obtained from mixing at least a first source track, for example a first vocal track, with a second source track, for example a first instrumental track or multiple instrumental tracks, and the method further ses the step of receiving second mixed input data ent from said first mixed input data, said second mixed input data being a sum signal obtained from mixing at least one third source track, for example a second vocal track different from the first vocal track, with at least one fourth source track, for example a second instrumental track different gomposingthe first instrumental track. Furthermore the method may comprise:the second mixed input data to obtain a third decomposed [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman track resembling the at least one third source track, and a fourth decomposed track resembling the at least one fourth source track, wherein in the step of reading the control input from a user (e.g. received via one or more control ts), said control input represents a desired setting of the first volume level of the first osed track, the second volume level of the second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track, and wherein, in the step of recombining, the recombined output data is generated by recombining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level.

Such embodiment can in particular be used in a DJ environment and can be implemented in DJ equipment where two different mixed input data such as two different audio files (e.g. two different songs) are played simultaneously at least for a certain amount of time. For example, during playback of a first song, the playback of a second song is started and the volume level of the first song is reduced to the benefit of the volume level of the second song, such as to smoothly blend over from the first to the second song. In the method of the ment described above, both first and second mixed input data may be decomposed into first and second decomposed tracks and third and fourth osed , respectively, and the user may have an unity to recombine the decomposed tracks individually at the desired volume levels. As bed above, such features may in particular be used by a DJ to smoothly blend between two songs while ng that the vocal tracks of different songs are audible at the same time. Moreover, as it will be possible according to such an embodiment to e one of the decomposed tracks of the first mixed input data with one of the decomposed tracks of the second mixed input data, a sum signal or mashup/remix of leetelyponents of the first and second mixed input data can be created that isnovel and may therefore achieve sing effects. For example, [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman the decomposed vocal track of the first mixed input data could be recombined with the decomposed instrumental track of the second mixed input data to let the auditory hear the singer of one song being accompanied by the instrumentals of the other song.

Preferably, at least one or more, most preferably all of the mixed input data and of the decomposed track(s) are stereo data, each comprising a left l signal portion and a right channel signal portion, respectively. Thus, the method is adapted to make use of the capabilities and acoustic effects of stereo audio. In other embodiments, mono data and mono tracks, or tracks having any other number of channels (e.g. 5.1 or 7.1 surround tracks or MP4 with multiple streams) could be used.

Decomposing the mixed input data to obtain the at least one decomposed track may be realized by any algorithm, filtering, effect application or other processing that is capable of separating at least one decomposed track (in particular a vocal track) from the mixed input data that is acoustically similar or equal to one or more source tracks of the original audio material that was mixed-down earlier to obtain the mixed input data. For example, decomposing may e calculation of a frequency um, for example by using a Fourier transformation algorithm, filtering the frequency spectrum to extract frequencies ing to a n component of the mixed input data, in particular a vocal ent, and retransforming the ted frequencies, for example by using an inverse r transformation, to obtain audio data of the first or second decomposed track.

In preferred embodiments of all aspects of the present invention, decomposing the mixed input data es processing the mixed input data by an Al system icial intelligence system), said Al system preferably being based on at least one deep neural network, for example a gainingvolutional neural network (CNN) and/or being trained by a plurality of setsaudio data. Each set of training audio data may at least include a [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman first source track, for example a vocal track, a mixed track being a sum signal obtained from mixing at least the first source track with a second source track, for example an instrumental track.

The use of an Al system allows high quality decomposing of the mixed input data such that the decomposition result very closely resembles al source tracks or sum signals of less than all original source tracks. For example resemblance with a MOS score of greater than 4.0, and/or with an SDR score of greater than 5.0 dB could be achieved by using an Al system.

An Al system may be provided and trained by audio data obtained from one or more record labels and/or music production companies or one or more music distribution/streaming companies or a ation between them. For the purpose of training the Al system, the recording company may not only provide the mixed audio file of a certain recording but also one or more training source tracks which are included in the mixed track and which are obtained from the original material of the production process, i.e. the audio data of individual tracks or sums of less than all tracks used in the mixing process. After training the Al system with a large number of sets of training audio data, the Al system will be able to generate a decomposed track from a new mixed track (a new audio file) that has not been analyzed before during the ng phase. For example, the Al system may be based on one of the above-mentioned conventional Al systems (eg. spleeter, Open-Unmix, Demucs).

In embodiments of the present invention, at least one Al system, ably multiple Al systems, may be fully stored and ing within a random access memory (RAM) of a device, such as to reduce the time required for decomposing the mixed input data and even allow nearly y-free operation in a live situation, such as in a DJ device. gording to an embodiment of the present ion, an Al system ed above is used in the step of decomposing the mixed input data, [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman wherein the invention, in an embodiment, proposes not only to extract a first decomposed track but also to extract a second decomposed track, such that a user can manipulate the volumes of the first as well as the second decomposed track individually and recombine both decomposed tracks for achieving a certain playback . Preferably, the second osed track is a complement of the first decomposed track which means that a sum of the first decomposed track and the second decomposed track closely resembles the entire audio signal of the mixed input data, except for the modified volume levels of the first and second decomposed tracks, and except for some deviations due to imperfections in the decomposing step (for e small signal components originating from the at least one first source track but being incorrectly fied by the Al system as originating from the at least one second source track or vice versa; other minor imperfections may result from the calculation processes, for e Fourier transformations, during the decomposing step).

In a red ment of the present invention, the mixed input data is simultaneously processed within a first Al system and a second Al system separate from the first Al system, wherein the first Al system processes the mixed input data to obtain only the first decomposed track and the second Al system processes the mixed input data to obtain only the second decomposed track. In particular, the method is preferably processing the mixed input data as first mixed input data and is further processing a second mixed input data simultaneously within a third Al system separate from the first and the second Al system, and within a fourth Al system separate from each of the first to third Al s, wherein the third Al system processes the second mixed input data to obtain only the third decomposed track and the fourth Al system processes the second mixed input data to obtain only the fourth decomposed track. The use of at least two separate Al systems which are arranged to operate simultaneously allows the calculation of at H, such that the processing speed can be greatly improved andtthe first and second decomposed tracks in parallel, i.e. at the sameit will be [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman possible to obtain the decomposed tracks with only small or even without recognizable time delay. If the method processes the first and the second mixed input data and thus four separate Al systems are used to be operable in parallel, fast decomposition of two separate audio files or segments thereof will be possible with only a small or even non-recognizable time delay. Such a method is therefore suitable for live performances such as by a DJ during a live show.

In a further embodiment of the invention, said mixed input data are first mixed input data based on a periodic beat structure (for example a 4/4 time signature) and the method further comprises: receiving second mixed input data ent from the first mixed input data and based on a periodic beat structure, and ming at least one of a tempo and beat matching processing and a key matching processing. In ular, the tempo matching processing may comprise: receiving first input data obtained from the first mixed input data and second input data obtained from the second mixed input data, time stretching or resampling of at least one of the first input data and the second input data, and outputting first output data and second output data which have ly ng tempos. Beat matching processing may comprise beat alignment between the first and second mixed input data, i.e. time position shifting of at least one of the first and second mixed input data, and outputting first output data and second output data which have mutually matching beat phases. Further, the key matching processing may comprise: receiving first input data obtained from the first mixed input data and second input data obtained from the second mixed input data, pitch shifting of at least one of the first input data and the second input audio data, and outputting first output data and second output data which have mutually matching keys.

In the above-described embodiment, first input data obtained from the first ged input data may be the first mixed input data itself, or any osedk (e.g. first or second decomposed track) obtained from the first mixed ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman input data, or the first recombined output data (i.e. obtained from first mixed input data through decomposition and recombination). Likewise, second input data obtained from the second mixed input data may be the second mixed input data itself, or any decomposed track (e.g. third or fourth decomposed track) obtained from the second mixed input data, or the second recombined output data (i.e. obtained from the second mixed input data through decomposition and recombination).

It should be noted that the first input data may be the first mixed input data, especially if the tempo and/or key matching processing is carried out in an early stage of the processing, i.e. before the step of decomposing.

Alternatively, the first input data may be the first decomposed track, if tempo and/or key matching and/or beat matching processing is carried out after the step of decomposing. As another example, the first input data may be a modification of the first decomposed track, eg. a cation obtained by applying an audio effect (such as delay, , equalizer etc.) to the first decomposed track. The same applies to the second input data, which may be the second mixed input data or the second decomposed track or a modification thereof.

In the above embodiment, "mutually matching tempos" means that the tempos, measured in beats per minute, of the first and second output data are either equal or multiples of one another such that the beats of the first and second output data can be onized to each other. Furthermore, "mutually matching keys" means that the harmonic keys of the first and second output data are either equal or in a relationship of a minor key and its parallel major key (the key of one of the first and second output data is a minor key having a first fundamental tone and the key of the other one of the first and second output data is a major key having a second fundamental tone that is three semitones higher than the first fundamental tone).

[Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Tempo and/or key matching processing as described in the embodiment above will icantly improve the application of the methods of the ion for live performances by a DJ, as it allows smooth blending of two songs, including crossfades between instrumental tracks and vocal tracks of two songs or swapping instrumental tracks or vocal tracks of two songs, i.e. recombining/remixing of decomposed tracks of two songs, e it allows both songs or parts thereof (decomposed tracks of both songs) to be e at the same time (at same or corresponding tempo, beat phase and key) without disturbing the flow the music.

According to a fourth aspect of the t invention, the above object is achieved by a device for processing and playing an audio signal, preferably DJ equipment, comprising an audio input unit for receiving mixed input data, said mixed input data being a sum signal obtained from mixing at least a first source track with at least a second source track, a decomposing unit connected to the audio input unit for decomposing the mixed input data to obtain at least a first decomposed track resembling the first source track, and a playing unit for playing output data based on the first decomposed track.

With such a device, the advantage as described above for the method of the third aspect of the ion can be achieved by a suitable device containing all re and re components, audio input and output means as well as processing units which are necessary to receive and decompose the mixed input data, and to play the output data.

In general, devices of the present invention may be embodied as DJ equipment comprising an electronic control unit (ECU) such as a computer, preferably a portable computer, and suitable hardware interfaces and speakers (e.g. built-in speakers or connectivity to connect the ECU to a PA system). The playing unit may have a l-to-analog-converter to convert ?tal audio data into an analog audio signal. The input unit may have a [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman decoding unit for decoding audio data encoded in different audio formats, such as MP3 or AAC.

In order to be configured for live applications, such as DJ ations as described above, the device may comprise a recombination unit for recombining at least the first decomposed track with a second track to generate the output data for the g unit. Furthermore, the device may comprise a ose controlling section adapted to be controlled by a user to generate a l input representing a desired setting of a first volume level of the first decomposed track and a second volume level of the second track, wherein the recombination unit is configured to recombine at least the first decomposed track at the first volume level with the second track at the second volume level to generate the output data. The recompose controlling section may be implemented by user ace controls displayed on a computer screen or alternatively a separate hardware which may e a housing, a control element such as a rotatable knob or a movable slider, a display, input and output ports etc.

The recompose controlling n may include a switch as a control element which allows input of only a limited number of te , in particular a switch having only two switch positions (ON/OFF, 0/1, activated/deactivated), in order to set the first and second volume levels of the first and second decomposed tracks to a high level or ON value, for example 100 %, or to a low level or OFF value, for example 0 %, respectively. For example, there could be a vocal switch in order to switch the volume level of a decomposed vocal track between ON and OFF and/or there could be an instrumental switch to switch the volume level of a decomposed instrumental track between ON and OFF. The switch may be embodied as a push button, for example provided on a touch screen display. In order to avoid artefacts due to rapid volume changes, the device may include an auto-fading unit which ?nmatically changes the volume level continuously and at a limited rate to [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman ed set by Rebecca.Chapman the desired value according to the switching position of the switch, said auto- fading being started as soon as the user operates the switch.

In an embodiment of the invention, there is provided a device, wherein the audio input unit is a first audio input unit for receiving first mixed input data being a sum signal obtained from mixing at least a first source track, for example a first vocal track, with at least a second source track, for example a first instrumental track, wherein the decomposing unit is a first decomposing unit for decomposing the first mixed input data to obtain at least the first decomposed track resembling the first source track, and the second decomposed track ling the second source track, and wherein the device r comprises: a second audio input unit for receiving second mixed input data different from the first mixed input data, said second mixed input data being a sum signal obtained from mixing at least a third source track, for example second vocal track different from the first vocal track, with a fourth source track, for example a second mental track ent from the first mental track, a second decomposing unit connected to the second audio input unit for decomposing the second mixed input data to obtain a third decomposed track resembling the third source track, and a fourth decomposed track resembling the fourth source track, wherein the recompose controlling section is adapted to be controlled by a user to generate a control input representing a desired setting of the first volume level of the first decomposed track, the second volume level of the second decomposed track, a third volume level of the third osed track, and a fourth volume level of the fourth decomposed track, and wherein the recombination unit is adapted to generate the recombined output data by recombining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level. ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman A device of this embodiment is prepared to be used in creative work to recombine, to smoothly blend over or transition between two different songs or to create mashups of the songs to achieve a variety of new effects. In particular, such device may be embodied as DJ equipment for live performance.

In embodiments of the present invention it is preferred that the ose controlling section ses at least one single recompose control element which is operable by a user in a single control operation for controlling the first volume level and the second volume level, in particular for changing a ratio between the first volume level and the second volume level from at least a value smaller than 1 to at least a value greater than 1 or vice versa.

In all aspects and embodiments of the present invention, a single recompose control element or a single control element for controlling a first volume level of a first track and a second volume level of a second track preferably refers to a control element operable by a user in a single l operation, such as single control movement (for example sliding movement, rotational movement etc.) or a single control switch operation (for example touching a button etc.), in order to change a ratio between the first and second volume levels from at least a value smaller than 1 to at least a value greater than 1 or vice versa, i.e. from a first ratio in which the first volume level is smaller than the second volume level, to a second ratio, in which the first volume level is higher or equal than the second volume level. At least in parts of a control range of the single pose) control element, volume changes controlled by the single (recompose) control element may be med simultaneously, for example by increasing the second volume level, while decreasing the first volume level. atively, or in other parts of a control range of the single (recompose) control element, volume changes controlled by the single (recompose) control element may be performed sequentially. For example g; the control range of the single (recompose) control element there mayfirst subrange in which the first volume level is increased or decreased [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman while the second volume level is maintained constant, and a second subrange in which the second volume level is increased or decreased while the first volume level is ined constant, wherein the first and second subranges do not overlap each other.

In a preferred ment, the single recompose control element may have a l range extending from a first end point at which the first volume level has a maximum value (for example about 100 percent) and the second volume level has a minimum value (for example about 0 percent) to a second endpoint at which the first volume level has a minimum value (for example about 0 percent) and the second volume level has a maximum value (for example about 100 percent). More preferably, in a middle region of the control range, the first and the second volume levels both have a maximum value (for example about 100 percent). Between the middle region and each endpoint, the first and second volume levels may remain substantially constant or may increase or decrease in a linear or non-linear manner, respectively.

This allows the user to fade or switch between the first decomposed track and the second decomposed track in a single control operation, for example a touch of a button or switch, by a single continuous rotation of a rotatable control knob or a single continuous sliding movement of a single fader, in order to run smooth linear transitions from the first mixed input data to the second mixed input data (for e from a first song to a second song) or vice versa. In particular, two volume levels of different osed tracks can be manipulated by the user with only one hand or even only one finger which improves live performance capabilities of the system. This means that, for example, one hand could be used for operating a decompose / recombine control, whereas the other hand could be used for a crossfader or for the decompose / ine control of the other song.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] a.Chapman Unmarked set by a.Chapman In a modification of the above-described embodiment, the device may further include a swap control element which, when operated by a user, controls the recombination unit such as to decrease one of the first and second volume levels and at the same time increase one of the third and fourth volume levels and/or which, when operated by a user, controls the recombination unit such as to increase one of the first and second volume levels and at the same time decrease one of the third and fourth volume levels. Note that "decrease" may include muting the volume level or setting the volume level to 0 percent, and "increase" may include setting the volume level to full scale or 100 percent.

For example, if the first decomposed track obtained from the first mixed input data is a vocal track of a first song and the third decomposed track ed from the second mixed input data is a vocal track of a second song, the swap control element described above can be activated by a user such as to control the recombination unit to switch the volume of a vocal track currently contained in the recombined output data from ON to OFF and to switch the volume of r vocal track currently not contained in the recombined output data from OFF to ON, in other words switch the ON-OFF gs of both vocal tracks inversely. As another example, if the second decomposed track ed from the first mixed input data is an instrumental track of a first song and the fourth decomposed track obtained from the second mixed input data is an instrumental track of a second song, the swap control element described above can be ted by a user such as to control the recombination unit to switch the volume of an instrumental track currently contained in the recombined output data from ON to OFF and to switch the volume of another instrumental track currently not contained in the recombined output data from OFF to ON, in other words switch the ON-OFF settings of both instrumental tracks inversely. Operation of such swap control element is preferably d in situations where the decomposed g of the osed track from the second mixed input data.k from the first mixed input data has an ON-OFF-setting different from [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman In another embodiment of the present invention, there is provided a device of the above-described type, wherein the recompose lling section comprises a first single recompose control element which is operable by a user in a single control ion for controlling the first volume level and the second volume level, in ular changing a ratio between the first volume level and the second volume level from at least a value r than 1 to at least a value greater than 1 or vice versa, and a single recombination l element, which is le by a user in a single control operation for lling a volume level of first sum signal and a volume level of a second sum signal, in ular changing a ratio between the volume level of the first sum signal and the volume level of the second sum signal from at least a value smaller than 1 to at least a value greater than 1 or vice versa, the first sum signal being a sum of the first decomposed track at the first volume level and the second decomposed track at the second volume level and the second sum signal being a sum of the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level, and preferably a second single recompose control element which is operable by a user in a single control operation for lling the third volume level and the fourth volume level, in particular changing a ratio between the third volume level and the fourth volume level from at least a value r than 1 to at least a value greater than 1 or vice versa. An advantage of a device of this embodiment is that complexity of control is greatly reduced for fast and intuitive operation, in particular by a DJ during a live show. Even if the device receives two different mixed input data which are both decomposed into at least two decomposed tracks, which already results in four individual tracks (preferably four individual stereo tracks each having right and left channels, resulting in a total of at least eight channels), by means of the first and second single recompose control elements and the single recombination control element, switching, swapping and fading between the tracks can be gsecutiveormed very quickly and intuitively by single movements or singleoperations of the control elements.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman In another embodiment of the present invention, there is provided a device which further ses an input audio file buffer for loading therein segments of an input audio file having a ermined file size and a predetermined playback on, which ns audio data to play the mixed input data, a first segment buffer connected to the decomposing unit to e and store a segment of the first decomposed track obtained from a segment of the input audio file, ally a second segment buffer connected to the decomposing unit to receive and store a segment of the second decomposed track obtained from the same segment of the input audio file, wherein the playing unit comprises an audio interface having an analog-to-digital converter to generate an analog audio signal from the output data, said audio interface having an audio buffer for buffering portions of the output data for playback, wherein the size of the first segment buffer and/or the second segment buffer is larger than the size of the audio buffer of the audio interface, but smaller than the entire audio data of the (decoded) input audio file. According to this embodiment, separate buffers are provided to store the segments (not all segments at the same time, but only one or a few of the segments) of the decomposed tracks ready for recombination and/or playback which increases the processing speed and reduces memory footprint as compared to a ion where the entire input audio file is decomposed before playback and completely stored in a separate buffer. In other words the size of each segment buffer is smaller than the size of the entire (decoded) input audio file data. On the other hand, the first and second segment buffers are each larger in size than the audio buffer of the audio interface which ensures that the audio buffer of the audio interface can always be refilled by the content of the segment buffers in a timely manner, such that a continuous output signal can be produced and played without any audio dropouts or recognizable time lags. This ment therefore also assists live capabilities of the device. ation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Preferably, the audio buffer of the audio interface has a fixed standard size to store 2" frames/samples of audio data (n being a natural number, preferably between 6 and 12), for example 512 audio frames, which corresponds to a ck duration of approximately 11 econds at a sampling rate of 44.1 kHz. In contrast the size of the segment buffers is preferably larger such as to store segments with a playback duration of more than 1 second.

In a further ment of the invention, the device may further comprise a display device for displaying a first waveform entative of the first decomposed track and a second waveform representative of the second decomposed track, wherein the first waveform and the second waveform are displayed in an overlaid manner using one single baseline, and whereas the first and second waveforms are displayed using different signal axes and/or different drawing styles such as to be visually distinguishable from one another. This allows a user to visually monitor the result of decomposition, preferably in real-time, and to adapt the control for a recombination of the decomposed tracks. In particular, it allows a user to see some future audio data to be played within the next seconds and to adapt control in time, for example to quickly fade out the vocal track of a first song before the onset of such vocals during a transition from the first song to a second song. By using only one single baseline (line of signal value null, Le. a line running along the time axis) for both decomposed rms and choosing different drawing styles or signal axes, the user will faster recognize the content of the audio data as components of the same mixed input data, such that the user can gather the information necessary for controlling the recombination unit more y or can visually cue to specific parts of the song more precisely, eg. a vocal onset at the beginning of a chorus. ing to a fifth aspect of the present invention, there is provided a method for representing audio data, said audio data comprising at least a first ghodk and a second track which are ents of a joint audio mix, saidcomprising displaying a first waveform representative of the first track [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman and a second waveform representative of the second track, wherein the first waveform and the second waveform are displayed in an overlaid manner using one single baseline, and s the waveforms are displayed using different signal axes and/or different drawing styles such as to be visually distinguishable from one r. Such id representation of waveforms according to the fifth aspect of the invention contains better semantic information about the audio content, in particular music, and thus is more akin to how humans perceive audio/music, as opposed to conventional waveform representation in a single rm per song or per mixed input data and only approximate or combined or global frequency dependent coloring.

In the present disclosure, a display in an overlaid manner using one single ne refers to a y of tracks which are drawn such that their baselines are displayed at identical positions on the coordinate system of the graphical display. Thus a display in an id manner is in contrast to a display of tracks in a stacked , in which the time axes or baselines are drawn in parallel on top of each other.

In the t of the present disclosure, components of a (joint) audio mix are in particular decomposed tracks which are obtained from a mixed input signal through a decomposition processing (for example voice/instrumental separation etc.), such as that used in the methods of the first and third aspects of the invention. Furthermore components of a (joint) audio mix may be obtained separately, i.e. separate from the audio mix, for example as so- called stems created from single source tracks or subgroups of source tracks during production of the audio mix, i.e. before the source tracks are mixed down to obtain the audio mix. Such stems are available through some record labels, for example. In any case, within the present disclosure, components of a joint audio mix belong to the same audio mix. For example, if a mixed ?g contains vocal tracks and instrumental tracks, one of the vocal tracks [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman and one of the instrumental tracks both are ents of a joint audio mix, Le. a joint song.

These advantages of the fifth aspect of the invention are particularly significant for DJ work, if one of the first and second tracks is a vocal track and the other one of the first and second tracks is an instrumental track. In particular, during blending or crossfading between two songs the DJ generally needs to keep special attention to the vocal parts of the songs and needs to avoid clashing of vocals of different songs, whereas the combination of instrumental parts of two songs is usually used as a creative effect to the t of the performance or is at least acceptable for the sake of blending/crossfading, provided that they have matching tempos, beats and keys.

Within the present disclosure (i.e. in all aspects and embodiments of the t ion), the use of different drawing styles may imply the use of different colors, line styles, hatchings or the like. Preferably the first waveform is displayed using a first drawing style which draws signal portions of the first waveform primarily or exclusively in a positive region relative to a baseline, and the second waveform is displayed using a second drawing style which draws signal ns of the second waveform primarily or exclusively in a negative region relative to the same baseline. For example, the first waveform may be displayed using a first drawing style which draws primarily or exclusively a positive signal portion of the first track, and the second waveform is displayed using a second drawing style which draws primarily or exclusively a negative signal portion of the second track. In an alternative embodiment, the first and second waveforms may be displayed using first and second drawing styles which both draw ily or exclusively the positive signal portion or which both draw primarily or exclusively the negative signal portion, wherein the first waveform may be displayed using a gsignal axis and the second waveform may be yed using a secondal axis that runs opposite to the first signal axis. In both alternative [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman embodiments the two waveforms therefore appear to be mirrored at the common baseline, with one waveform ing primarily or exclusively in one direction with t to the baseline, and the other waveform extending ily or exclusively in the other direction with respect to the baseline. This allows the user to clearly distinguish the waveforms while still having both waveforms on the same baseline for fast perception. The embodiments both make use of finding that sufficient information about the audio data can be obtained by a user from viewing only a part of the rm, in particular only a positive or only a negative half portion of the waveform. As a further alternative, only one of the two waveforms may be drawn as a half waveform (only positive signal portions or only negative signal portions, or using absolute values of the signal portions drawn in ve or negative direction), while the other one of the two waveforms is drawn with both positive and negative signal portions.

In addition to displaying the waveforms with different signal axes and/or different drawing styles, in all aspects and embodiments of the present invention, the first waveform and/or the second rm may further be displayed by rendering the waveform in a predetermined time al with a color that depends on a frequency information of the respective track within the predetermined time interval, said frequency information preferably being indicative of a dominant frequency of the audio data within the predetermined time interval, which is preferably obtained from a frequency analysis of an audio signal derived from the audio data of the respective track within the ermined time al. In this manner, the user may further be able to recognize dominant frequencies occurring at certain ons of the tracks, which may indicate the presence of certain instruments such as bass, kick drums, cymbals, guitars, etc., and thus the visualization and speed of perception of the audio content can further be improved, which allows r and more intuitive operation of a device implementing such method. An 8 B1.mple for frequency dependent coloring of waveforms is disclosed in USIf frequency dependent coloring according to the above- [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman described embodiment is applied to decomposed tracks, in particular to a decomposed instrumental track, vocal components are excluded from the frequency analysis, which allows to obtain a much better (colored) visual representation of the audio signal as it is perceived by the ears. On the other hand, if the frequency analysis is carried out for the vocal frequency spectrum tely by analyzing a decomposed vocal track, results of frequency analysis and thus correct coloring will be improved for the vocal track as well.

In particular, it is preferred to use a method of the fifth aspect of the invention in tion with a method of the first or third aspect of the invention or within a device of the second or fourth aspects of the invention, for example in order to y decomposed tracks in an overlaid manner, such as a decomposed vocal track and a osed instrumental track.

In addition, according to a sixth aspect of the invention, there may be provided a device for representing audio data, for example a display device of a computer, said audio data comprising at least a first track and a second track, which are adapted to be played in a mix, said device comprising a first waveform generator generating a first waveform entative of the first track, a second waveform generator generating a second waveform representative of the second track, and an overlay-waveform generator generating an y-waveform showing the first waveform and the second waveform in an overlaid manner using one single baseline, wherein the waveforms are overlaid by the overlay-waveform generator using different signal axes and/or different drawing styles such as to be visually distinguishable from one another.

In a further embodiment of the fifth aspect of the present invention, the method may r include the steps of ing mixed input data, said ged input data being a sum signal obtained from mixing at least one firstrce track with at least one second source track; decomposing the mixed [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman input data to obtain at least a first decomposed track resembling the at least one first source track, and a second decomposed track resembling the at least one second source track; reading a control input from a user, said control input representing a desired setting of a first volume level of the first decomposed track and a second volume level of the second decomposed track; displaying a first waveform representative of the first decomposed track and displaying a second waveform representative of the second osed track, wherein the first waveform and the second waveform are yed in an overlaid manner using one single ne, and wherein the waveforms are displayed using different signal axes and/or different drawing styles such as to be visually distinguishable from one another, wherein the first waveform is yed with its signal axis being scaled or its appearance (for example color or opacity) being modified depending on the first volume level, and wherein the second waveform is yed with its signal axis being scaled or its appearance (for example color or opacity) being modified depending on the second volume level. Such embodiment allows the volume settings or recompose gs received from the user via the l input to be directly visualized in the overlaid waveform representation. The user can therefore see volume ments directly on the display, while also seeing amplitude values of the decomposed tracks.

Preferably, in the above-described embodiment, the first and second waveforms are displayed with their signal axes being scaled or their appearance (for example color or opacity) being modified on the basis of current values of the first and second volume levels within a time period of not more than 2 seconds, preferably not more than 100 milliseconds, more preferably not more than 35 milliseconds. If the time period is smaller than 2 s, the method is usable for live performances, because it allows a user, for example a DJ, to verify or monitor the recombination of the tracks visually. If the time period is smaller than 100 milliseconds, the control setting grh as volume levels can be visualized virtually in real time, which gives thea feeling of direct feedback when changing the gs. Moreover, if the [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman time period is smaller than 35 milliseconds, which is smaller than the time period of a frame of a usual display (for example with a frame rate of 30 frames per seconds), the user will not even realize at all that there is any time delay between manipulation of the control element and visualization of the volume levels within the overlaid waveforms.

According to a sixth aspect of the invention there is provided a device for processing and playing audio data, ably DJ equipment, sing a processing unit for processing audio data of at least a first track and a second track, a controlling section adapted to be controlled by a user to generate a control input representing a d setting of a first volume level of the first track and a second volume level of the second track, a recombination/mixing unit configured to ine the first track at a first volume level with the second track at a second volume level to generate output data, a visualization unit configured to generate waveform data for visualizing at least one waveform based on the first track, the second track and the l input, a playing unit for playing the output data, and optionally a display unit for displaying the waveform data. A device of the sixth aspect of the invention provides visual feedback to a user about audio data currently played, including audio data played or to be played within a n time interval before and after the current playing position, wherein the visualization includes direct and instantaneous information about the first and/or second volume level as currently set by the user through the controlling section. For e, a DJ is then able to ly see his/her current inputs at the controlling section and will have visual information about r work in on to audible information output by the playing unit.

Preferably, the visualization unit is configured to generate a first waveform based on the first track, wherein a scaling of a signal axis of the first waveform or a drawing style of the first waveform is set depending on the first gme level, and/or to generate a second waveform based on the secondk, wherein a scaling of a signal axis of the second waveform or a drawing [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman style of the second waveform is set depending on the second volume level.

The waveforms may be visualized and displayed as separate waveforms, for example on parallel baseline a, or they may be drawn in an overlaid manner using one single baseline, preferably with different signal axes and/or different drawing styles such as to be visually distinguishable from one another, as will be described later.

In another embodiment of the device of the sixth aspect, the visualization unit may be configured to calculate a combination track representing a combination of at least the first track at the first volume level and the second track at the second volume level, and to te the rm data such as to ize the rm of the ation track. This results in the y of a waveform representing the output data obtained from the recombination/mixing unit, i.e. including the influences to the rm as affected through the control input by the user.

Preferably, at least one of the first track and the second track is a decomposed track obtained from decomposing a mixed audio signal, in particular within the , or is a ent of an audio mix provided externally and input into the device. More particularly, the device of any of the second, fourth and sixth aspects can be adapted to carry out the method of any of the first third and fifth aspects of the ion, and/or can be a device according to any other of the second, fourth and sixth aspects of the invention, wherein the first track is preferably the first decomposed track and/or the second track is preferably the second decomposed track, and the controlling section is preferably the recompose controlling section.

The present invention will now be further described based on specific examples shown in the drawings.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Fig. 1 shows a schematic view of the components of the device for processing and playing an audio signal according to a first embodiment of the present invention.

Fig. 2 shows a functional diagram of the elements and signal flows in the device ing to the first embodiment.

Fig. 3 shows a further functional diagram illustrating a signal flow in the device of the first embodiment.

Figs. 4 to 10 show second to eighth embodiments of the present invention which are each modifications of the first ment.

Fig. 11 shows a diagram illustrating a swap process able in a device of the eighth ment of the invention.

Fig. 12 and 13show a graphical representations of waveforms according to embodiments of the invention.

Fig. 14 shows an audio player according to a ninth embodiment of the Fig. 15 and 16 show tenth and eleventh embodiments of the present invention which are each modifications of the first embodiment.

Fig. 17 and 18 show a twelfth embodiment of the t invention, which is a modification of the previous embodiments.

With reference to Fig. 1, the first embodiment of the t invention is a glee 10, ably a DJ device. Device 10 comprises an input section 12able of loading a first input audio file A such as a first song A, and a [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman second input audio file B such as a second song B. Both input audio files A, B may contain audio data in a common audio file format such as MP3, WAV or AIFF, and they have a fixed file size and playback duration (in particular song length in seconds) as conventionally known to be input into DJ equipment or other playback devices. Audio files A and B may be provided, downloaded or streamed from a remote server via Internet or other network connection, or may be provided by a local computer or a storage device integrated in the device 10 itself. Input section 12 may include suitable user interface means allowing a user to select one of a plurality of available audio files as input audio file A and another one of the ity of audio files as input audio file B.

Device 10 further comprises a processing section 14, preferably including a RAM storage 16, a ROM storage 18, a persistent e 19 (such as a hard drive or flash drive), a microprocessor 20, and at least one cial intelligence system 22, for example first to fourth AI systems 22-1, 22-4 which are connected to the microprocessor 20. The processing section 14 is ted to the input section 12 to receive audio data of audio files A and B.

Device 10 further comprises a recompose controlling section 24 including at least one recompose control element 26, for example a first control element 26-1, a second recompose l element 26-2 and a mix control element 28. Recompose controlling section 24 may further comprise a first play control element 30-1 and a second play control element 30-2 for starting or ng playback of audio s originating from the first or second mixed input data, respectively.

In on, device 10 may include a recombination unit 32 connected to the recompose controlling section 24 for ining audio data based on the settings of the control elements. Recombination may be carried out by Qingstiplying different channels of audio data with scalar values based on theof the control elements and then adding the channels together [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman sample by sample. Furthermore, an audio interface 34 (for example a sound card) having a digital-to-analog-converter is preferably connected to the recombination unit 32 to receive recombined output data and to convert the lly recombined output data into an analog audio . The analog audio signal may be provided at an audio output 36 which may feature conventional audio connectors to connect audio cables such as line connectors or XLR connectors or wireless output (e.g. Bluetooth), which allow the audio output 36 to be connected to a PA system or speakers or headphones etc. (not illustrated). The PA system may include an amplifier connected to rs to output the audio signal. As an alternative, internal speakers of the device such as tablet speakers or computer rs or headphones might be used to output the analog audio signal.

Some or all components and features described above with t to the first ment may be provided by an electronic control unit (ECU), such as a er, in particular a tablet computer 35 running a software application that is programmed to operate the ECU to allow input, decomposition, recombining and output of audio data as described above with respect to Fig. 1, and to receive control input from a user, for example via a touchscreen 37 that displays the control elements of the recompose controlling section 24.

Further details of the internal components and the signal flow within the device 10 are explained in the following with respect to Fig. 2. Within input section 12, first and second input audio files A and B are obtained as described above. Input audio files A, B are then transmitted to processing section 14, which contains at least a first decomposition unit 38 and a second decomposition unit 40. First osition unit 38 includes a first segmentation unit 42 and at least one Al system, ably a first Al system 44-1 and a second Al system 44-2. The second decomposition unit 40 may ly' wise include a second segmentation unit 46 and at least one Al system, a third Al system 44-3 and a fourth Al system 44-4.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman [Annotation] a.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman The first tation unit 42 of the first decomposition unit 38 receives the first input audio file A and is adapted to partition the audio file into a number of consecutive segments. Preferably, the complete input audio file A is partitioned into segments that pond to time intervals in the audio signal that is playable from the audio file. Preferably, the starting segment is defined such that the starting point of the starting segment corresponds to the beginning of the audio file (playing position 0:00) on the time scale and the end point of the starting segment corresponds to the end of a first time interval at the beginning of the audio file. A second and each subsequent segments are then defined by consecutive time als of a same length, such that the starting points of the time intervals increase from one time interval to the next time interval.

More particular, consider an audio file as a l representation of an analogue audio signal that is sampled with a ermined sampling rate fs given by the number of samples per second. Sampling may be carried out during recording through an analog-to-digitaI-converter, such as an audio interface, for example. In case of digitally produced audio data (for example from digital synthesizers, drum computers etc.), the samples and in particular the audio data represented by each sample, are computer generated values.

Each sample represents the signal value (e.g. a measured average value) within a sampling period T, wherein fs=1/T. For audio files, fs may be 44.1 kHz or 48 kHz, for example. One sample is also referred to as one frame.

Now, in the t embodiment, a starting frame of the first segment may be the very first frame of the audio data in the audio file at a time position 0, the starting frame of the second t may be the frame immediately following the end frame of the first t, the starting frame of the third t may be the frame immediately following the end frame of the second segment and so on. The segments may all have the same size with ect to their time scale of the playable audio signal or may have the same ber of frames, except for the last segment, which may have an end point [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman d by the end point or the last frame of the (decoded) audio file or the end point of the playable audio signal on the time scale.

In fact, in methods and devices of the present invention, processing and in particular decomposition is preferably carried out on the basis of segments exactly defined by and/or corresponding to the frames of the input audio file, which ensures frame accurate positioning within the tracks, in particular within the decomposed tracks during recombining or playback, and direct translation of audio positions in the mixed input signal to audio positions in the osed track. A decomposed track obtained in this manner may therefore have exactly the same time scale as the mixed input track and can be further processed, for example by applying effects, resampling, time hing, and seeking, eg. for tempo and beat matching, without shift or loss in accuracy on the time scale. Preferably, a decomposed t contains exactly the same amount of frames as the original input audio data corresponding to the t.

Preferably, the size of the segments is chosen such that the length of the corresponding time intervals is smaller than 60 s and larger than one second. This ensures sufficient segmentation of the input audio file to achieve able acceleration of the processing necessary to start playing from any given position. More preferably, the ts have a size corresponding to time intervals having a length which is between 5 seconds and 20 seconds. This ensures sufficient audio data for the Al systems 44 to e satisfying decomposition results on the one hand and reduce the audio data to be decomposed in one segment to a value small enough to achieve virtually immediate availability of the decomposed audio data to allow application of the device in a live performing ion.

In the output of the first tation unit 42 a segment of the input audio file g provided to be transmitted to the at least one Al system 44. Preferably,segment is doubled or copied to be transmitted to the first Al system 44-1 [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman and, at the same time, i.e. in el, to the second Al system 44-2. One and the same segment of the input audio file A can therefore be processed at the same time in the first Al system 44-1 as well as in the second Al system 44- Each of the Al systems used in the embodiments of the t invention may be trained artificial neural networks (trained ANN) as described above in this disclosure. In particular, a d ANN as described by Prétet et al. could be used which is able to extract a first decomposed track representing a vocal track or a singing voice track from the mixed audio data. In particular, the Al systems 44 may calculate a Fourier transformation of the audio data (i.e. of the audio data contained in a segment of the input audio file) such as to obtain a spectrum of the ncies contained in the audio data, wherein the spectrum is then introduced into the convolutional neural network which filters parts of the spectrum recognized as belonging to a certain source track or the sum of certain source tracks, for example belonging to the vocal part of the mix. The ed spectrum is then retransformed into a waveform signal or audio signal which, when played back, contains only the filtered part of the original audio signal, for e the vocal part.

To be capable of this filtering analysis, an Al system such as an ANN may be used as described by Prétet et al. for example, which was trained by data sets containing large s of professionally recorded or produced songs from different genres, for example Hip Hop, Pop, Rock, Country, Electronic Dance Music etc., wherein said data sets do not only include the finished songs but also the respective vocal and instrumental tracks as separate recordings.

Stored within the first decomposition unit 38 of device 10 of the first embodiment rably within a RAM memory thereof, especially the al nil/stemsof the computer 35) may be two separate and fully trained instances of(different or equal Al systems) of the above-mentioned type such [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman as to be operable simultaneously and independent from one another to te a first decomposed track and a second decomposed track, tively. Preferably, first and second osed tracks are ments, which means that the sum of the first decomposed track and the second decomposed track, when recombined with normal volume levels (i.e. each at 100 percent), les the original mixed input data. For example, the first decomposed track may resemble the complete vocal part of the mixed input data, s the second decomposed track may le the complete remainder of the mixed input data, in ular the sum of all instrumental tracks, such that recombining both decomposed tracks at appropriate volume levels results in an audio signal that, in terms of its acoustic perception, very closely resembles or cannot even be distinguished from the original mixed input data.

Preferably, the first and/or second decomposed track are each stereo tracks containing a left-channel signal portion and a right-channel signal portion, respectively. Alternatively they may each or both be mono tracks or multi- channel tracks with more than two channels (such as 5.1 surround tracks, for example).

The second decomposition unit 40 may be configured in a manner similar or corresponding to that of the first composition unit 38, thus including the second segmentation unit 46 which partitions the second input audio file B into a number of segments of fixed starting points and end points, transmitting the segments utively to both a third Al system and a fourth Al system for parallel processing and decomposition to obtain a third decomposed track and a fourth decomposed track (each of which may be mono tracks, stereo tracks, or multi-channel tracks with more than two channels (such as 5.1 surround tracks, for example)).

H decomposed tracks from the first and second decomposition units 3840 are then transmitted to the recombination unit 32 which is configured [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman to recombine at least two of the decomposed tracks at specified and controllable volume levels and to generate recombined output data. The volume levels of the decomposed tracks may be controlled by a user by virtue of at least one control element. For example, a first control element 26- 1 may be provided which allows a user to control a ratio between a first volume level of the first decomposed track and a second volume level of the second osed track, whereas, alternatively or in addition, a second control element 26-2 may be ed which allows a user to control a ratio between a third volume level of the third decomposed track and a fourth volume level of the fourth decomposed track.

In the recombination unit 32 the first and second decomposed tracks are then recombined with one another in a first recombination stage 32-1 based on the volume levels set by the first control element 26-1 to obtain a recombination A’ from the first input audio file A. r, the third and fourth decomposed tracks may be recombined in a second ination stage 32- 2 of the recombination unit 32 according to the third and fourth volume levels set by the second control element 26-2 such as to obtain a second recombination B’ from the second input audio file B. Furthermore, recombination A’ and recombination B’ may be introduced into a mixing stage 48 which mixes the first recombination A’ and second recombination B’ according to the setting of the mix control element 28 controllable by the user. The mix control element 28 may be adapted to control a ratio between the volume levels of the first and second recombinations A’ and B’.

The recombined output data generated by the recombination unit 32 is then itted to a playing unit which may include audio interface 34 connected to audio output 36.

As can be seen in Fig. 2, the first and second decomposed tracks as output Ehe first decomposition unit 38 may be input into a first visualization unit- . In on, the third and fourth decomposed tracks as output by the [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman second decomposition unit 40 may be input into a second visualization unit 49-2. Moreover, first and/or second visualization units 49-1 and 49-2 may be connected to the recombination unit 32 to obtain information about the current settings of the control elements 26-1, 26-2, for example. First and/or second visualization units 49-1 and 49-2 are preferably configured to y an overlay waveform of recombination A’ and ination B’, respectively, as will be explained in more detail later on.

Processing of the audio data within device 10 of the first embodiment of the invention is further illustrated with respect to Fig. 3, which shows the processing of only the first input audio file A as an example, which can be applied to the processing of the second input audio file B, or any additional other input audio file, in the same manner. As can be seen in Fig. 3, after the osition process in the processing section 14, segments of the first and second osed tracks are stored in an audio buffer (for example a ring buffer) for immediate further processing and in particular for playback, preferably real time ck. The audio buffer has multiple data arrays in order to store audio data from the current segment of the first decomposed track as well as audio data from the current segment of the second decomposed track, each with the given number of channels (Mono, Stereo, Surround, etc.). For example, if both decomposed tracks represent stereo signals, a four-array buffer may be used in order to store left and right channel portions of the first and the second decomposed track segments, respectively.

Output of the buffer may be connected to the recombination unit 32 which generates a recombined track according to the g of the first control element 26-1.

If the device 10 includes one or more audio effect chains to apply audio gets to the signals, such as delay effects, reverb effects, equalizer effects,or tempo changing effects, for example achieved by pitch-shifting, [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ling and/or time stretching s, etc. as conventionally known as such for DJ equipment, such effect chains could be inserted at ent positions in the signal flow. For example, the decomposed tracks (segments) output by the buffer may each be routed through audio effect chains 51-1 and 51-2, respectively, such as to apply effects individually to the respective decomposed track as d. The output of the audio effect chains 51 -1, 51- 2 may then be connected to the recombination unit 32. In addition or as an alternative, an effect chain 51-3 could be arranged at a on with respect to the signal flow at which the first and second decomposed tracks are recombined in accordance with the first and second volume levels set by the first l t 26-1, in particular at a position after the recombination unit 32 or after the first recombination stage 32-1 of recombination unit 32.

The advantage of this arrangement is that the number of channels to be submitted to the audio effect chain 51-3 is reduced within the recombination process to at least one half of the number of channels before the first recombination stage and is in particular equal to the number of channels of the first mixed input data (one channel for a mono , two channels for a stereo signal, more than two channels for other formats such as surround signals). Thus, the additional functionality of the decomposition units of the present embodiment will not bring about any increased complexity or performance overload of the audio effect chain 51-3 as compared to the conventional processing of the mixed input data. The same audio effect chains as for conventional DJ equipment may even be used.

With reference to Figs. 4 to 10, second to eighth embodiments are explained below. Each embodiment is a modification of the first embodiment described above with respect to Figs. 1 to 3 and all features and functions described above for the first embodiment are preferably included in the same corresponding manner in each of the second to eighth embodiments unless described differently in the following. These same or corresponding features ?unctions will not be described again.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman In the second embodiment rated in Fig. 4, a first DJ deck 50a and a second DJ deck 50b are displayed on a display, in particular a touch display which allows a user to operate them by means of gestures or movements corresponding to the ion of physical DJ decks. The second embodiment may in particular be advantageous to allow a user, in particular a DJ, to m scratching effects during live performance or to skip to different time positions in a song.

As a further feature of the second embodiment, which may be provided independent from (in addition or alternatively to) the DJ decks 50a, 50b, the first control t 26-1, and preferably also the second control element 26- 2, may be ed as s, either as hardware sliders mechanically movable by a user, or by virtual sliders presented on a touch screen or on a computer screen movable by a touch gesture or by a pointer, a er mouse or any other user input. The slider of the first control element 26-1 allows continuous variation of the ratio between the first volume level of the first decomposed track and the second volume level of the second decomposed track in a range from one end position at which the first volume level is set to 100 % and the second volume level is set to 0 % to another end position at which the first volume level is set to 0 % and the second volume level is set to 100 %. Between the end positions, when moving the slider in one direction, one of the first and second volume levels is increased, while the other one of the first and second volume levels is decreased at the same proportion.

As a red default g, at a center position of control element 26-1, both first and second volume levels are set to full/normal volume = 100%, Le. the recombination corresponds to the original first mixed input data. The volume adjustment curve can be user configurable though if needed. By default the volume levels may be calculated as follows: first volume level = M|N(1.0, sliderValue*2.0), [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman second volume level = MIN(1.0, (1.0 - sliderValue)*2.0), wherein "MIN (value 1, value2)" ents a minimum value of value 1 and value 2, "sliderValue" represents a setting of control element 26-1 running from 0 (left end value) to 1.0 (right end value). Increasing and decreasing of the volume levels is reversed when moving the slider in the other direction.

The user will thus be able to smoothly crossfade between the first osed track and the second decomposed track or adjust a desired recombination between both decomposed tracks by a single continuous movement with only one hand or even only one finger. ably, the second control element 26-2 is operable in the same manner as the first control element 26-1 to control the third and fourth volume levels of the third and fourth decomposed tracks, respectively.

Preferably, the mix control element 28 is also realized as a slider and may be positioned n the first and second control elements 26-1, 26-2 for intuitive operation of the device. As in the first embodiment, the mix control element 28 may be a crossfader and/or may be d to control a ratio between the volume levels of the first and second recombinations A’ and B’, wherein recombination A’ is obtained from recombining the first decomposed track and the second decomposed track, and recombination B’ is ed from recombining the third decomposed track and the fourth osed track.

Device 10 may further be configured to display a first waveform section 52-1 in which waveforms representing the first and second decomposed tracks or a recombination thereof are displayed. First and second decomposed tracks may be visualized in an overlaid manner such as to share a common baseline/time axis, but using different signal axes and/or different drawing styles so as to be visually distinguishable from one r. In the example gwn in figure 4, the first waveform section 52-1 displays a zoom-in n- of the first and second waveforms, in which first and second waveforms ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman are displayed in an overlaid manner using a common baseline that is scaled to view a time interval containing the current play position and preferably having a size between 1 second and 60 s, more preferably n 3 seconds and 10 seconds. The zoom-in version 53-1 may scroll along with the playback such as to maintain a current playing position visible, in particular at a fixed position on the display. In addition or alternatively, the first waveform section 52-1 may display a zoom-out version 55-1 of the first and second waveforms, in which first and second waveforms are displayed in an overlaid manner using a common baseline that is scaled to view a time interval containing the current play position and preferably having a size corresponding to the length of an input audio file, for example the whole song A and/or a size between 60 seconds and 20 minutes. Preferably, the zoom- out version 55-1 does not move with respect to the time axis, but rather shows a play head 58 representing the current playing position, which moves along the time axis.

Likewise, device 10 may be configured to display a second waveform section 52-2 in which rms representing the third and fourth decomposed tracks are displayed in the same manner as described above for the first waveform section 52-1 and the first and second decomposed tracks, in particular by means of a zoom-in n 53-2 and a zoom-out version 55-2.

First and/or second waveform sections 52-1, 52-2 may be configured to receive user input commands such as touch gestures or mouse/pointer input ds in order to change the current playing position and to jump to a d position within the audio data, for example by simple clicking or touching the desired position on the ne in the zoom-out version 55- 1/55-2.

In the example of Fig. 4, the first and second decomposed tracks of the gem-in version 53-1 of the first waveform section 52-1 are displayed usingrent signal axis and different drawings styles. In particular, the signal axis [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman of the first osed track, for example the decomposed vocal track, is scaled icantly smaller than that of the second decomposed track, for example the osed instrumental track, such that the first decomposed track is ized as lying within the second decomposed track and thus being visually distinguishable. Furthermore the waveform of the first decomposed track is displayed with a drawing style using a dark color, whereas the waveform of the second decomposed track is displayed with a drawing style using a lighter color.

Likewise the first and second decomposed tracks of the ut version 55- 1 of the first waveform section 52-1 are y using ent drawings styles. In ular, only an upper half of the waveform of the first decomposed track and only a lower half of the waveform of the second osed track are displayed. Furthermore the waveform of the first decomposed track may be displayed with a drawing style using a dark color, whereas the waveform of the second decomposed track may be displayed with a drawing style using a lighter color. Of , all these drawing styles could be interchanged or modified and/or applied to the waveforms of the second waveform section 52-2.

The overlaid representations of the decomposed tracks in the first and second rm sections 52-1, 52-2 may be ed by a method according an embodiment of the invention, which will be described in more detail below with respect to Fig. 12 and 13.

Furthermore, settings of the control elements 26-1, 26-2, 28 and 30-1, 30-2 may be reflected in the visualization of the decomposed tracks in the first and second waveform sections 52-1, 52-2 through respective signal amplitude changes of the individual waveforms displayed. In particular, the signal axes of the waveforms of the decomposed tracks as displayed in the first and gingsond waveform sections 52-1, 52-2 are scaled depending on the currentof the volume levels of the respective decomposed tracks as set by ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman the user through the l elements 26-1, 26-2, 28 and 30-1, 30-2. This allows direct and preferably immediate visual feedback of the volume settings to the user.

Device 10 may have a first cue control element 31-1 and/or a second cue control element 31-2, associated to the first and second mixed input files (songs A and B), respectively, which can be operated by a user to store a current playing position and to retrieve and jump to it at any point in time later as desired.

In the third embodiment illustrated in Fig. 5, first and second control elements 26-1, 26-2 are similar in function to the respective control elements in the second embodiment except that they are rotatable knobs instead of sliders.

However, the knobs can also be rotated between two end ons in which one of the first and second volume levels is set to 100 % whereas the other one of the first and second volume levels is set to 0 %. Again, the user may ade between the first and second decomposed tracks by means of a single continuous movement using only one hand or only one finger. The same configuration may be implemented for the second control t 26- 2.

Fig. 6 illustrates a fourth embodiment of the t invention which uses a different controlling section to control the recombination unit. In particular, instead of or in addition to the first and second control ts 26-1, 26-2 as described for the first to third embodiments, in the fourth embodiment there is provided a third control element 26-3 which controls a ratio between the first volume level of the first decomposed track and the third volume level of the third decomposed track, in other words, volume levels of decomposed tracks of different decomposition units 38, 40. It furthermore may comprise a fourth control element 26-4 which allows a user to control a ratio between the gond volume level of the second decomposed track and the fourth volume| of the fourth decomposed track. By means of these control elements [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman 26-3, 26-4 it will be possible, for example to easily and directly l, by means of a single movement with one hand or one , a ratio between a vocal part of the first audio file and a vocal part of the second audio file by manipulating the third l element 26-3. Likewise, by lating the fourth control element 26-4 in a single movement by only one hand or only one finger, a user may control a ratio between the volume level of the instrumental part of the first audio file and the instrumental part of the second audio file. This allows a DJ for example to make an even more seamless transition by first cross fading the vocal track from song A to song B and subsequently cross fading the instrumental track from song A to song B, thus achieving a more continuous flow of the music.

The third control element 26-3 and/or fourth control element 26-4 may be implemented as sliders (hardware slider or software user interface, e.g. virtual touch screen sliders) or as rotatable knobs (likewise as hardware knobs or virtual knobs on a touch screen, computer screen or any other display device).

In the first to fourth embodiments described above, device 10 was preferably realized as an all-in one device including input section 12, processing section 14, recombination unit 32, playing unit (in particular audio interface 34 (eg. sound card) and audio output 36), in one single housing or, alternatively, as a complete virtual equipment realized as a software running on an electronic control unit (ECU) with the control elements being visualized on a display of the ECU and the electronic components of the sing section 14 being ed by the integrated electronic components of the ECU. Such ECU may be a standard personal computer, a multi-purpose computing device, a laptop computer, a tablet computer, a smartphone or an integrated, standalone DJ ller. gs r alternative, according to a fifth embodiment shown in Fig. 7,ice 10 may be implemented as a combination of a er 54 (personal [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman computer, laptop er, tablet or smartphone or other multi-purpose computing device) and a periphery device 56 which is an external hardware component that can be connected to the computer by cable (such as USB connection, MIDI connection, HID connection, fire wire tion, LAN connection etc.) or by any ss connection using the usual wireless protocols (WlFl, GSM, Bluetooth etc.). Preferably, the periphery device 56 includes the recompose controlling section 24 with the control elements such as control ts 26-1, 26-2 and 28. Furthermore, the periphery device 56 may include Jog wheels 50a, 50b or other features known from conventional DJ ent. The conventional hardware of the computer 54 may be used as the processing n 14, in particular to store and run the Al systems and the segmentation units in the RAM memory of the computer 54.

Furthermore, a sor/CPU may also be included in the peripheral device 56 to perform some or all of the tasks of the sing section 14.

A sixth ment of the present invention as shown in Fig. 8 is a slight modification of the fifth embodiment, wherein the periphery device 56 of the sixth embodiment is relatively compact and just includes the recompose controlling section and the control elements in order to reduce the additional hardware required to carry out the present invention to a minimum and still provide for mechanical control elements.

In a seventh embodiment shown in Fig. 9, the device 10 comprises a song-A instrumental button 26-5 controllable by the user to switch ON or OFF the decomposed instrumental track of song A, and/or a song-A vocal button 26-6 controllable by the user to switch ON or OFF the decomposed vocal track of song A, and/or a song-B instrumental button 26-7 controllable by the user to switch ON or OFF the decomposed instrumental track of song B, and/or a song-B vocal button 26-8 controllable by the user to switch ON or OFF the decomposed vocal track of song B. By realizing some or all of these buttons ?eto 26-8 as separate buttons, the user can individually and by only ration (one tap with the finger) switch ON or OFF a selected one of ation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman the decomposed tracks. Note that in the present description, switching ON and OFF a track refers to unmuting and muting the track, tively.

Preferably, upon operation of one of the buttons 26-5 to 26-8 by the user, the respective decomposed track is not switched ON or OFF immediately, but the device is controlled to uously or se increase or decrease the volume of the respective track within a certain time period of preferably more than 5 milliseconds or even more than 50 milliseconds, such as to avoid acoustic artefacts arising from instant signal transitions.

In an eighth embodiment shown in Fig. 10, the device 10 may comprise a first recombination stage ured to obtain a first recombination A’ by recombining the decomposed vocal track of song A with the decomposed instrumental track of song A, and a second recombination stage configured to obtain a second recombination B’ by recombining the decomposed vocal track of song B with the decomposed instrumental track of song B.

Furthermore, device 10 may comprise a mix control element 28 configured such as to be operable by a user in a first direction to increase a volume level of the first recombination A’ or in a second direction to increase a volume level of the second recombination B’. In addition, there is preferably provided a mixing stage which mixes the first and second recombinations A’ and B’ with one another according to their tive volume levels to obtain the recombined output track. Such signal flow is similar to that explained above with reference to figure 2.

Now, in the eighth embodiment, the device 10 may further include a vocal swap button 26-9 controllable by the user, in particular through one single ion such as simply pushing the button, to route the osed vocal track of song A to the second recombination stage and to route the decomposed vocal track of song B to the first recombination stage. In other galds, operation of the vocal swap button 26-9 swaps the two decomposedtracks of songs A and B before they enter the first and second [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ination stages, respectively. Repeated operation of the vocal swap button 26-9 may again swap the two decomposed vocal tracks and so on.

In addition or alternatively, the device 10 may include an instrumental swap button 26-10 llable by the user, in particular through one single operation such as simply pushing the button, to route the decomposed instrumental track of song A to the second recombination stage and to route the decomposed instrumental track of song B to the first ination stage. In other words, operation of the instrumental swap button 26-10 swaps the two decomposed instrumental tracks of songs A and B before they enter the first and second recombination stages, respectively. Repeated ion of the instrumental swap button 26-10 may again swap the two decomposed instrumental tracks and so on.

Preferably, upon operation of one of the buttons 26-9 or 26-10 by the user, the tive swapping of the tracks will not be immediate, but the device is controlled to continuously or stepwise increase or decrease the respective volumes of the tracks within a certain time period of preferably more than 5 milliseconds or even more than 50 milliseconds, such as to avoid acoustic artefacts arising from instant signal transitions. atively the vocal swap button 26-9 can be lled by the user to achieve a similar remix/mashup by obtaining a first recombination A’ by recombining the decomposed vocal track of song A at normal volume (in particular maximum volume) with the muted decomposed instrumental track of song A, and by obtaining a second ination B’ by recombining the muted decomposed vocal track of song B with the decomposed instrumental track of song B at normal volume (in particular maximum volume), while setting the mix control element 28 to its center position such as to have inations A’ and B’ both audible at the same ?lme levels and at the same time.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman Fig. 11 shows a modification of the method of the eighth embodiment, especially as regards the operation of the swap buttons, for example the vocal swap button 26-9. Device 10 receives a track A (song A) as a master track and track B (song B) as a slave track. Track A is decomposed as described above to obtain decomposed tracks 1 and 2, whereas track B is decomposed as described above to obtain decomposed tracks 3 and 4, respectively. In order to prepare osed track 3 for the swap, its key, tempo and beat phase will be matched to that of the master track A. In particular, the device 10 determines a tempo (e.g. a BPM value (beats per minutes» of track A and track B and if they don’t match, decomposed track 3 will be resampled or time-stretched such as to match the tempo of the master track A. In addition, key matching will be carried out and the key of decomposed track 3 will be d, if necessary, such as to match that of the master track A. er, after tempo matching of decomposed track 3, the beat phase of decomposed track 3 is shifted in a synchronization step as ary, such as to match the beat phase of track A.

As a result, device 10 prepares a modified decomposed track 3’ which matches track A as regards tempo, beat phase and key such that it can be seamlessly ined with decomposed track 2 of track A. If the swap button is activated, as can be seen in Fig. 11, in the following processing of track A, decomposed track 3’ will be used instead of decomposed track 1 and will be routed to the recombination stage for recombination with osed track 2 and audio output.

Optionally, one or more audio effect chains may be ed in the signal flow of any of the tracks, for example between the swapping step and the recombination stage such as to be applied to the respective decomposed tracks 1, 2 or 3’, for example.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Fig. 12 and 13 show graphical representations of audio data which may be displayed on a display device in a method or device of an embodiment of the present invention, in particular in a device according to one of the first to eighth embodiments described above, during operation of the device. In particular, the graphical representation could be yed on a display of the ECU, in particular a er screen or on an integrated display of a separate peripheral device connected to a computer or as a standalone device, on a tablet, smartphone or a similar . The graphical representation may be generated by le software which runs on the ECU (i.e. the computer, the standalone device, the tablet, the smartphone etc.) and which may be part of the software that carries out a method according to the present invention as described in the claims or in the embodiments above. The software may e a c interface, such as a graphic card. ing to the embodiment, audio data are visualized as waveforms.

Waveforms in this sense are representations having a linear time axis t which represents the playback time (usually a horizontal axis), and a signal axis (orthogonal to the time axis t, preferably a vertical axis), which represents an average signal strength or a signal amplitude of the audio data at each specific playback time. A playhead 58 may be provided which tes the current playing position. During playback of the audio data, the playhead 58 is moving with t to the waveform along the time axis t by visually moving either the waveform or the ad or both.

Figure 12 schematically shows the processing steps to arrive at the novel graphical entation of the invention. Mixed input data 60, for example song A, is received and decomposed to obtain first decomposed track 61 -1, for example a decomposed vocal track, and second decomposed track 61 -2, for example a decomposed instrumental track. First and second osed gespondsks 61-1 and 61-2 may be complementary tracks such that their sumto the mixed input data 60.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] a.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman Actually displayed is then an overlay waveform 64 which is an overlaid representation of the first and second decomposed tracks 61-1, 61-2 using one single baseline for the waveforms of both decomposed , which means that the time axes t of both waveforms are not running el to each other in a distance but are identical to form one common line. In order to allow a differentiation between both waveforms, they are displayed using different drawing styles. For example one of the two waveforms of the decomposed tracks may be displayed in a different color than the other waveform. In the example shown in Fig. 12, for one of the waveforms of the decomposed tracks, here the decomposed vocal track 61-1, only positive signal portions are displayed, while ve signal portions are left out, whereas for the waveform of the other of the decomposed tracks, here the decomposed instrumental track 61-2, only negative signal ns are displayed, while positive signal portions are left out. Alternatively the waveforms could be drawn using differently scaled signal axes or by using different g styles such as to allow the waveforms to be distinguished from one another. As an e of different drawing styles, one of the waveforms could be drawn as a dashed or a dotted line, or of different color, or of different opacity or transparency or any combination thereof.

In another example shown in Fig. 13 one of the waveforms of the decomposed tracks, here the waveform of the decomposed vocal track 61 -1, is displayed with a signal axes scaled differently, here smaller, than that of the waveform of the other decomposed track, here the decomposed instrumental track 61 -2. In addition the waveforms may be displayed with different colors.

Waveforms of decomposed tracks are preferably displayed such as to ent the settings of the control elements of the ose controlling tion and/or the gs of the recombination unit such as to provide a back to the user about the signal volumes assigned to the respective [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman decomposed . Preferably, at the same time as a user is manipulating one of the control elements to increase or decrease the volume of at least one decomposed track, the associated waveform of this osed track is displayed with an increasing or decreasing size with regard to its signal axis, or visually faded in or out. This graphical feedback is ably immediate, thus with a delay time which is not disturbing or even not recognizable for the user, in particular a delay time below 500 milliseconds, preferably below 35 milliseconds such that it is not noticeable to the eye at a frame rate of 30 frames per second. Such display greatly s operation of the device during live performance.

Fig. 14 shows a ninth embodiment of a device 10 of the present invention, which is an audio player including a recompose controlling n 24 having a control element 26-13 for controlling the first and second volume levels of respective first and second decomposed tracks (here decomposed vocal track and decomposed instrumental track) obtained from one audio file, and optionally a display region 66 displaying an overlaid representation of the first and second decomposed tracks. The device 10 of Fig. 14 may be adapted to play audio files one after another, for example from a playlist or based on individual user selection, and might have an input unit for receiving audio files via streaming from an audio streaming service, and may thus be adapted to play only one audio file at most of the time (apart from al ading s at a transition from the end of one song to the beginning of the next song). The user can start or stop playback by operation of a play control element 30 and/or can change the playback position by moving the playhead along the time axis.

Through the control element 26-13 the user may control playback of a song such as to hear only the decomposed vocal track or only the decomposed instrumental track or a ination of both tracks. Such configuration th be useful for a karaoke ation or a play-along application, formple. Preferably, device 10 is a computer or a mobile device, such as a [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman W0 75456 smartphone or tablet, which runs a suitable software application to realize the above-described functionalities.

Fig. 15 shows a tenth embodiment of the t invention which comprises separate ON-OFF buttons 26-14 to 26-17 for each of the first to fourth decomposed tracks, in particular the first decomposed vocal track, the first decomposed instrumental track, the second decomposed vocal track and the second decomposed instrumental track, respectively. By operating one of the buttons, the volume of the respective decomposed track is switched between 0 and 100 percent or vice versa.

Fig. 16 shows an eleventh embodiment of the present invention which comprises te faders 26-18 to 26-21 for each of the first to fourth decomposed tracks, in particular the first osed vocal track, the first decomposed instrumental track, the second osed vocal track and the second decomposed instrumental track, respectively. By operating one of the faders, the volume of the respective decomposed track are continuously changed between 0 and 100 percent or vice versa.

A twelfth embodiment of the present invention will be bed in the following with reference to figures 17 and 18. The twelfth embodiment is a cation of the first to eleventh embodiments and may therefore comprise any or all of the above-mentioned features and advantages of any of the first to eleventh embodiments unless otherwise described in the following.

A device 110 of the twelfth embodiment comprises an input unit with a first input section to receive and/or provide an input audio file A, for e a first song A, and preferably a second input section for receiving or providing a second input audio file B, for e a second song B. The first input audio file may be decoded or decompressed if provided in encoded or compressed ?tat, and may be partitioned into ts in a first segmentation unit 142 [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman W0 75456 in a same or corresponding manner as bed above for the first embodiment.

The input audio file A (or its segments) are then transferred to a first Al system 144 capable of separating the audio data into at least four decomposed tracks, i. e. a drum track D1, a bass track D2, a vocal track D3, and a complement track D4. The drum track D1 contains ents of the input audio file A which have a drum timbre, the bass track D2 contains components of the input audio file A that have a bass timbre, the vocal track D3 contains components of the input audio file A that have a vocal timbre and the complement track D4 is a remainder of the input audio file A, which means that a mixture of the drum track D1, the bass track D2, the vocal track D3, and the complement track D4 will result in an audio signal substantially equal to that of the input audio file A. In modifications of this embodiment, the Al system 144 may be configured and trained to separate from the input audio file A decomposed tracks D1 to D3 of any other timbres.

Decomposed tracks D1 to D4 are routed to recombination unit 132 which is configured to recombine selected tracks out of the decomposed tracks D1 to D4 according to user settings and/or a user control input. In particular, recombination unit 132 may comprise a first recombination section 132a which receives the individual decomposed tracks D1 to D4 as an input and outputs two tracks S1, obtained from g through one of the decomposed tracks D1 to D4, and 82, obtained from grouping selected tracks out of D1 to D4. The selection of decomposed tracks and the respective grouping of the decomposed tracks D1 to D4 may be controlled by a mode l unit 145.

In the example shown in figures 17 and 18, mode control unit 145 may selectively be set into a first operational mode shown in figure 17 or a second gombination n 132a is configured such that the drum track D1rational mode shown in figure 18. In the first operational mode, the firstis [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman routed to the first track S1, i.e. S1 equals D1, whereas bass track D2, vocal track D3, and complement track D4 are selected and recombined into a single track, i. e. the second track 82. In other words, in the first operational mode D2, D3, and D4 are grouped to form a single track 82 and D1 is passed h such as to form track S1. On the other hand, in the second operational mode shown in figure 18, the first recombination section 132a is configured such that drum track D1, bass track D2, and complement track D4 are selected for a recombination, i. e. grouped to form a single track 82, whereas vocal track D3 is routed to track S1 alone.

Mode control unit 145 may comprise a mode control element (such as a genre button or genre switch) to be operated by a user to selectively switch between the first ional mode and the second ional mode. The first operational mode may for e be used primarily for electronic music (i.e. usually without ), while the second operational mode may be used for music usually containing vocals such as Hip-hop or Pop.

Tracks S1 and S2 are then routed to a second recombination section 132b which contains a single control element 126-1 llable by a user to control a first volume level to be associated to the first track 81 and a second volume level to be associated to the second track 82. Preferably, control element 126-1 is operable by a user in a single control operation, for example as a crossfader between the first volume level and the second volume level, i.e. such as to change a ratio between the first and second volume levels. In ular, the single control t 126-1 may be configured to have a l range, wherein at least in part of the control range volume changes of the first and second volume levels are performed simultaneously, for example by sing one of the first and second volume levels, and/or decreasing the other of the first and second volume levels. Preferably, the single control element 126-1 may have a control range extending from a first gondpoint at which the first volume level has a maximum value and thevolume has a minimum value, to a second end point at which the first [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman volume level has a minimum value and the second volume level has a maximum value. In the middle region of the control range the first and second volume levels may both have a maximum value.

Preferably, the single control element 126-1 is a single ble knob or a single fader element. Based on the gs of the first and second volume levels as input by the user through the control element 126-1, the second recombination section 132b recombines the first track S1 and the second track 32 in order to obtain a second recombined track A’ routed towards an audio interface 134 for ck.

Tracks S1 and 82 may further be routed to a visualization unit 149-1 for visualization of their waveforms on a display or the like, as described above for the visualization units 49-1 and 49-2 in the previous embodiments.

The second audio input file B may be processed in a similar manner as the first input audio file A, for e in a second decomposition unit 140 which may se a second Al-system. Decomposed tracks obtained from the second decomposition unit 140 may then be routed through the recombination unit 132 and recombined therein in groups or individually in the same or corresponding manner as described above for the first input audio file A. A recombined track 3’ ed in this manner from the second input audio file B may then be recombined/mixed with the recombined track A’ obtained from first input audio file A, in ular within a further mixing stage lled by a mix control element 128 in the manner described in more detail above for the first to eleventh embodiments. The output of this mixing stage may then be routed to audio interface 134 for playback.

Aspects and embodiments of the present invention can r be described by the following items: [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman Item 1: Method for processing and playing audio data, comprising the steps a) receiving mixed input data, said mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track, b) decomposing the mixed input data to obtain at least a first osed track resembling the at least one first source track, c) ting output data based on the first decomposed track, d) playing the output data through an audio output.

Item 2: Method of item 1, further comprising the following steps: - reading a control input from a user, said control input representing a desired setting of a first volume level of the first decomposed track and a second volume level of a second track, and - recombining at least the first decomposed track at the first volume level with the second track at the second volume level to te recombined output data, - g the recombined output data.

Item 3: Method of item 2, wherein the second track is obtained in the step of decomposing the mixed input data and forms a second decomposed track resembling the at least one second source track.

Item 4: Method of at least one of the preceding items, wherein decomposing the mixed input data is carried out segment-wise, n decomposing is carried out based on a first segment of the mixed input data such as to obtain a first segment of output data, and wherein decomposing of a second segment of the mixed input data is carried out while playing the first segment of output data.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Item 5: Method of at least one of the preceding items, wherein the method steps, in particular steps (a) to (d), are d out in a continuous process.

Item 6: Method of at least one of the preceding items, n the mixed input data are received via ing from a remote server, preferably through the internet.

Item 7: Method of at least one of the preceding items, wherein in step (a) an input audio file having a predetermined file size and a predetermined playback duration is received, which ns audio data to play the mixed input data, and a first segment is extracted from the input audio file, which contains audio data to play the mixed input data within a first time al smaller than the predetermined playback duration, wherein in step (b) the first segment of the input audio file is decomposed to obtain a first t of the first decomposed track and optionally a first segment of the second decomposed track, wherein in step (c) a first segment of the output data is generated from the first segment of the first decomposed track, preferably by recombining at least the first segment of the first decomposed track at the first volume level with the first t of the second decomposed track at the second volume level, and wherein the method further comprises the steps of: a2. extracting a second segment from the input audio file, which is different from the first segment and which ns audio data to play the mixed input data within a second time interval smaller than the predetermined playback duration of the input audio file and shifted in time with respect to the first time interval, b2. decomposing the second segment of the input audio file to obtain a second segment of the first decomposed track and optionally a second segment of the second decomposed track, optionally c2. recombining at least the second segment of the decomposed track at the first volume level with the second segment of [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman the second decomposed track at the second volume level to generate a second segment of the ined output data, wherein at least one of the steps (a2), (b2) and (c2) is performed while playing the first segment of the output data, and wherein generation of the second segment of the output data is completed before playing the first segment of the output data is completed.

Item 8: Method of at least one of items 4 to 7, wherein the size of the first segment or the length of the first time interval is set such that the time ed for osing the first segment is smaller than 2 seconds, preferably r than 150 milliseconds, most preferably smaller than 50 milliseconds.

Item 9: Method of at least one of the preceding items, comprising the steps of receiving an input audio file having a predetermined file size and a predetermined playback duration, which contains audio data to play the mixed input data, partitioning the input audio file into a plurality of segments in succession, which contain audio data to play the mixed input data within a plurality of time intervals following each other, receiving a play position command from a user representing a user’s command to play the input audio file from a n start play on, identifying a first segment out of the plurality of segments such that the start play position is within the time interval which corresponds to the first segment, decomposing the first t of the input audio file to obtain a first segment of the first decomposed track and optionally a first segment of the second decomposed track, generating a first segment of the output data based on the first segment of the first decomposed track, preferably by ining at least the first t of the first decomposed track at the first volume level with the D segment of the second decomposed track at the second volume level, [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman g the first segment of the output data starting at the start play position, which is a play position later than or equal to the start of the time interval of the first segment.

Item 10: Method of at least one of the preceding items, wherein the mixed input data are first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track and wherein the method further comprises the steps of - receiving second mixed input data, said second mixed input data being a sum signal obtained from mixing at least one third source track with at least one fourth source track, - decomposing the second mixed input data to obtain a third decomposed track resembling the at least one third source track, and a fourth decomposed track resembling the at least one fourth source track, wherein in the step of reading the control input from a user, said control input ents a desired setting of the first volume level of the first decomposed track, the second volume level of the second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth osed track, and wherein, in the step of recombining, the recombined output data is generated by ining the first decomposed track at the first volume level, the second decomposed track at the second volume level, the third osed track at the third volume level and the fourth decomposed track at the fourth volume level.

Item 11: Method of at least one of the preceding items, wherein at least one, preferably all of the mixed input data and the decomposed track signals represent stereo signals, each comprising a left-channel signal portion and a at-channel signal portion, respectively.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Item 12: Method of at least one of the preceding items, wherein decomposing the mixed input data es processing the mixed input data by an Al system, said Al system preferably being trained by a plurality of sets of training audio data, wherein each set of training audio data at least includes a first source track and a mixed track being a sum signal ed from mixing at least the first source track or a track that resembles the first source track, with a second source track.

Item 13: Method of at least one of the preceding items, wherein the mixed input data are processed within a first Al system and a second Al system separate from the first Al system, wherein the first Al system processes the mixed input data to obtain only the first decomposed track and the second Al system processes the mixed input data to obtain only the/a second decomposed track, wherein the method is preferably processing the mixed input data as a first mixed input data and is further processing a second mixed input data within a third Al system separate from the first and the second Al system, and within a fourth Al system te from each of the first to third Al systems, wherein the third Al system processes the second mixed input data to obtain only the third decomposed track and the fourth Al system processes the second mixed input data to obtain only the fourth osed track.

Item 14: Method of at least one of the preceding items, wherein said mixed input data are first mixed input data based on a periodic beat structure and wherein the method further comprises: - receiving second mixed input data ent from the first mixed input data and having a periodic beat signal, - performing at least one of a tempo ng processing and a key matching processing, - wherein the tempo ng processing comprises: D receiving first input data obtained from the first mixed input data and second input data obtained from the [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by a.Chapman second mixed input data, time hing or resembling of at least one of the first input data and the second input data, and outputting first output data and second output data which have mutually matching tempos, - wherein the key matching processing comprises: ing first input data obtained from the first mixed input data and second input data obtained from the second mixed input data, pitch shifting of at least one of the first input data and the second input audio data, and outputting first output data and second output data which have mutually matching keys.

Item 15: Device (10) for processing and playing audio data, preferably DJ equipment, comprising - an audio input unit for receiving mixed input data (A, B), said mixed input data being a sum signal obtained from mixing at least a first source track with at least a second source track, - a decomposing unit (38, 40) connected to the audio input unit for decomposing the mixed input data to obtain at least a first decomposed track resembling the first source track, and - a playing unit (34, 36) for playing output data based on the first decomposed track.

Item 16: Device (10) of item 15, further comprising a recombination unit (32) for recombining at least the first decomposed track with a second track to generate the output data for the playing unit.

Item 17: Device (10) of item 15 or item 16, further comprising a recompose controlling n (24) adapted to be controlled by a user to generate a control input representing a desired g of a first volume level of the first mrecombination unit (32) is configured to recombine at least the firstomposed track and a second volume level of the second track, wherein ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman decomposed track at the first volume level with the second track at the second volume level to generate the output data.

Item 18: Device (10) of at least one of items 15 to 17, wherein the audio input unit is a first audio input unit for receiving first mixed input data (A) being a sum signal obtained from mixing at least a first source track with a second source track, and the osing unit is a first decomposing unit (38), and wherein the device further comprises: - a second audio input unit for receiving second mixed input data (B), said second mixed input data being a sum signal obtained from mixing at least a third source track with a fourth source track, - a second decomposing unit (40) connected to the second audio input unit for decomposing the second mixed input data (B) to obtain a third decomposed track resembling the third source track and a fourth decomposed track resembling the fourth source track, wherein the recompose controlling section (24) is adapted to be controlled by a user to generate a control input representing a desired g of the first volume level of the first decomposed track, the second volume level of the second decomposed track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track, and wherein the recombination unit (32) is adapted to generate the recombined output data by recombining the first decomposed track at the first volume level, the second osed track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level.

Item 19: Device (10) of at least one of items 15 to 18, wherein the recompose controlling section (24) comprises at least one single recompose control Hontrolling the first volume level and the second volume level, preferablyent (26-1, 26-2) which is operable by a user in a single control operation ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by a.Chapman [Annotation] a.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman (1) increasing one of the first volume level and the second volume level, while at the same time decreasing the other one of the first volume level and the second volume level, or (2) changing a ratio between the first volume level and the second volume level from at least a value smaller than 1 to at least a value greater than 1 or vice versa.

Item 20: Device (10) of at least one of items 15 to 19, wherein the ose controlling section (24) comprises - a first single recompose control element (26-1) which is operable by a user in a single control operation for controlling the first volume level and the second volume level, preferably (1) sing one of the first volume level and the second volume level, while at the same time decreasing the other one of the first volume level and the second volume level, or (2) changing a ratio n the first volume level and the second volume level from at least a value smaller than 1 to at least a value greater than 1 or vice versa, and - a single mix control element (28), which is operable by a user in a single control operation for controlling a first sum signal and a second sum signal, preferably (1) increasing one of the first sum signal and the second sum signal, while at the same time decreasing the other one of the first sum signal and the second sum signal, or (2) changing a ratio between a volume level of the first sum signal and a volume level of the second sum signal from at least a value smaller than 1 to at least a value greater than 1 or vice versa, the first sum signal being a sum of the first decomposed track at the first volume level and the second decomposed track at the second volume level and the second sum signal being a sum of the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level, and - preferably a second single ose control element (26-2) which is operable by a user in a single control operation for controlling the third m; volume level and the fourth volume level, and/or decreasing the otherrne level and the fourth volume level, preferably (1) increasing one of the [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman one of the third volume level and the fourth volume level, or (2) changing a ratio between the third volume level and the fourth volume level from at least a value smaller than 1 to at least a value greater than 1 or vice versa.

Item 21: Device (10) of at least one of items 15 to 20, further comprising an input audio file buffer for loading therein segments of an input audio file having a predetermined file size and a predetermined playback duration, which contains audio data to play the mixed input data, a first segment buffer connected to the osing unit to receive and store a segment of the first decomposed track obtained from a segment of the input audio file, a second segment buffer connected to the osing unit to receive and store a segment of the second decomposed track obtained from the same segment of the input audio file, n the playing unit (34, 36) comprises an audio ace having an analog-to-digital converter to generate an analog audio signal from the ined output data, said audio interface having an audio buffer for buffering portions of the output data for playing, wherein the size of the first segment buffer and/or the second segment buffer is larger than the size of the audio buffer of the audio interface, but r than the input audio file data or the predetermined file size of the input audio file.

Item 22: Device (10) of at least one of items 15 to 21, wherein the audio input unit is a first audio input unit for receiving first mixed input data based on a ic beat structure, and wherein the device further comprises: - a second audio input unit for receiving second mixed input data (B) different from the first mixed input data (A) and based on a periodic beat signal, - at least one of a tempo matching unit and a key matching unit, D - wherein the tempo matching unit is arranged to e a first input data obtained from the first mixed input data [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman ation] Rebecca.Chapman Unmarked set by Rebecca.Chapman ation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman W0 75456 and second input data obtained from the second mixed input data, and wherein the tempo matching unit comprises a time stretching unit adapted to time stretch at least one of the first input data and the second input audio data, and to output first output data and second output data which have mutually matching tempos, and/or - wherein the key matching unit is arranged to receive a first input data obtained from the first mixed input data and second input data obtained from the second mixed input data, and wherein the key matching unit comprises a pitch shifting unit adapted to pitch shift at least one of the first input data and the second input audio data, and to output first output data and second output data which have mutually matching keys.

Item 23: Device (10) of at least one of items 15 to 22, wherein all of its components, in particular the audio input unit, the decomposing unit and the playing unit, are integrated within a single unit or within a number of local units connected to one another via a local k or via peripheral cable connections or via nearfield wireless tion.

Item 24: Method for representing audio data, said audio data comprising at least a first track and a second track, which are components of a joint audio mix, said method comprising displaying a first waveform representative of the first track and displaying a second waveform representative of the second track, wherein the first waveform and the second waveform are displayed in an id manner using one single baseline, and n the waveforms are displayed using ent signal axes and/or different drawing styles such as to be visually distinguishable from one another.

[Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman Item 25: Method of item 24, wherein the first waveform is displayed using a first drawing style which draws signal portions of the first rm primarily or exclusively in a positive region relative to a baseline, and the second waveform is displayed using a second drawing style which draws signal portions of the second rm primarily or exclusively in a negative region relative to the same baseline, wherein, preferably, the first waveform is displayed using a first drawing style which draws primarily or exclusively a positive signal n of the first track, and the second waveform is displayed using a second drawing style which draws primarily or exclusively a negative signal portion of the second track.

Item 26: Method of item 24 or item 25, wherein the first and second waveforms are displayed using first and second drawing styles which both draw primarily or ively the positive signal portion or which both draw primarily or exclusively the negative signal portion, and wherein the first waveform is displayed using a first signal axis and the second waveform is displayed using a second signal axis that runs opposite to the first signal axis.

Item 27: Method of at least one of items 24 to 26, wherein the first waveform and/or the second waveform is yed by rendering the waveform in a predetermined time interval with a color that depends on a ncy information of the tive track within the predetermined time interval, said frequency information preferably being indicative of a dominant frequency of the audio data over the predetermined time interval, which is preferably obtained from a frequency is of an audio signal derived from the audio data of the respective track within the predetermined time al.

Item 28: Method of at least one of items 24 to 27 and preferably one of items 1 to 14, comprising the steps of [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman - receiving mixed input data, said mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track, - decomposing the mixed input data to obtain at least a first osed track resembling the at least one first source track, and a second decomposed track resembling the at least one second source track, - reading a control input from a user, said control input representing a desired g of a first volume level of the first osed track and a second volume level of the second decomposed track, - displaying a first waveform representative of the first decomposed track and displaying a second waveform representative of the second osed track, wherein the first waveform and the second waveform are yed in an overlaid manner using one single baseline, and wherein the waveforms are displayed using different signal axes and/or different drawing styles such as to be ly distinguishable from one another, wherein the first rm is displayed with its signal axis being scaled or its appearance being ed depending on the first volume level, wherein the second waveform is displayed with its signal axis being scaled or its appearance being modified depending on the second volume level.

Item 29: Method of item 28, wherein the first and second waveforms are displayed with their signal axes being scaled on the basis of current values of the first and second volume levels within a time period of not more than 2 seconds, preferably not more than 100 milliseconds, more preferably not more than 35 milliseconds.

Item 30: Device (10) for processing and playing audio data, preferably DJ gprocessingipment, comprisingunit for processing audio data of at least a first track and a [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman ed set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman second track, - a controlling section adapted to be controlled by a user to generate a control input representing a desired setting of a first volume level of the first track and a second volume level of the second track, - a recombination/mixing unit ured to combine the first track at a first volume level with the second track at a second volume level to generate output data, - a visualization unit configured to te waveform data for izing at least one waveform based on the first track, the second track and the control input - a playing unit (34, 36) for playing audio data based on the output data, and - optionally a display unit for displaying the waveform data.

Item 31: Device of item 30, wherein the visualization unit is ured to generate a first waveform based on the first track, n a scaling of a signal axis or a drawing style of the first waveform is set depending on the first volume level, and/or to generate a second waveform based on the second track, wherein a scaling of a signal axis or a drawing style of the second waveform is set depending on the second volume level.

Item 32: Device of item 30 or item 31, wherein the visualization unit is configured to calculate a combination track representing a combination of at least the first track at the first volume level and the second track at the second volume level, and to generate the waveform data such as to visualize the waveform of the combination track.

Item 33: Device of at least one of items 30 to 32, wherein the device is configured to allow waveform data based on a particular control input to be ted and yed on the display within a time period of not more than 2 seconds, preferably not more than 100 milliseconds, more ably not He than 35 milliseconds, after the particular control input is generated by [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by a.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by Rebecca.Chapman ation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] a.Chapman Unmarked set by Rebecca.Chapman the user.

Item 34: Device of at least one of items 30 to 33, wherein the visualization unit is ured to te waveform data for visualizing a first waveform based on the first track and the control input, and a second waveform based on the second track and the control input, and wherein the waveform data are generated such as to display the first waveform and the second waveform in an overlaid manner using one single baseline, but different signal axes and/or ent g styles such as to be visually distinguishable from one another.

Item 35: Device of at least one of items 30 to 34, wherein the device is adapted to carry out the method of at least one of items 1 to 14, and/or is a device according to at least one of items 15 to 23, wherein the first track is preferably the first decomposed track and/or the second track is preferably the second decomposed track.

Item 36: Device for processing and playing audio data, ably according to at least one of the items 15 to 23 and 30 to 35 and/or device configured to carry out a method of at least one of items 1 to 14 and 24 to 29, the device comprising: - an audio input unit for receiving a first track and a second track, said first track being a component of an audio mix track, - a controlling section (24) adapted to be controlled by a user to generate a control input representing a desired setting of a first volume level of the first track and a second volume level of the second track, - a playing unit (34, 36) for playing output data base on the first track at the first volume level and the second track at the second volume level, wherein the controlling section comprises at least one single control element (26-1, 26-2) which is operable by a user in a single control Doperation for controlling the first volume level and the second volume level, in particular changing a ratio between the first volume level and the [Annotation] Rebecca.Chapman None set by Rebecca.Chapman [Annotation] Rebecca.Chapman MigrationNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman [Annotation] Rebecca.Chapman None set by a.Chapman [Annotation] Rebecca.Chapman ionNone set by Rebecca.Chapman [Annotation] Rebecca.Chapman Unmarked set by Rebecca.Chapman second volume level from at least a value smaller than 1 to at least a value greater than 1 or vice versa.

Item 37: Device of item 36, wherein the first track and the second track are ents of the same audio mix track, wherein preferably the first track is a vocal track and the second track is a corresponding instrumental track.

Claims

1. A method for processing and playing audio data, comprising the steps 5 receiving mixed input data, the mixed input data being a sum signal obtained from mixing at least one first source track with at least one second source track; decomposing the mixed input data to obtain at least a first decomposed track resembling the at least one first source track; 10 generating output data based on the first decomposed track; wherein the mixed input data are first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track and wherein the method further comprises the steps of: receiving second mixed input data, the second mixed input data 15 being a sum signal obtained from mixing at least one third source track with at least one fourth source track; decomposing the second mixed input data to obtain a third decomposed track resembling the at least one third source track, and a fourth decomposed track resembling the at least one fourth source 20 track, wherein a control input represents a desired setting of a first volume level of the first decomposed track, a second volume level of a second osed track, a third volume level of the third decomposed track, and a fourth volume level 25 of the fourth decomposed track; and wherein recombined output data is generated by recombining the first osed track at the first volume level, the second osed track at the second volume level, the third decomposed track at the third volume level and the fourth 30 decomposed track at the fourth volume level; and playing the recombined output data h an audio output.

2. The method of claim 1, wherein at least one of the first mixed input data, the second mixed input data, the first osed track, the second decomposed track, the third decomposed track, and the fourth decomposed track represent stereo signals, each comprising a left-channel signal portion 5 and a right-channel signal portion, respectively.

3. The method of claim 1 or 2, wherein decomposing the first and second mixed input data includes processing the first and second mixed input data by an AI system.

4. The method of claim 3, wherein said AI system is trained by a plurality 10 of sets of training audio data, wherein each set of training audio data at least includes a first source track and a mixed track being a sum signal obtained from mixing at least the first source track or a track that resembles the first source track, with a second source track.

5. The method of claim 1 or 2, wherein the first mixed input data are 15 processed within a first AI system and a second AI system te from the first AI system, wherein the first AI system processes the first mixed input data to obtain only the first decomposed track and the second AI system processes the first mixed input data to obtain only the second osed track. 20 6. The method of claim 1, 2, or 5, wherein the second mixed input data are processed within a third AI system separate from the first and the second

6.AI system, and within a fourth AI system separate from each of the first to third AI systems, wherein the third AI system processes the second mixed input data to obtain only the third decomposed track and the fourth AI system 25 processes the second mixed input data to obtain only the fourth decomposed track.

7. The method of any one of the preceding claims, wherein the first mixed input data has a periodic beat structure, the second mixed input data has a periodic beat structure, and the method further ses: performing at least one of a tempo matching processing and a key 5 matching sing, wherein the tempo matching processing comprises: receiving first input data obtained from the first mixed input data and second input data obtained from the second mixed input data; time stretching or resampling of at least one of the first input 10 data and the second input data; and outputting first output data and second output data which have mutually matching tempos, wherein the key matching processing comprises: receiving the first input data obtained from the first mixed input 15 data and the second input data ed from the second mixed input data; pitch ng of at least one of the first input data and the second input data; and outputting first output data and second output data which have 20 mutually matching keys.

8. The method of any one of the preceding claims, further comprising: displaying a first waveform entative of the first decomposed track and displaying a second waveform representative of the second decomposed track, wherein the first waveform and the second waveform are 25 displayed in an overlaid manner using one single ne, and wherein the waveforms are displayed using different signal axes and/or different drawing styles such as to be ly distinguishable from one another.

9. The method of claim 8, wherein the first waveform is displayed with its 30 signal axis being scaled or its appearance being modified depending on the first volume level, wherein the second waveform is displayed with its signal axis being scaled or its appearance being modified depending on the second volume level.

10. The method of claim 8 or 9, wherein the first and second waveforms are displayed with their signal axes being scaled on the basis of current 5 values of the first and second volume levels within a time period of not more than 2 seconds, not more than 100 milliseconds, or not more than 35 milliseconds.

11. The method of any one of the preceding claims, wherein decomposing the mixed input data is d out segment-wise, wherein decomposing is 10 carried out based on a first segment of the mixed input data such as to obtain a first segment of output data, and wherein decomposing of a second segment of the mixed input data is carried out while playing the first segment of output data.

12. The method of any one of the preceding claims, wherein the receiving, 15 decomposing, ting and g steps are d out in a continuous process.

13. The method of any one of the preceding claims, wherein the mixed input data are received via streaming from a remote server.

14. A device for processing and playing audio data, comprising: 20 an audio input unit for receiving mixed input data, the mixed input data being a sum signal obtained from mixing at least a first source track with at least a second source track, wherein the audio input unit is a first audio input unit for receiving first mixed input data being a sum signal obtained from mixing at least a first source track with a second source track, 25 a first osing unit connected to the audio input unit for decomposing the mixed input data to obtain at least a first decomposed track resembling the first source track; a second audio input unit for receiving second mixed input data, the second mixed input data being a sum signal obtained from mixing at least a third source track with a fourth source track; a second decomposing unit connected to the second audio input unit 5 for decomposing the second mixed input data to obtain a third decomposed track resembling the third source track and a fourth decomposed track resembling the fourth source track; a recompose controlling section d to be controlled by a user to generate a l input representing a desired setting of a first volume level 10 of the first decomposed track, a second volume level of a second track, a third volume level of the third decomposed track, and a fourth volume level of the fourth decomposed track and a playing unit for playing recombined output data; a recombination unit for recombining at least the first decomposed 15 track with the second track to generate recombined output data for the g unit, wherein the recombination unit is adapted to generate the recombined output data by recombining the first decomposed track at the first volume level, the second track at the second volume level, the third decomposed track at the third volume level and the fourth decomposed track 20 at the fourth volume level.

15. The device of claim 14, n the recompose controlling section comprises a first single ose control element which is operable by a user in a single control operation for controlling the first volume level and the second volume level. 25

16. The device of claim 15, wherein increasing one of the first volume level and the second volume level, via the first single recompose control element, decreases the other one of the first volume level and the second volume level.

17. The device of any one of claims 14 to 16, wherein the recompose controlling section comprises: a single mix control element, which is operable by a user in a single control operation for controlling a first sum signal and a second sum signal, 5 the first sum signal being a sum of the first decomposed track at the first volume level and the second decomposed track at the second volume level and the second sum signal being a sum of the third decomposed track at the third volume level and the fourth decomposed track at the fourth volume level. 10

18. The device of claim 17, wherein increasing one of the first sum signal and the second sum signal, via the single mix l element, decreases the other one of the first sum signal and the second sum signal.

19. The device of claim 17 or 18, wherein the recompose controlling section further ses a second single recompose control element which 15 is operable by a user in a single control operation for controlling the third volume level and the fourth volume level.

20. The device of claim 19, n increasing one of the third volume level and the fourth volume level, via the second single recompose control element, decreases the other one of the third volume level and the fourth 20 volume level.

21. The device of any one of the preceding claims, further comprising: an input audio file buffer for loading n ts of an input audio file having a predetermined file size and a predetermined playback duration, which contains audio data to play the first mixed input data, 25 a first segment buffer connected to the decomposing unit to receive and store a segment of the first decomposed track ed from a segment of the input audio file, a second segment buffer connected to the decomposing unit to receive and store a t of the second decomposed track obtained from the same segment of the input audio file, wherein the playing unit comprises an audio interface having an 5 analog-to-digital converter to generate an analog audio signal from the recombined output data, said audio interface having an audio buffer for buffering portions of the recombined output data for playing, wherein a size of the first segment buffer and/or the second segment buffer is larger than a size of the audio buffer of the audio ace, but 10 smaller than the input audio file data or the ermined file size of the input audio file.

22. The device of any one of the preceding claims, wherein the first mixed input data is based on a periodic beat structure, the second mixed input data is based on a periodic beat structure, and the device further comprises: 15 at least one of a tempo matching unit and a key matching unit, wherein the tempo matching unit is arranged to receive first input data obtained from the first mixed input data and second input data obtained from the second mixed input data, wherein the tempo matching unit comprises a time stretching unit 20 adapted to time h at least one of the first input data and the second input audio data, and to output first output data and second output data which have mutually ng tempos, and wherein the key matching unit is arranged to receive the first input data obtained from the first mixed input data and the second input data 25 obtained from the second mixed input data, and wherein the key ng unit comprises a pitch shifting unit adapted to pitch shift at least one of the first input data and the second input audio data, and to output the first output data and the second output data which have mutually matching keys.

23. The device of any one of claims 14 to 22, wherein the first audio input 30 unit, the second audio input unit, the first decomposing unit, the second decomposing unit, the ose controlling section, the recombination unit, and the playing unit are: integrated within a single unit; or within a number of local units connected to one another via a local 5 network or via peripheral cable connections or via eld wireless connection.

24. The device of any one of claims 14 to 23, further comprising: a visualisation unit configured to display a first waveform representative of the first decomposed track and a second waveform 10 entative of the second decomposed track, wherein the first waveform and the second waveform are displayed in an overlaid manner using one single baseline, and wherein the waveforms are displayed using different signal axes and/or different drawing styles such as to be visually distinguishable from one r, 15 wherein the first waveform is displayed with its signal axis being scaled or its appearance being modified depending on the first volume level, wherein the second rm is displayed with its signal axis being scaled or its appearance being modified depending on the second volume level.

25. The device of claim 24, wherein the visualization unit is configured to 20 calculate a combination track representing a combination of at least the first decomposed track at the first volume level and the second decomposed track at the second volume level, and to display a waveform representative of the combination track.

26. The device of claim 24 or 25, wherein the device is configured to allow 25 waveforms based on a particular l input to be generated and displayed on the display unit within a time period of not more than 2 seconds, not more than 100 milliseconds, or not more than 35 milliseconds, after the ular l input is generated by the user. None set by Rebecca.Chapman MigrationNone set by Rebecca.Chapman Unmarked set by Rebecca.Chapman None set by Rebecca.Chapman MigrationNone set by Rebecca.Chapman ed set by Rebecca.Chapman (3:! ?ppiim?on First A! - First decom— system 44—1 posed track ' Input; _ audio? fileA Second Al - Second decom-i’i : system 44-2 posed track Mixed i MP“ Audio .......................................................................................................... track interface? .......................................................................................................... ; 48 Third A! - Third decom— 5 : : E system 44-3 posed track Input; audioé fileB Fourth A! - system 44-4 posed track