JP4301158B2 - Audio data transmitter - Google Patents

Description

  The present invention relates to a technique for supporting a music session using a network.

A service called “net session” that allows remote music players to perform music sessions by using a network has become widespread. This service is provided by accessing from a “session terminal”, which is a dedicated device specialized for a net session, to a server device that mediates exchange of audio data between the session terminals.
This session terminal has a function of encoding and decoding audio data in addition to a communication function. When a performer plays an electronic musical instrument connected to his / her session terminal, the audio data is transmitted from his / her session terminal via the server device to another performer's session terminal in near real time. Is emitted as music from the terminal. A music session is realized by performing such processing mutually.

Patent Documents 1 and 2 disclose a system that optimizes encoding and decoding when audio data is exchanged via a network. In the systems disclosed in these documents, a server device is provided with a function of encoding audio data using a plurality of compression methods. When the server device is requested to provide a service, the server device transmits audio data appropriately encoded by a compression method that can be decoded by the terminal device.
JP-A-10-11097 WO99 / 069691

By the way, many session terminals are equipped with a function for giving various acoustic effects such as reverb and delay to audio data input to the terminal itself. If this function is used as appropriate, each performer can also make the other party of the music session listen to the performance sound to which the acoustic effect is added.
However, there is a problem that the real-time property of the music session is impaired depending on the content of the acoustic effect given to the audio data.
This problem occurs for the following reason. In the network serving as the data transmission path for the session terminal, the allowable amount of data stream is determined for each band, and the terminal cannot transmit data larger than the allowable amount of the band used by itself. On the other hand, among the sound effects provided by each session terminal, there is one in which the amount of data after the application is increased is larger than the original audio data. If the amount of audio data after applying this type of sound effect increases to a level that exceeds the network capacity, there will be a delay in the time it takes for the audio data to reach the other terminal of the session. become.
The present invention has been devised under such a background, and is capable of giving various audio effects to the audio data while suppressing the data amount of the audio data exchanged via the network as much as possible. The purpose is to provide a mechanism.

An audio data transmitting apparatus according to a preferred aspect of the present invention is an audio data transmission apparatus in which each audio data for each musical instrument having a predetermined frequency characteristic is subjected to data processing for imparting various acoustic effects. storage means for association with each data quantity and the type of the data processing and the musical instrument type, input means for the data processing of a series of the sound effects to be imparted to sequentially input audio data set, the Of the data processing types stored in the storage unit corresponding to the instrument type of the audio data input to the input unit, the data processing amount of the data processing to be performed on the audio data is the smallest. A type is specified, and the data processing of the specified type among the data processing set for the audio data is performed. Wherein the the indication information generating means for generating a sound effect instruction information, the sound effects to the data processing of the type specified by the instruction information generating means, input by said input means for indicating the data processing to be performed after Effect applying means for applying to audio data ; transmitting means for transmitting the sound effect instruction information generated by the instruction information generating means; and the audio data to which the sound effect is applied by the effect applying means; Is provided.

  According to the present invention, it is possible to give various audio effects to audio data while suppressing the data amount of the audio data exchanged via the network as much as possible.

(Embodiment of the Invention)
An embodiment of the present invention will be described. The feature of the session system according to the present embodiment is that when audio data to which various sound effects are given is exchanged between session terminals that perform a music session together, the sound effect that should be originally given by the terminal on the audio data transmission side is A part or all of them can be contracted to the receiving terminal.
Here, main terms used in the following description are defined. “Operator” means an operator that provides various services using a network. “Performer” means an individual who uses the service as the owner of the session terminal under the support of the operator.

FIG. 1 is an overall configuration diagram of a session system according to the present embodiment. As shown in the figure, this system includes a plurality of session terminals 10 and a session management server device 30.
FIG. 2 is a block diagram illustrating a hardware configuration of the session terminal 10. As shown in the figure, the terminal 10 includes an audio data input unit 11, an audio data output unit 12, a communication unit 13, a terminal ID storage unit 14, a sound effect applying unit 16, an encoding / decoding unit 17, a display unit 18, and an operation. A unit 19 and a control unit 20 are provided.

The audio data input unit 11 inputs audio data from the electronic musical instrument 40 to the own terminal 10. On the other hand, the audio data output unit 12 outputs audio data from the terminal 10 to the speaker 50. The communication unit 13 establishes a connection with the session management server device 30 based on a pre-registered network address, and transmits and receives various data.
The terminal ID storage unit 14 stores in advance a terminal ID that is unique identification information for identifying the terminal 10 itself.

  The sound effect imparting unit 16 is a DSP (digital signal processor), a program memory storing a calculation execution unit that executes various calculations, a coefficient register that stores coefficients for calculation, and a microprogram that describes various data processing procedures. Etc. are built-in. The sound effect applying unit 16 applies a series of data processing set in advance by the performer to the audio data supplied to the player, thereby sequentially changing the data contents to those provided with various sound effects. The types of sound effects that can be provided by performing data processing include distortion, overdrive, delay, chorus, and reverb. For example, when adding sound effects in the order of “distortion” → “reverb”, the first audio data processing is applied to the input audio data to change the data content to the audio data with distortion. After that, the second data processing is performed to further modify the data contents to reverb-attached audio data.

  As shown at the beginning, the present embodiment allows the receiving terminal 10 to contract a part of the acoustic effect that should be originally provided in the audio data transmitting terminal 10. It has characteristics. Therefore, the sound effect providing unit 16 of each session terminal 10 not only performs data processing on the audio data input from the electronic musical instrument 40 to the session terminal 10, but also from the other session terminal 10 that is the partner of the music session. Data processing is also appropriately performed on the audio data transmitted to 10. The content of data processing to be performed on the former audio data is set in accordance with the operation content of the operation unit 19 of the terminal 10, while the content of data processing to be performed on the latter audio data is that of the audio data. It is set according to the content of the sound effect instruction information transmitted from the other terminal 10 prior to exchange. The details of the sound effect instruction information will be described in detail later in the description of the operation.

  The encode / decode unit 17 supplies the compressed audio data obtained by compressing and encoding the audio data to the communication unit 13 when the audio data to which the sound effect is given by the sound effect giving unit 16 is supplied. When the compressed audio data transmitted from the other terminal 10 is supplied, the audio data obtained by decoding the compressed audio data is supplied to the acoustic effect applying unit 16.

The display unit 18 is a display that displays various types of information. The operation unit 19 is a touch panel for inputting various information.
The control unit 20 controls each of the above units. The characteristic operation performed by the control unit 20 will be described in detail later with reference to a flowchart.

FIG. 3 is a block diagram illustrating a hardware configuration of the session management server device 30. This server device 30 operates under the management of the operator, and as shown in the figure, a CPU 31 that performs various controls, a RAM 32 that provides a work area to the CPU 31, and a ROM 33 that stores an IPL (initial program loader). In addition, a network interface 34, a computer display 35, a keyboard 36, a hard disk 37, and the like are provided. The hard disk 37 stores a music session support program 37a.
The music session support program 37a is a program for causing the CPU 31 to realize functions unique to the present embodiment. When this program is executed by the CPU 31, a session management table is formed in the RAM 32.

  FIG. 4 is a data structure diagram of the session management table. This table is a collection of a plurality of records each corresponding to one music session. In the example of FIG. 4, four records are provided. This means that the session management server device 30 can manage four music sessions individually. One record constituting this table has three fields of “session”, “participating member”, and “in-session flag”.

  In the “session” field, session identification information which is a unique number assigned in advance to each music session is stored. In the “participating member” field, the terminal ID transmitted from the session terminal 10 to the session management server device 30 is stored. Since this system is constructed on the assumption that one-to-one music sessions are provided, if two terminal IDs are stored in this field, a music session by the session terminal 10 that is the source of both the terminal IDs is stored. Will be set immediately. In the “in-session flag” field, information “1” indicating that the music session has already started or “0” indicating that the music session has not yet started is stored.

Next, the operation of this embodiment will be described. 5 to 7 are flowcharts showing the operation of this embodiment.
The process shown in the figure is started when any performer accesses the session management server device 30 from his / her session terminal 10.
The CPU 31 of the session management server device 30 that has received access from the session terminal 10 transmits the display data of the session room guidance screen generated based on the stored contents of the session management table in the RAM 32 to the accessed session terminal 10. (S100).

The control unit 20 of the session terminal 10 that has received the display data displays a session room guidance screen on the display unit 18 (S110).
FIG. 8 is a session room guidance screen. This screen represents the stored contents of each record in the session management table as presence / absence of entry into a virtual room called “session room”. In the upper part of the screen, a character string indicating that “Please select a session room to enter. However, you cannot enter a session room where a session has already started.” Is displayed. Further, in the lower part of the screen, the stored contents of the session management table are displayed as a table in which three items of “session room”, “entranced person”, and “state” are associated with each other. In this table, the session identification information stored in the “session” field of the session management table is displayed as an item of “session room”, and the terminal ID stored in the “participating member” field is “entering person”. It is displayed as an item. Further, for a music session whose stored content in the “session flag” field of the session management table is “1”, the “status” item of the corresponding session room is displayed as “in session” while “0” is displayed. Is displayed as “recruiting room attendees”.

The performer who refers to the session room guide screen selects the session room in which he / she enters the room by using the operation unit 19.
When the session room is selected by the operation unit 19, the CPU 31 of the session terminal 10 sends the session identification information of the selected session room and the terminal ID read from the terminal ID storage unit 14 to the session management server device 30. It transmits as a set (S120).

The CPU 31 of the session management server device 30 that has received the set of session identification information and terminal ID specifies a record corresponding to the received session identification information from the session management table (S130).
Subsequently, the CPU 31 stores the terminal ID in the “participating member” field of the record identified in step 130 (S140). In this step 140, the session terminal 10 in which the terminal ID transmitted by itself is stored in the “participating member” field is already stored in the same field, or another terminal ID stored later is transmitted. A music session can be performed with the original session terminal 10 as a partner.

CPU31 which memorize | stored terminal ID in the field of "participating member" judges whether the total of the terminal ID memorize | stored in the field became two (S150).
In step 150, the CPU 31 having determined that the total number of stored terminal IDs has become two, the storage content of the “in-session flag” field of the record identified in step 130 is changed from “0” to “1”. Rewrite (S160).
Subsequently, the CPU 31 reads out two terminal IDs stored in the “participating member” field of the record specified in step 130 into the RAM 32 (S170).
The CPU 31 transmits the display data of the sound effect setting request screen obtained by embedding the two terminal IDs read in step 170 to a predetermined template to the session terminal 10 that is the source of these two terminal IDs ( S180).

The control unit 20 of the session terminal 10 that has received the display data causes the display unit 18 to display a sound effect setting request screen (S190).
FIG. 9 is a sound effect setting request screen. In the upper part of this screen, you will see “This time, the following two people will have a session. Before starting the session, set the sound effect to be added to your performance sound.” A character string is displayed, and below it, both terminal IDs read out in step 160 are displayed as persons entering the room.

The player who refers to the sound effect setting screen sequentially inputs each sound effect desired to be given to the performance sound of the electronic musical instrument 40 to be played by the operation unit 19.
When each sound effect is input by the operation unit 19, the control unit 20 of the session terminal 10 performs a series of data processing for providing each input sound effect for the input audio data. (S200).
Next, the control unit 20 displays a trial performance request screen on the display unit 18 (S210). On the trial performance request screen, a character string “The session environment is optimized so please perform a trial performance” is displayed.

A player who refers to the trial performance request screen performs a predetermined melody with his / her electronic musical instrument 40. Then, the played melody audio data is input from the electronic musical instrument 40 to the session terminal 10.
When the audio data is input, the sound effect imparting unit 16 of the session terminal 10 applies the data processing set in step 200 to the audio data in the order in which they are set, so that the data contents are changed. A series of modified audio data is acquired (S220).

When the series of audio data is acquired, the control unit 20 measures the data amount of each acquired audio data (S230).
The control unit 20 that has measured the data amount identifies the stage in which the data content has been changed to the audio data having the smallest data amount, among the stages in which the acoustic effect applying unit 16 has performed each of the series of data processing. (S240).

The control unit 20 generates acoustic effect instruction information indicating one or a plurality of data processes that were to be performed after the stage specified in Step 240 in the series of data processes set in Step 200. (S250).
Furthermore, the control unit 20 transmits the sound effect instruction information generated in step 250 to the session management server device 30 (S260).

The CPU 31 of the session management server device 30 transfers the sound effect instruction information received from one session terminal 10 as it is to the other session terminal 10 that is the partner of the music session (S270).
The control unit 20 of the session terminal 10 that has received the sound effect instruction information from the session management server device 30 sets one or a plurality of data processes indicated by the sound effect instruction information in the encode / decode unit 17 for received audio data. (S280). As a result, two systems of data processing for input audio data and reception audio data are individually set in the encoding / decoding unit 17.

Next, the control part 20 transmits the message which shows that the setting of the acoustic effect was completed to the session management server apparatus 30 (S290).
When the CPU 31 of the session management server device 30 receives a message indicating that the sound effect setting has been completed from both session terminals 10 which are the destinations of the display data of the sound effect setting request screen, the remaining time until the start timing of the music session is received. A countdown signal representing the time length is transmitted to each of the session terminals 10 (S300).
The control unit 20 of the session terminal 10 that has received the countdown signal displays the remaining time length until the start timing of the music session on the display unit 18 (S310). Specifically, “3 seconds before start” → “2 seconds before start” → “1 second before start” is displayed every other second in this order, and “1 second before start” is also displayed. The character string “START!” Is displayed at the timing when one second has elapsed since the character string was displayed.
Each player refers to the countdown display displayed on the display unit 18 of his / her session terminal 10 and starts playing the electronic musical instrument 40 all at the same time when the character string “START!” Is displayed.

When the performance is started, the session management server device 30 performs a process of immediately transferring the compressed audio data transmitted from one session terminal 10 to the other session terminal 10.
In addition, the control units 20 of both session terminals 10 that exchange compressed audio data with each other via the session management server device 30 perform the following three processes in parallel.
a. Input Audio Data Output Control Process This is a process of supplying the audio data output unit 12 with the audio data that has been subjected to the series of data processing set in step 200 to the end by the sound effect applying unit 16. By performing this process, the audio data input from the electronic musical instrument 40 to the terminal 10 is emitted from the speaker 50 as a musical sound to which all of the sound effects input via the sound effect setting screen are added. Will be.
b. Input Audio Data Transmission Control Process This is a process of supplying the encode / decode unit 17 with audio data that has been subjected to the data process at the stage specified in step 240 in the series of data processes set in step 200. is there. By performing this processing, the audio data input from the electronic musical instrument 40 to the terminal 10 is transmitted to the session terminal 10 of the other party as compressed audio data having the smallest data amount.
c. Received audio data output control process This is a process of supplying the audio data output unit 12 with the audio data that has been subjected to the data processing set in the sound effect applying unit 16 in step 280. By performing this process, the audio data that has undergone the decoding process by the encoding / decoding unit 17 is a musical sound to which all of the acoustic effects that have been input via the acoustic effect setting screen on the partner terminal 10 side are added. Sound is emitted from the speaker 50.

These three processes will be described more specifically.
FIG. 10 is a diagram conceptually showing the processing content performed by a certain pair of session terminals 10. In this figure, the performer using the session terminal 10Y connected to the electronic musical instrument 40Y and the speaker 50Y has “acoustic effect A” → “acoustic effect B” → “acoustic effect C” → “ The player who performs the instruction to give the “sound effect D” and uses the session terminal 10Z connected to the electronic musical instrument 40Z and the speaker 50Z, “sound effect A” → “sound effect B” → “ It is assumed that an input instructing the application of “acoustic effect E” → “acoustic effect F” is performed.

Under such circumstances, first, attention is focused on the flow of processing performed on audio data input from the electronic musical instrument 40Y to the session terminal 10Y. When audio data is input from the electronic musical instrument 40Y, the control unit 20 of the session terminal 10Y executes an input audio data output control process and an input audio data transmission control process. By performing the former process, audio data to which all of the acoustic effects A, B, C, and D are applied is emitted from the speaker 50Y. On the other hand, by performing the latter processing, the audio data to which the acoustic effect C is given is transmitted to the session terminal 10Z as compressed audio data. The reason why the audio data is transmitted at the stage where the acoustic effect C is applied is that the data amount at that stage is determined to be the smallest.
When the compressed audio data is received, the control unit 20 of the session terminal 10Z executes a received audio data output control process. By performing this process, audio data to which the sound effect D is further applied is emitted from the speaker 50Z.

Next, attention is focused on the flow of processing performed on audio data input from the electronic musical instrument 40Z to the session terminal 10Z. When audio data is input from the electronic musical instrument 40Z, the control unit 20 of the session terminal 10Z also executes input audio data output control processing and input audio data transmission control processing. By performing the former process, audio data to which all of the acoustic effects A, B, E, and F are applied is emitted from the speaker 50Z. On the other hand, by performing the latter processing, the audio data to which the acoustic effect B is given is transmitted to the session terminal 10Y as compressed audio data. The reason why the audio data is transmitted at the stage where the sound effect B is applied is that the data amount at that stage is determined to be the smallest.
When the compressed audio data is received, the control unit 20 of the session terminal 10Y executes a received audio data output control process. By performing this process, the audio data further added with the acoustic effects E and F is emitted from the speaker 50Y.

  According to the present embodiment described above, the session terminal 10 that transmits audio data has the largest amount of data at the stage where a part of a series of data processing to be executed on the own terminal 10 side is performed. A small amount of audio data is transmitted to the other session terminal 10. Then, the session terminal 10 that has received the audio data performs the remaining data processing that is not performed on the transmission side, and then emits the sound as a musical sound. Accordingly, it is possible to cause the receiving session terminal 10 to emit the musical sound to which all of the acoustic effects intended by the transmitting session terminal 10 are given while suppressing the amount of audio data distributed through the network.

(Other embodiments)
This embodiment can be modified in various ways.
In the above embodiment, when a series of data processing is set in the sound effect imparting unit 16, a trial performance is required for the performer. When the performer performs a trial performance of the electronic musical instrument 40 in response to the request, a series of data processing registered in the acoustic effect applying unit 16 is performed on the audio data input from the electronic musical instrument 40 to the session terminal 10. Therefore, the data processing with the smallest data amount has been specified. This is because the amount of data after data processing depends on the frequency characteristics of the audio data to be processed, so if each data processing is not performed on the audio data of the electronic musical instrument 40 used for performance, This is because it cannot be determined whether the data amount is the smallest at the stage of data processing.

  On the other hand, as shown in FIG. 11, a table is prepared in advance that associates various types of musical instruments having substantially similar frequency characteristics with the data amount when data processing of each acoustic effect is performed on the audio data. By doing so, the trial performance as described above may be omitted. According to this modification, each session terminal 10 specifies a record corresponding to the type of the electronic musical instrument 40 connected to it from the table, and refers to the data amount defined for each data process in the record. By doing so, it is possible to uniquely specify the data processing with the smallest data amount among the series of data processing set in the sound effect imparting unit 16, so that it is not necessary to force the performer to perform a trial performance.

  The configuration and operation of the session terminal 10 according to this modified example are conceptually described as “obtained by performing data processing for imparting various acoustic effects to audio data obtained by playing a certain type of musical instrument. Storage means for storing the data amount of the audio data to be associated with the respective data processes, input means for sequentially inputting the audio data, and audio data sequentially input from the input means. By applying a series of data processing, the effect imparting means for acquiring a series of audio data whose data contents are respectively modified, and the storage means is associated with the smallest data amount in the set series of data processing Identify the data processing that was to be performed later than the specified data processing, and indicate the specified data processing Instruction information generating means for generating sound effect instruction information, transmission means for transmitting the generated sound effect instruction information, and a processing result at a stage where the effect applying means has executed the specified data processing. An audio data processing apparatus provided. "

  In the above-described embodiment, each session terminal 10 transmits audio data at a stage where data processing with the smallest amount of data is performed among a series of data processing set by the performer to the terminal 10 of the session partner. It was like that. On the other hand, if it is less than the stage where the set series of data processing is performed to the end, it may be transmitted at the stage where the data processing is performed so that the data amount is the second smallest. The data may be transmitted at a stage where data processing is performed to reduce the data amount. In short, as long as the data amount can be transmitted in a smaller amount than the case where all sound effects are given on the own terminal 10 side, the data may be transmitted at any stage.

1 is an overall configuration diagram of a session system. It is a hardware block diagram of a session terminal. It is a hardware block diagram of a session management server apparatus. It is a data structure figure of a session management table. It is a flowchart which shows operation | movement of embodiment. It is a flowchart which shows operation | movement of embodiment. It is a flowchart which shows operation | movement of embodiment. It is a session room guide screen. It is a sound effect setting request screen. It is a conceptual diagram which shows the processing content of a session terminal. It is the table which defined the relationship between a sound effect and data amount.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Session terminal, 11 ... Audio data input part, 12 ... Audio data output part, 13 ... Communication part, 14 ... Terminal ID memory | storage part, 15 ... Sound effect setting part, 16 ... Sound effect provision part, 17 ... Encode / decode , 18 ... display unit, 19 ... operation unit, 20 ... control unit, 30 ... session management server device, 31 ... CPU, 32 ... RAM, 33 ... ROM, 34 ... network interface, 35 ... computer display, 36 ... keyboard, 37 ... hard disk, 40 ... electronic musical instrument, 50 ... speaker

Claims (1)

A data amount of each audio data, a type of the data processing, and a type of the instrument when each data processing for giving various sound effects is performed on each audio data of each musical instrument having a predetermined frequency characteristic. Storage means for storing in association with each other;
Input means for sequentially inputting audio data in which the data processing of the series of acoustic effects to be applied is set ;
The data processing in which the data amount of the data processing to be performed on the audio data is the smallest among the data processing types stored in the storage unit corresponding to the instrument type of the audio data input to the input unit An instruction information generating means for generating acoustic effect instruction information indicating the data processing to be performed after the specified data processing among the data processing set in the audio data ;
Effect applying means for applying the acoustic effect up to the type of data processing specified by the instruction information generating means to the audio data input by the input means ;
An audio data transmitting apparatus comprising: a transmitting unit that transmits the sound effect instruction information generated by the instruction information generating unit and the audio data to which the sound effect is applied by the effect applying unit .

Images