JP7633799B2 - Audio Control Console - Google Patents

Description

The present invention relates to an audio mixing console that utilizes AI technology.

Broadcasting studios for radio and television, and music recording studios are equipped with audio mixing consoles that adjust the volume and sound quality of input audio signals. A microphone is used as one of the input devices for audio signals. The audio mixing console adjusts the volume and sound quality of each audio signal input from multiple microphones or from CDs that serve as background music sources, and then mixes and outputs the audio signals. The audio mixing console is operated by an operator called a mixing technician. The operator operates the audio mixing console to manually adjust the audio signal mixing balance to optimize it.

In order to keep production costs down, broadcasting stations are required to produce programs efficiently with a small number of staff. To make effective use of the studio, the tools to be used in the program are prepared beforehand before entering the studio. This has become increasingly important with the recent spread of remote work. Also, during the program production process that actually uses the studio, it is important to prevent a decline in program quality due to operating errors and an increase in production time due to re-recording.

Similarly, staff cuts have resulted in some local broadcasting stations being forced to cut production staff, resulting in a decline in program quality. Furthermore, the baby boomer generation, which brings with them the skilled know-how that has supported program production up until now, is retiring in recent years, and there is the problem that their know-how will disappear. This means that broadcasting stations' assets will disappear, posing a management problem.

To solve the above problems with audio control consoles, production tools such as offline program data creation tools, automatic faders, and automatic mixing functions have been proposed. For example, as shown in Patent Documents 1 to 3, technology has been proposed in which various parameters adjusted or set by an operator are stored and then recalled and used when necessary.

JP 2004-80735 A JP 2013-197764 A JP 2016-157996 A

However, the inventions described in the above patent documents are designed to call up parameters set during rehearsals during the actual performance, or to call up parameters adjusted or set for a particular scene for use in a similar scene, and are not designed to call up and use appropriate parameters in response to a wide variety of situations.

In particular, there are a wide variety of input/output devices such as microphones and speakers used in broadcasting stations, each with different input/output characteristics. In addition, there are a huge number of different volumes and sound qualities of the voices of individual performers that are input and output to these devices, as well as the types of background music used. Furthermore, when these combinations change, the parameters that need to be adjusted or set accordingly also change. For this reason, even if parameters are stored for specific scenes or devices, as in conventional technology, this lacks versatility, and it is difficult for operators to call up appropriate data depending on the situation they want to use.

The present invention has been proposed to solve the problems of the conventional technology described above. The object of the present invention is to provide an audio adjustment console that uses AI technology to automatically call up various audio adjustment parameters according to a wide variety of situations, such as each connected device, the input/output audio signals, and each scene used, thereby enabling highly versatile and easy-to-use audio adjustment.

In order to achieve the above object, the audio mixing console of the present invention has the following configuration.
(1) An audio signal input unit to which multiple audio signals are input, multiple audio signal adjustment units that adjust the input multiple audio signals based on predetermined parameters, an audio signal mixing unit that mixes the audio signals output from the multiple audio signal adjustment units, and an audio signal output unit that outputs the audio signals mixed by the audio signal mixing unit.
(2) An information input unit that inputs information regarding equipment connected to the audio adjustment console, information regarding audio signals input/output to the equipment, and information regarding the parameters of the audio adjustment performed on the audio signals by the audio signal adjustment unit.
(3) An information set generation unit that links each piece of information input to the information input unit to each other to generate a single information set so as to include information regarding parameters of audio adjustments performed on the audio signal.
(4) A storage keyword generation unit that extracts multiple values related to each piece of information contained in each information set generated by the information set generation unit based on predetermined criteria, and sets each extracted value as a storage keyword for that information set.
(5) A storage unit that links and stores each of the information sets and their keywords.
(6) A search keyword generation unit that extracts multiple values related to each piece of information from at least one or a combination of information related to the equipment connected to the audio adjustment console and information related to the audio signal input/output to the equipment, which are input to the information input unit when the audio signal adjustment unit performs audio adjustment, based on predetermined criteria, and sets each extracted value as a keyword for searching for that information.
(7) An information set search unit that searches for information sets stored in the memory unit using the search keywords and the storage keywords, and extracts information sets in which the search keywords and the storage keywords match specified criteria.
(8) An adjustment control unit that acquires information regarding parameters of audio adjustment included in the information set extracted by the information set search unit, and causes the audio signal adjustment unit to perform audio adjustment based on the acquired parameters.

The audio mixing console of the present invention also includes the following configuration.
(1) the information set search unit extracts a plurality of information sets in which the search keyword and the storage keyword match a predetermined criterion;
The adjustment control unit calculates and processes a plurality of audio adjustment parameters contained in the plurality of extracted information sets, and causes the audio signal adjustment unit to perform audio adjustment based on the parameters obtained as a result of the calculation.

(2) the information input unit has a progress table input unit for receiving a progress table of a program for which the audio signal adjustment unit performs audio adjustment;
a program progress status extraction unit that extracts information on a device connected to an audio control console, information on an audio signal input/output to the device, and a time-dependent change in program progress from the information described in the progress table,
the information set search unit extracts different information sets in response to a change over time in the program progress input from the progress table input unit;
The adjustment control section performs different audio adjustments in accordance with changes over time in the progress of a program, based on audio adjustment parameters contained in the different information sets extracted by the information set search section.

(3) An output level detection unit that, at the instruction of the adjustment control unit, performs adjustment processing on the input audio signal based on the acquired parameters, and detects whether the signal level after the adjustment processing is lower than a certain level.
(4) The output level detection unit has a function of notifying the audio input unit of an abnormality in the input of the audio signal when the signal level after the adjustment process is lower than a preset threshold value.

(5) When the signal level after the adjustment process is lower than a preset threshold value, the output level detection unit:
searching for a plurality of information sets linking information on the audio signal input to the device having a search keyword common to the audio signal on which an adjustment process has been performed based on the acquired parameters and information on an output level of the audio signal on which an adjustment process has been performed, from information on the audio signals input to the device and output from the audio signal;
The adjustment control unit performs a calculation process on information regarding the output level of the audio signal contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so that the output level becomes the output level obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters.

(6) A gain reduction value detection unit is provided to detect a gain reduction value of a compressor for the audio signal after the adjustment process,
When the gain reduction value of the compressor after the adjustment process is out of a preset threshold, the gain reduction value detection unit:
searching for a plurality of information sets linking information on the audio signal input to the device having a search keyword common to the audio signal on which an adjustment process has been performed based on the acquired parameters and information on a gain reduction value for the audio signal on which an adjustment process has been performed, from information on the audio signals input to and output from the plurality of devices;
The adjustment control unit performs a calculation process on information regarding the gain reduction value of the audio signal on which the adjustment process has been performed, which is contained in the extracted multiple information sets, and causes the audio signal adjustment unit to perform audio adjustment based on parameters that correct the gain reduction value that deviates from the threshold value so as to obtain the gain reduction value obtained as the calculation result.

(7) A balance comparison unit is provided that compares a direct sound and an indirect sound of the audio signal after the adjustment process,
When the balance between the direct sound and the indirect sound after the adjustment process deviates from a preset threshold, the balance comparison unit
searching for a plurality of information sets linking information on an audio signal input to the device having a search keyword common to an audio signal that has been subjected to an adjustment process based on the acquired parameters, and information on the balance between direct sound and indirect sound for the audio signal that has been subjected to the adjustment process, from information on the audio signals input to and output from the plurality of devices;
The adjustment control unit calculates and processes information regarding the balance between direct sound and indirect sound of the audio signal for which the adjustment process has been performed, which is included in the extracted information sets, corrects the parameters for which the adjustment process has been performed so as to achieve the balance obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters.

(8) The audio signal mixing unit includes a mixing balance comparison unit that compares mixing balances of the plurality of audio signals after the adjustment process,
The mixing balance comparison unit is
searching for a plurality of information sets linked together information on the audio signal input to the device having a search keyword common to the audio signal on which adjustment processing has been performed based on the acquired parameters, and information on the mixing balance when the audio signal mixing unit mixes the plurality of audio signals on which adjustment processing has been performed, from among information on the audio signals input to and output from the plurality of devices;
The adjustment control unit performs calculations on information regarding the mixing balance by the audio mixing unit contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so as to achieve the mixing balance obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters.

(9) An EQ/FIL detection unit is provided for detecting an equalizing effect and/or a filtering effect on the audio signal after the adjustment process,
the EQ/FIL detection unit detects whether the equalizing effect and/or filtering effect after the adjustment process deviates from a preset threshold value, and if it deviates from the threshold value, searches for a plurality of information sets in which information on an audio signal input to a device having a search keyword common to the audio signal on which the adjustment process has been performed and information on the equalizing effect and/or filtering effect for the audio signal on which the adjustment process has been performed are linked, from information on audio signals input to and output from a plurality of devices;
The adjustment control unit performs calculations on information regarding the equalizing effect and/or filtering effect contained in the extracted information sets, corrects the parameters on which the adjustment process was performed so that the equalizing effect or filtering effect obtained as a result of the calculation is obtained, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters.

The present invention makes it possible to provide a highly versatile audio adjustment console that allows for excellent audio adjustment with minimal operations.

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention. 2 is a diagram showing an example of a program progress table to be input to an information input unit in the embodiment of FIG. 1;

[1. Configuration of the embodiment]
[1-1. Basic configuration]
Hereinafter, an embodiment of the present invention will be specifically described with reference to Fig. 1. As shown in Fig. 1, the audio mixing console of this embodiment has a plurality of audio signal input units 2a, 2b, ..., 2n, a plurality of audio signal adjustment units 3a, 3b, ..., 3n that adjust audio signals input from each of the audio signal input units 2a, 2b, ..., 2n to predetermined frequency characteristics and levels, direct sound ratios, and mixing balances of each audio signal, an audio signal mixing unit 4 that mixes the audio signals that have been adjusted by each of the audio signal adjustment units 3a, 3b, ..., 3n, and an audio signal output unit 5 that outputs the audio signal mixed by the audio signal mixing unit 4.

The audio signal input units 2a, 2b, ... 2n are connected to multiple input devices 1a, 1b, ... 1n, which are sound sources such as playback devices such as microphones and CDs, and the audio signals output by these input devices 1a, 1b, ... 1n are input. For example, in a broadcasting studio, the voices of a male announcer, a female announcer, and multiple commentators, and in a recording studio for a band performance, the voices of vocals and other instruments are input to the audio signal input units 2a, 2b, ... 2n as audio signals A, B, ... and N of each channel. Note that when playback devices are used as input devices 1a, 1b, ... 1n, the playback devices only output audio signals A, B, ... and N.

The audio signal adjustment units 3a, 3b, ... 3n detect the audio signals input from the audio signal input units 2a, 2b, ... 2n and adjust them to a predetermined frequency characteristic. The audio signal adjustment units 3a, 3b, ... 3n are, for example, parametric equalizers (PEQs), and adjust setting parameters such as center frequency, equivalent amount, and Q value (Quality Factor) so that the frequency characteristics of the input audio signals become uniform or become musically suitable frequencies. In addition, the output level (gain), the ratio of direct sound to indirect sound, dynamics, and the mixing balance of each audio signal are also adjusted based on the setting parameters, and the characteristics of the mixed output sound are adjusted. The audio signal mixing unit 4 mixes the audio signals that have been adjusted by the audio signal adjustment units 3a, 3b, ... 3n. The audio signal output unit 5 outputs the audio signals mixed by the audio signal mixing unit 4 to an output device such as a speaker or recording device, or a television or radio transmitter in the case of live broadcasting.

In this embodiment, the main components include an information input unit 7, an information set generation unit 11, a storage keyword generation unit 12, a memory unit 13, a search keyword generation unit 14, an information set search unit 15, and an adjustment control unit 20. In addition, the device has an output level detection unit 16, a gain reduction value detection unit 17, a direct sound/indirect sound balance comparison unit 18, and a mixing balance comparison unit 19 to analyze the signal after the audio adjustment process by the audio signal adjustment units 3a, 3b, ... 3n and automatically correct parameters for audio adjustment according to the analysis results.

[1-2. Configuration of main parts]
[1-2-1. Information input section 7]
The information input unit 7 inputs three types of information, i.e., information on the device connected to the audio adjustment console, information on the audio signal input/output to the device, and information on the parameters of the audio adjustment performed on the audio signal by the audio signal adjustment units 3a, 3b, ..., 3n. For this reason, the information input unit 7 has an equipment/audio signal/parameter information input unit 8.

The method of inputting information into the device/audio signal/parameter information input unit 8 can be appropriately selected from known methods, such as reading various information in a data format such as a list or CSV, or organizing and reading the information manually or using a sorting program according to a predetermined format. In addition to the three types of information, it is also possible to read information about changes in the program over time in the form of a program progress chart, which will be described later, or to read other information such as alarms that occur during audio adjustment and operator comments.

Information about the devices connected to the audio control console includes the characteristics of input devices 1a, 1b, ... 1n, such as microphones, speakers, VTRs, CDs, and outside telephones, and output device 6, as well as the data format, number of bits, sampling frequency, and type of instrument.

The information about the audio signals input to and output from the equipment includes information about the user of the equipment connected to the audio mixing console and information about the recording of the audio signals. For example, the information includes the following:
(1) Information about the production of the program: Type and value of audio adjustment parameters; Optimal values of parameters re-adjusted by the operator; Cue sheet; Program comments; Production date and time; Microphones used; Performers; Recording location

(2) Information about the recording of the audio signal, such as the recording date and environmental conditions (season, weather, temperature, humidity, location, indoor/outdoor, latitude, altitude, etc.)
・Recording conditions (studio, live music venue, hall, number of people)
・Settings (number of people, microphone position, etc.)

(3) Information about the user: the person using the service (professional or amateur, male or female, age, and the announcer's physical condition)
Recording engineer

(4) Information required by broadcasting stations regarding multiple announcer comments, music, VTR, background music, etc., such as song titles, composers, performers, producers, resolution, and other information about audio sources other than audio.

The information about the audio adjustment parameters includes the types and values of various parameters, information about the operator who set the parameters, and information about alarms that were generated and output by the operator or automatically when the parameters were set. Specifically, the information includes what processing the audio adjustment console of this embodiment will perform for the following parameter types and their values:

(1) Microphone gain setting: Set the optimum gain depending on the type of microphone.
・Set the initial value based on conditions such as microphone + announcer.
- Monitors the input level of the signal processing unit, and determines and fine-tunes the optimal gain based on the average value and headroom.

(2) Filter settings: Set the optimal cutoff frequency depending on the type of microphone.
・Set the optimum frequency based on environmental conditions.

(3) Equalizer settings: The FFT of the input signal is compared with the FFT of the reference signal to determine the EQ setting value so that the frequency characteristics are consistent.
In addition to the adjustment results of this embodiment, the settings of a specified engineer can also be used as reference.

(4) Compressor setting: The current compressor gain reduction value is compared with the reference gain reduction value, and the compressor setting value is determined so that the characteristics are consistent.
In addition to the adjustment results of this embodiment, the settings of a specified engineer can also be used as reference.

(5) Fader setting: When speaking alone, adjust the fader according to the announcer’s voice volume.
- BG level can be controlled according to the cue sheet.
- When you start talking, the audio source such as a CD will automatically be set to background level.

(6) Effector settings (reverb)
- Compare the balance of direct and indirect sound of the current sound source with the balance of direct and indirect sound of the reference, and determine the setting value so that the characteristics are aligned.

[1-2-2. Input of progress chart into information input unit 7]
The information input unit 7 includes a progress table input unit 9 that receives a progress table of a program for which the audio signal adjustment units 3a, 3b, ..., 3n are to perform audio adjustment. That is, Fig. 2 shows an example of a program progress table, which chronologically lists the microphones used as the program progresses, their users (caster, commentator, guest), microphone levels, VTR audio, VTR level, BGM audio, BGM level, and others (comments, etc.). The progress table input unit 9 reads, for example, a progress table in this format. Note that the format of the progress table is not limited to that shown in Fig. 2, and other formats such as an EDL list can also be used.

The progress table is entered primarily when new audio adjustments are to be made, in order to input information about the equipment connected to the target audio adjustment console and about the audio signals input and output to the equipment. This is because audio adjustment parameters and their values are not normally entered into the progress table. However, if the program progress and information about the audio adjustment parameters at that time (microphone level, VTR level, BGM level in Figure 2) are linked and left as is the case with the progress table in Figure 2, it can also be used to read each piece of information when creating an information set that will serve as training data.

The information input unit 7 includes a program progress extraction unit 10 that extracts information on the devices connected to the audio control table, information on audio signals input and output to the devices, and changes in program progress over time from the information listed in the progress table input to the progress table input unit 9. That is, since a progress table such as that shown in FIG. 2 is generally prepared in the form of a list, the various information contained therein is not organized into the three types of information as in this embodiment. Therefore, the program progress extraction unit 10 uses a dictionary file or a sorting program separately prepared to extract information on the devices connected to the audio control table and information on audio signals input and output to the devices from the information listed in the progress table. At the same time, information that allows confirmation of changes over time as the program progresses, such as the scene numbers in Table 2 and the start times of each scene based on the start time of the program, is also extracted from the progress table.

[1-2-3. Information set generation unit 11]
The information set generator 11 generates one information set by linking the pieces of information input to the information input unit 7 to each other so as to include information on the parameters of the sound adjustment performed on the sound signal. In this embodiment, when a new sound adjustment is performed, the created information set is used as training data to automatically obtain new required sound adjustment parameters and their values. Therefore, it is necessary to compare information on each device and sound signal when performing new sound adjustment with various information included in the training data and call up the optimal training data.

The various types of information input to the information input unit 7 do not necessarily include the three types of information evenly depending on the input data format and input method, and some information is missing information on voice adjustment parameters necessary for the teacher data. In this embodiment, the information set generation unit 11 links the three types of information so as to include information on voice adjustment parameters essential for new voice adjustment, and generates a single information set that can be extracted by search.

The information set generator 11 can be used to generate information sets as so-called growing teacher data. In other words, the three types of information that make up the information set contain various information, and at the time that the information is input to the information input unit 7, it is unclear whether or not some information is necessary for the audio adjustment process. In this embodiment, when generating the information set generator 11, the input information is stored as is in each information set without being cut.

In other words, in conventional search engines, if the words and values contained in the generated information set were directly used as search keywords, the amount of information to be processed would be enormous. Therefore, in conventional AI-type search devices, the operator or the creator of the search engine appropriately extracts each piece of information contained in the information set when the information set is generated. Therefore, the information set is generated and stored in the memory unit with seemingly worthless information that was included in the information set at the time of input deleted. However, as information accumulates, a large amount of information that was thought to be worthless at a certain point in time will be generated, and it is expected that this will become important when performing voice adjustment.

In anticipation of such a situation, in this embodiment, the information entered in the information set is stored as much as possible without deleting any information. Instead, the saved keyword generation unit 12, which will be described later, generates keywords for each information set to improve search speed. As a result, even if a large number of information sets are accumulated and information that was initially thought to be worthless is accumulated and then changes to something meaningful for searching, the saved keywords can be reconstructed by the saved keyword generation unit 12, thereby updating the information set, i.e., growing the training data.

Of course, the information input to the information input unit 7 may include data other than the three types of information mentioned above that does not need to be reflected in the teacher data. Such data may be deleted by using an appropriate filter or dictionary file, or an alarm may be sent to an operator, etc.

[1-2-4. Save keyword generation unit 12]
The storage keyword generating unit 12 extracts a plurality of values related to each piece of information input to the information input unit 7 and/or each piece of information included in each piece of information set generated by the information set generating unit 11 according to a predetermined criterion, and sets each extracted value as a storage keyword for the information set. That is, since the words and values included in the three types of input information are huge in number, it is not preferable in terms of search efficiency to use these words and values as keywords. In this embodiment, the three types of input information are super-conceptualized by a dictionary file or a sorting program, and numerical values are divided into a certain range or averaged, thereby generating a simplified storage keyword. For example, the storage keyword is generated by super-conceptualizing the type of BGM into pop and classical music by referring to the songs included in each piece of information, classifying individual announcers and MCs into female and male, and further performing calculation processing such as rounding off parameter values.

The generation of the storage keywords is not only performed when the information set is generated, but is performed successively according to a certain standard, for example, periodically by a program incorporated in the storage keyword generation unit 12, or according to the operator's instructions, etc., since the information set functions as growing teacher data as described above. In this way, by changing the keywords that the storage keyword generation unit 12 assigns to each information set, the information set extracted during the search can follow the changes in each piece of information input. For example, even if the amount of information is almost zero when it is first input, such as a new announcer, a new microphone, or a sound source, and is information that cannot become a keyword in a normal search engine, in this embodiment, the input raw information is saved in the information set, and the storage keywords are periodically reviewed, so that it can be extracted as a search target when the amount of information has accumulated.

[1-2-5. Search keyword generation unit 14]
The search keyword generating unit 14 extracts a plurality of values related to each piece of information from at least one or a combination of information related to the device connected to the audio adjustment console and information related to the audio signal input/output to the device, which are input to the information input unit 7 when the audio signal adjustment units 3a, 3b, ..., 3n perform audio adjustment, based on a predetermined criterion, and sets each extracted value as a keyword for searching the information. That is, for the same reason as for the storage keywords, in this embodiment, the search keywords are also simplified by conceptualizing the three types of input information as a higher level by a dictionary file or a sorting program, or dividing or averaging numerical values into certain ranges.

[1-2-6. Information set search unit 15]
The information set search unit 15 searches for information sets stored in the memory unit 13 using the search keyword and the storage keyword, and extracts information sets in which the search keyword and the storage keyword match a predetermined criterion. That is, the number of information sets stored in the memory unit 13 increases enormously as teacher data is accumulated, so that when the information sets stored in the memory unit 13 are searched, a large number of information sets may be extracted.

In this embodiment, the information set to be extracted is optimized by setting specific criteria. For example, from the extracted information sets, an optimal information set is extracted based on criteria such as the most recent information set, voice adjustments performed by an operator with a high level of experience, the number of matches between search and saved keywords, and the weighting of keywords.

It is also possible for the information set search unit 15 to extract multiple information sets. In this case, the data of the multiple extracted information sets is output to the adjustment control unit 20, which then performs averaging and other calculation processing to determine optimal voice adjustment parameters and their values.

When new audio adjustments are made by inputting a progress table, the information set search unit 15 extracts multiple information sets in which the search keywords and storage keywords match a specified criterion, in line with changes in the program over time. In other words, as the program progresses over time, the program title, performers, equipment used, and so on change from moment to moment, and different audio adjustment parameters and their values are required. In this embodiment, different information sets are extracted in response to these changes.

[1-2-7. Adjustment control unit 20]
The adjustment control unit 20 acquires information on the parameters of the sound adjustment included in the information set extracted by the information set search unit 15, and causes the sound signal adjustment units 3a, 3b, ..., 3n to perform sound adjustment based on the acquired parameters. When multiple information sets are extracted, the adjustment control unit 20 performs calculation processing on multiple sound adjustment parameters included in the multiple extracted information sets, and causes the sound signal adjustment units 3a, 3b, ..., 3n to perform sound adjustment based on the parameters obtained as a result of the calculation. As a method of calculation processing, averaging, median selection, weighting, etc. can be used, but other methods are also acceptable.

When the adjustment control unit 20 searches for information sets that change over time according to the input progress table, it performs different audio adjustments according to changes in the program progress over time, based on the audio adjustment parameters contained in the different extracted information sets. This makes it possible to perform appropriate audio adjustments for each scene in the program.

[1-2-8. Storage unit 13]
The storage unit 13 stores each information set and its keyword in association with each other. In addition, it stores information input to other parts of this embodiment and various data generated by other parts. When new audio adjustment is performed, new parameters and their values generated by the audio adjustment console of this embodiment can be stored as teacher data in association with various information. In addition, it is also possible to output the stored contents in the form of a list or progress chart so that the operator or performer can verify the determined new parameters and their values, as well as alarms. Furthermore, parameters set by the operator himself are also stored in the storage unit 13.

[1-3. Configuration for automation]
[1-3-1. Output level detection]
The output level detection unit 16 performs adjustment processing on the audio signal input to the audio signal adjustment units 3a, 3b, ..., 3n based on the acquired parameters in accordance with an instruction from the adjustment control unit 20, and detects whether the signal level after the adjustment processing is lower than a certain level. The output level detection unit 16 has a function of notifying an abnormality in the input of the audio signal to the audio signal input units 2a, 2b, ..., 2n when the signal level after the adjustment processing is lower than a preset threshold value. For example, an alarm is output when the signal level after the adjustment processing is lower than a certain level due to an inappropriate microphone setting (off-mic, inappropriate distance, angle, etc.), voice quality, obstacles, etc.

The output level detection unit 16 has a function of automatically adjusting the appropriate signal level when the signal level after the adjustment process is lower than a preset threshold value or higher than a preset threshold value. One method is to obtain new, more appropriate parameters by averaging the parameters of multiple information sets or performing calculations using a method such as taking the median value, instead of the parameters of the information set that is the basis of the signal level that deviates from the threshold value. In this case, the output level detection unit 16 searches for multiple information sets that link information about an audio signal input to a device that has a search keyword in common with the audio signal that has been adjusted, and information about the output level of the audio signal that has been adjusted, from information about audio signals input to and output from multiple devices, and outputs the search results to the adjustment control unit 20.

The adjustment control unit 20 averages information related to the output level of the audio signal contained in the extracted multiple information sets and performs other calculations to correct the parameters on which the adjustment process was performed so that the output level is obtained as a result of the calculation, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on the corrected parameters.

As another method for automatically correcting the output level, instead of or in addition to the method of arithmetic processing of parameters of multiple information sets, the following can be adopted. That is, when the level of the audio signal input from each audio signal adjustment unit 3a, 3b, ... 3n is equal to or lower than a preset threshold, the level value is set as the audio signal level value, and when the level value of the audio signal exceeds the threshold, the threshold value of the audio signal is set as the audio signal level value, a gain share coefficient is calculated based on the sum of the level values of each audio signal, and the audio adjustment parameters are corrected based on the gain coefficient.

[1-3-2. Gain reduction value detection unit 17]
The audio signal processing device includes a gain reduction value detection unit 17 that detects a gain reduction value of a compressor for an audio signal after adjustment processing. When the gain reduction value after adjustment processing is outside a preset threshold, the gain reduction value detection unit 17 searches for a plurality of information sets in which information on an audio signal input to a device having a search keyword common to the audio signal after adjustment processing and information on a gain reduction value for the audio signal after adjustment processing are linked together, from information on audio signals input to and output from a plurality of devices.

In this case, the adjustment control unit 20 performs a calculation process on information regarding the gain reduction value of the audio signal that has been subjected to the adjustment process, which is included in the extracted multiple information sets, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on parameters that correct the gain reduction value that deviates from the threshold value so as to obtain the gain reduction value obtained as the calculation result.

[1-3-3. Direct sound/indirect sound balance comparison unit 18]
The balance comparison unit 18 compares the direct sound and the indirect sound of the audio signal after the adjustment process to detect whether the balance between the direct sound and the indirect sound after the adjustment process is outside a preset threshold value. If it is outside the threshold value, the balance comparison unit 18 searches for a plurality of information sets in which information about an audio signal input to a device having a search keyword common to the audio signal after the adjustment process is linked to information about the balance between the direct sound and the indirect sound for the audio signal after the adjustment process, from information about audio signals input to and output from a plurality of devices.

In this case, the adjustment control unit 20 performs a calculation process on information regarding the balance between the direct sound and indirect sound of the adjusted audio signal contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so as to achieve the balance obtained as a result of the calculation, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on the corrected parameters. For example, the direct sound is convolved with any impulse data contained in the extracted information set, and the delay and reverb time of the indirect sound are adjusted to achieve this characteristic.

[1-3-4. Mixing balance comparison unit 19 for multiple audio signals]
The audio signal mixing unit 4 includes a mixing balance comparison unit 19 that compares the mixing balances of the multiple audio signals after the adjustment process. The mixing balance comparison unit 19 searches for multiple information sets in which information on audio signals input to and output from multiple devices is linked to information on audio signals input to a device having a search keyword common to the audio signal on which the adjustment process has been performed, and information on the mixing balance when the audio signal mixing unit 4 mixes the multiple audio signals on which the adjustment process has been performed.
In this case, the adjustment control unit 20 performs calculations on the information regarding the mixing balance contained in the extracted multiple information sets, corrects the parameters on which the adjustment processing was performed so that the mixing balance obtained as a result of the calculation is achieved, and has the audio signal adjustment units 3a, 3b, ... 3n perform audio adjustment based on the corrected parameters.

As with the automatic adjustment of the output level, when correcting the parameters in the mixing balance comparison unit 19, instead of or in addition to the method of arithmetic processing of the parameters of multiple information sets, the following method can also be adopted. That is, when the mixing balance of the audio signals input from each audio signal adjustment unit 3a, 3b, ... 3n is equal to or less than a preset threshold, the mixing ratio is set to the mixing ratio of the audio signals, and when the mixing balance of the audio signals exceeds the threshold, the threshold of the audio signals is set to the mixing ratio of the audio signals. The mixing balance is calculated based on the sum of the mixing ratios of each audio signal, and the audio adjustment parameters are corrected based on the mixing balance.

[1-3-5. EQ/FIL detection unit 21]
The EQ/FIL detection unit 21 detects the equalizing effect and the filtering effect on the audio signal after the adjustment process, and detects whether the effect after the adjustment process is outside a preset threshold value. If it is outside the threshold value, the EQ/FIL detection unit 21 searches for a plurality of information sets in which information on an audio signal input to a device having a search keyword common to the audio signal after the adjustment process is linked to information on the equalizing effect and the filtering effect on the audio signal after the adjustment process, from information on audio signals input to and output from a plurality of devices.
In this case, the adjustment control unit 20 performs calculations on the information regarding the equalizing effect and filtering effect contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so that the equalizing effect and filtering effect obtained as a result of the calculation are obtained, and has the audio signal adjustment units 3a, 3b, ... 3n perform audio adjustment based on the corrected parameters.

[2. Effects]
According to the embodiment having the above-mentioned configuration, the following effects are achieved.

[2-1. Basic Effects]
(1) The information input unit 7 inputs information about the devices connected to the audio adjustment console, information about audio signals input/output to the devices, and information about parameters of audio adjustment performed on the audio signals by the audio signal adjustment units 3a, 3b, and 3n. That is, for audio adjustment, information about the devices connected to the audio adjustment console is required, such as the number and arrangement of sound sources such as microphones, output speakers and recording devices, characteristics of the devices themselves, and the purpose of the output audio. In addition, the parameters of audio adjustment differ depending on what kind of performance or performer the audio signal to be adjusted is performed by, the location or environment where it is input or output, whether it is a human voice or another sound source such as an instrument or CD, and further whether it is the main audio or background music.

In this embodiment, the combination of information about the devices connected to these audio adjustment consoles, information about the audio signals input and output to the devices, and the parameters used when an operator or an automatic audio adjustment device performs audio adjustment on such combinations are linked together and saved as a single information set. As a result, a large amount of information about past audio adjustments is stored in the memory unit 13, and when new audio adjustments are to be performed, audio adjustments similar to those performed in the past can be automatically performed simply by providing the audio adjustment console with the type and state of the audio signal to be adjusted and information about the devices that input and output the audio signal.

As a result, by implementing AI in the library function of the audio adjustment console, it becomes possible to call up appropriate settings. In addition, by accumulating the know-how of many operators (for example, voices that are easy for viewers to listen to), it is possible to further stabilize program quality. Furthermore, by analyzing and statistically converting the many accumulated information sets, it is possible to visualize audio adjustment technology, which is effective for human resource development and archiving. This makes it possible to ensure stable viewer service with program quality, while also reforming the way production staff work, and improving program production efficiency.

(2) In this embodiment, a storage keyword generation unit 12 is provided that extracts multiple values related to each piece of information included in each information set generated by the information set generation unit 11 according to a predetermined criterion, and sets each extracted value as a storage keyword for that information set. Therefore, storage keywords can be created by automatically extracting information related to a large number of programs broadcast daily by broadcasting stations and other organizations, and information such as parameters for audio adjustments performed in relation to that information. As a result, a large amount of training data required to build an AI-based search system can be easily created.

(3) The audio signal adjustment units 3a, 3b, ... 3n have a search keyword generation unit 14 that extracts multiple values contained in each piece of information input to the information input unit 7 when the information adjustment units 3a, 3b, ... 3n perform audio adjustment, based on predetermined criteria, and sets each extracted value as a keyword for searching the information.
Therefore, search keywords can be automatically generated from names and numbers written in commonly used materials such as a list of equipment to be used, a list of performers, a program progress chart, etc. As a result, it is possible to automatically extract search keywords from the large amount of program-related information that is carried out daily by broadcasting stations, etc., and when new audio adjustment is carried out, it is no longer necessary to manually select search keywords and input them into the audio adjustment console, reducing the effort required for new audio adjustment.

(4) When performing audio adjustment, even for the same performers and performances, the parameters for audio adjustment will differ depending on the characteristics of the equipment used. In this embodiment, however, information about the equipment connected to the audio adjustment desk, such as the characteristics of input devices 1a, 1b, ... 1n, such as microphones and speakers, and output device 6, data format, number of bits, sampling frequency, instrument type, etc., are also used as reference for parameter determination, so that optimal audio adjustment can be performed to suit input devices 1a, 1b, ... 1n, and output device 6.

(5) The information about the audio signal includes information about the user of the equipment connected to the audio adjustment desk and information about the recording of the audio signal, so various information associated with the input audio signal, such as the type and gender of the performers, the program, and the environment, such as the studio or hall used, can be appropriately registered or called up and used. In particular, this embodiment can be applied when mixing multiple sound sources, such as multiple announcer comments, music, and VTRs, as required by broadcasting stations. As a result, it becomes possible to automatically perform audio adjustment that comprehensively reflects all the various elements required for audio adjustment without omission, and even an operator with little experience can perform audio adjustment that is comparable to that of an experienced operator.

(6) The information on the audio adjustment parameters includes the types and values of various parameters, information on the operator who set the parameters, and information on alarms generated by the operator or automatically when the parameters were set. This makes it possible to confirm not only the parameter values set during audio adjustment, but also the circumstances and operator by which the parameters were set. As a result, by referring to various information, more appropriate guidelines can be obtained when setting new parameter values or correcting already-set parameter values.

In particular, in addition to the parameters used in conventional audio adjustment (e.g. ratio, threshold, attack, release, frequency, etc.), it is now possible to use subjective parameters (e.g. clarity, clarity, thickness, comment emphasis, senior voice, etc.) as information on audio adjustment parameters, which can assist in audio adjustment to match human senses.

(7) The information set search unit 15 extracts multiple information sets in which the search keyword and the storage keyword match according to a predetermined criterion, performs calculations on multiple voice adjustment parameters contained in the extracted multiple information sets, and the adjustment control unit performs voice adjustment based on the parameters obtained as a result of the calculation. Therefore, if only a single information set is extracted, even if the degree of match between the storage keyword and the search keyword is high, the mismatch may have a significant impact on the selection of parameters, and therefore it may not be possible to select optimal parameters. However, as in this embodiment, by calculating parameters contained in multiple information sets by means such as averaging, median selection, and weighting the names of performers, programs, and past operators, the possibility of extreme parameters being selected is reduced, and high-quality voice adjustment is possible.

(8) A program progress status extraction unit 10 is provided which extracts information on the devices connected to the audio adjustment console, information on the audio signals input and output to the devices, and changes in the program progress over time from the information listed in the progress chart, linking them together. The information set search unit 15 also extracts different information sets in response to changes in the program progress over time input from the progress chart input unit 9. Therefore, even if the performers, programs, input devices 1a, 1b, ... 1n, and output devices 6 change according to changes in the program progress over time, by automatically selecting different information sets according to the program progress, appropriate audio adjustments can be automatically made according to the program progress without the operator's operation.

[2-2. Effects of automation configuration]
(9) The output level detection unit 16 has the function of notifying the audio signal input unit 2a, 2b, ... 2n of an abnormality in the audio signal input when the signal level after the adjustment process is lower than a preset threshold value. Therefore, the operator can easily discover the cause of the input abnormality by checking the status of each device connected to the audio adjustment desk and the performers using each device.

(10) The output level detection unit 16 performs a calculation process on the information related to the output level of the audio signal contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so as to obtain the output level obtained as a result of the calculation, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on the corrected parameters.

Therefore, even if audio adjustment based on a certain information set results in a decrease in the output level, new parameters can be generated by extracting multiple similar information sets and performing calculations such as averaging the parameters contained in those information sets or taking the median. As a result, it becomes possible to correct the parameters, and by repeating this process, it becomes possible to automatically correct the output level of the audio signal to an appropriate state.

(11) The gain reduction value detection unit 17 performs calculations on information related to the gain reduction value of the compressor of the audio signal that has been subjected to the adjustment process, which is included in the extracted multiple information sets, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on parameters that correct the gain reduction value that deviates from the threshold value so as to obtain the gain reduction value obtained as a result of the calculation.

Therefore, even if the compressor's gain reduction value deviates from the threshold as a result of audio adjustment based on a certain information set, a new gain reduction value can be generated by extracting multiple similar information sets and performing calculations such as averaging the gain reduction values contained in those information sets or taking the median. As a result, it becomes possible to correct the parameters based on the new gain reduction value, and by repeating this process, it becomes possible to automatically correct the parameters of the audio signal to an appropriate state.

(12) When the balance between the direct sound and the indirect sound after the adjustment process deviates from a preset threshold, the balance comparison unit 18 performs calculations on information regarding the balance between the direct sound and the indirect sound of the audio signal on which the adjustment process has been performed, corrects the parameters on which the adjustment process has been performed so as to achieve the balance obtained as a result of the calculation, and causes the audio signal adjustment units 3a, 3b, ... 3n to perform audio adjustment based on the corrected parameters.

Therefore, even if the balance between direct and indirect sounds deviates from a preset threshold as a result of audio adjustment based on a certain information set, a new balance between direct and indirect sounds can be generated by extracting multiple similar information sets and performing calculations such as averaging the balance between direct and indirect sounds contained in those information sets or taking the median. As a result, it becomes possible to correct the balance between direct and indirect sounds, and by repeating this process, it becomes possible to automatically correct the balance between direct and indirect sounds in the audio signal to an appropriate state.

(13) The mixing balance comparison unit 19 searches for multiple information sets linked to information about the mixing balance when the audio signal mixing unit 4 mixes multiple audio signals that have been subjected to adjustment processing, and obtains the mixing balance to be used when mixing the audio by averaging the information about the mixing balance contained in the multiple extracted information sets or taking the median. Therefore, in addition to the audio adjustment processing, the accumulated information can be used to obtain an appropriate mixing balance when mixing the processed audio signals, and a series of processes from audio adjustment to audio mixing can be automated.

(14) When the equalizing effect or filtering effect on the audio signal after adjustment processing deviates from a preset threshold, the EQ/FIL detection unit 21 extracts multiple information sets, performs calculations on the information on the equalizing effect or filtering effect contained therein, and corrects the parameters based on the calculation results. Therefore, the audio signal adjustment units 3a, 3b, ... 3n can automatically obtain an appropriate equalizing effect or filtering effect by controlling the equalizer or filter based on the corrected parameters.

3. Other embodiments
The present invention is not limited to the above-described embodiment, but also includes other embodiments such as the following.
(1) In the embodiment, the information set generation unit 11, the storage keyword generation unit 12, and the search keyword generation unit 14 are configured so that the audio control console automatically generates the information sets and each keyword, but in addition, the information sets and each keyword may also be generated manually.

(2) The information set is generated based on information about the devices connected to the audio adjustment console, information about the audio signals input and output to the devices, and information about the parameters of the audio adjustment performed on the audio signals by the audio signal adjustment units 3a, 3b, and 3n. In addition to this information, other information, such as operator comments or third-party evaluations, may be input to the information input unit 7, and the information set may be generated or searched based on that information.

(3) As for the training data, in addition to the type of growth type described in the embodiment, specific condition growth type or simple growth type can be used by appropriately setting the conditions to be given to the information set generation unit 11 and the saved keyword generation unit 12.

1a, 1b, 1n...Input device 2a, 2b, 2n...Audio signal input section 3a, 3b, 3n...Audio signal adjustment section 4...Audio signal mixing section 5...Audio signal output section 6...Output device 7...Information input section 8...Device, audio signal, parameter information input section 9...Progress chart input section 10...Program progress extraction section 11...Information set generation section 12...Keyword for storage generation section 13...Memory section 14...Keyword for search generation section 15...Information set search section 16...Output level detection section 17...Gain reduction value detection section 18...Direct sound and indirect sound balance comparison section 19...Mixing balance comparison section 20...Adjustment control section 21...EQ/FIL detection section

Claims (10)

an audio signal input unit to which a plurality of audio signals are input, a plurality of audio signal adjustment units that adjust the input plurality of audio signals based on predetermined parameters, an audio signal mixing unit that mixes the audio signals output from the plurality of audio signal adjustment units, and an audio signal output unit that outputs the audio signal mixed by the audio signal mixing unit;
an information input unit for inputting information on a device connected to the audio adjustment console, information on an audio signal input/output to the device, and information on parameters of audio adjustment performed on the audio signal by the audio signal adjustment unit;
an information set generation unit that generates one information set by linking each piece of information input to the information input unit to each other so as to include information regarding parameters of audio adjustments performed on the audio signal;
a storage keyword generation unit that extracts a plurality of values related to each piece of information inputted to the information input unit and/or each piece of information included in each piece of information set generated by the information set generation unit based on a predetermined criterion, and sets each of the extracted values as a storage keyword for the piece of information set;
a storage unit that associates each of the information sets with the saved keywords and saves them;
a search keyword generating unit that extracts a plurality of values related to each piece of information from at least one of information related to the device connected to the audio adjustment console, which is input to the information input unit when the audio signal adjustment unit performs audio adjustment, and information related to the audio signal input/output to the device, based on a predetermined criterion, and sets each of the extracted values as a search keyword;
an information set search unit that searches for an information set stored in the storage unit using the search keyword and the storage keyword, and extracts an information set in which the search keyword and the storage keyword match a predetermined criterion;
an adjustment control unit that acquires information about parameters of sound adjustment included in the information set extracted by the information set search unit, and causes the sound signal adjustment unit to perform sound adjustment based on the acquired parameters;
having
The adjustment control unit is
A signal output from the audio signal adjustment unit after the audio adjustment process based on the acquired parameters is input, and at least one of a signal level of the input signal , a gain reduction value of a compressor, a balance between direct sound and indirect sound, and a mixing balance of a plurality of audio signals is detected;
an audio adjustment console that corrects the parameters used for the audio adjustment process when the detection result deviates from a preset value, and causes the audio signal adjustment section to perform the audio adjustment based on the corrected parameters. The information set search unit extracts a plurality of information sets in which the search keyword and the storage keyword match a predetermined criterion;
The audio adjustment console according to claim 1, wherein the adjustment control unit performs calculation processing on a plurality of audio adjustment parameters contained in the plurality of extracted information sets, and causes the audio signal adjustment unit to perform audio adjustment based on the parameters obtained by the calculation processing . The information input unit includes:
a progress table input unit for receiving a progress table of a program for which the audio signal adjustment unit is to perform audio adjustment;
a program progress status extraction unit that extracts information on the device connected to the audio control console, information on the audio signal input/output to the device, and a time-dependent change in program progress from the information described in the progress table, and links the information to the device,
the information set search unit extracts different information sets in response to a change over time in the program progress input from the progress table input unit;
The audio adjustment desk according to claim 1 or claim 2, wherein the adjustment control unit performs different audio adjustments in accordance with changes over time in the progress of a program based on audio adjustment parameters contained in different information sets extracted by the information set search unit. An audio adjustment console according to any one of claims 1 to 3, further comprising an output level detection unit that performs an adjustment process on the input audio signal based on the acquired parameters in response to an instruction from the adjustment control unit, and detects whether the signal level after the adjustment process is lower than a certain level. 5. The audio mixing console according to claim 4, wherein the output level detection section has a function of notifying an abnormality in the input of the audio signal to the audio signal input section when the signal level after the adjustment process is lower than a preset threshold value. When the signal level after the adjustment process is lower than a preset threshold value, the output level detection unit
searching for a plurality of information sets in which information on the audio signal input to the device having a search keyword common to the audio signal on which an adjustment process has been performed based on the acquired parameters and information on an output level of the audio signal on which an adjustment process has been performed are linked, from information on the audio signals input to the device and input to the device;
The audio adjustment console of claim 4, wherein the adjustment control unit performs a calculation process on information regarding the output level of the audio signal contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so as to achieve the output level obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters. a gain reduction value detection unit that detects a gain reduction value of a compressor for the audio signal after the adjustment process;
When the gain reduction value after the adjustment process is out of a preset threshold, the gain reduction value detection unit
searching for a plurality of information sets in which information on the audio signal input to the device having a search keyword common to the audio signal on which an adjustment process has been performed based on the acquired parameters and information on a gain reduction value for the audio signal on which an adjustment process has been performed are linked from information on the audio signals input to the device and input to the device;
The adjustment control unit performs a calculation process on information regarding the gain reduction value of the audio signal on which the adjustment process has been performed, which is contained in the extracted multiple information sets, and causes the audio signal adjustment unit to perform audio adjustment based on parameters that correct the gain reduction value that deviates from the threshold value so as to obtain the gain reduction value obtained as the calculation result. a balance comparison unit that compares a direct sound and an indirect sound of the audio signal after the adjustment process,
When the balance between the direct sound and the indirect sound after the adjustment process deviates from a preset threshold, the balance comparison unit
searching for a plurality of information sets linking information on an audio signal input to the device having a search keyword common to an audio signal that has been subjected to an adjustment process based on the acquired parameters with information on the balance between direct sound and indirect sound for the audio signal that has been subjected to the adjustment process, from information on audio signals input to and output from the device;
The adjustment control unit performs a calculation process on information regarding the balance between direct sound and indirect sound of the audio signal for which the adjustment process has been performed, which is included in the extracted multiple information sets, corrects the parameters on which the adjustment process has been performed so as to achieve the balance obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters. the audio signal mixer includes a mix balance comparator that compares mix balances of the plurality of audio signals after the adjustment process,
the mixing balance comparison unit searches for a plurality of information sets in which information on the audio signals input to and output from the device, the information on the audio signals having a search keyword common to the audio signals on which adjustment processing has been performed based on the acquired parameters, and information on the mixing balance when the audio signal mixing unit mixes the plurality of audio signals on which adjustment processing has been performed, are linked together;
The adjustment control unit performs a calculation process on information regarding the mixing balance by the audio signal mixing unit contained in the extracted multiple information sets, corrects the parameters on which the adjustment process was performed so as to achieve the mixing balance obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters. an EQ/FIL detection unit for detecting an equalizing effect and/or a filtering effect on the audio signal after the adjustment processing;
The EQ/FIL detection unit detects whether the equalizing effect and/or the filtering effect after the adjustment process deviates from a preset threshold value, and when the equalizing effect and/or the filtering effect deviates from the threshold value, selects, from among information about audio signals input/output to a plurality of devices, information about an audio signal input to a device having a search keyword common to the audio signal that has been subjected to the adjustment process;
retrieving a plurality of sets of information associated with information regarding the equalization effect and/or filtering effect of the adjusted audio signal;
The adjustment control unit performs a calculation process on information regarding equalizing effects and/or filtering effects contained in the extracted information sets, corrects parameters used for the adjustment process so as to obtain the equalizing effect or filtering effect obtained as a result of the calculation, and causes the audio signal adjustment unit to perform audio adjustment based on the corrected parameters.