JP5373552B2 - Swell detection device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a climax detector and a program, capable of detecting a climax section of a musical piece waveform data. <P>SOLUTION: The climax detector includes: a smoothing unit (3401) for smoothing a musical piece waveform data; a minimum point-detecting unit for detecting the minimum point of the musical piece waveform data which is smoothed by the smoothing unit; and a climax section-detecting unit (3403) for detecting a section where a maximum value in a section between minimum points is a threshold value or more, as the climax section, on the basis of the minimum points detected by the minimum point detecting unit. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a swell detector and a program.

  The following Patent Document 1 is known as a tempo detection device that detects a beat position from a music acoustic signal (audio signal) mixed with a plurality of instrument sounds such as a music CD. In Patent Document 1, as a beat position detection method, an input waveform is subjected to an FFT calculation at a predetermined time interval (hereinafter referred to as a frame), the power of each scale sound is obtained from the obtained power spectrum, and the power of each scale sound is calculated. The increment value for each frame is calculated, and this is summed with all the scales to obtain the overall sound change level for each frame, and the autocorrelation of the overall sound change level for each frame is calculated to calculate the periodicity. The average beat interval (so-called tempo) is obtained from the frame interval at which the autocorrelation value is maximum.

  Further, in Patent Document 2 below, the tapping detection unit is used while playing the head of the beat detection waveform, and the beat position is tapped by the user, and the tapping fluctuation is within a certain range in the fluctuation calculation unit. A tempo detection device is described that selects a beat interval that is numerically close to the tapping tempo from beat interval candidates detected by a tempo candidate detection unit.

JP 2007-52394 A JP 2008-40284 A

  In addition to detecting the beat position, it is desired to detect a rising section in order to synchronize with music more. However, there has not been a device for detecting a rising section of music waveform data until now.

  An object of the present invention is to provide a swell detector and a program that can detect a swell section of music waveform data.

The swell detection device of the present invention includes a smoothing unit that smoothes curved waveform data, a minimum point detection unit that detects a minimum point of the curved waveform data smoothed by the smoothing unit, and a minimum point detection unit. possess a raised section detecting unit that the maximum value in the interval between the minimum point based on the detected local minima are detected as a section climax the threshold or more sections, the smoothing unit smoothing the music waveform data A first smoothing unit that performs the second smoothing unit that smoothes the curved waveform data smoothed by the first smoothing unit, and the minimum point detection unit includes the second smoothing unit. The minimum point of the curved waveform data smoothed by the smoothing unit is detected, and the swell section detecting unit is configured to check the minimal check based on the amount of change in the curved waveform data smoothed by the first smoothing unit. The start point of the interval between the minimum points detected by the exit and Fixed point, and detecting the climax section.
Further, the swell detection device of the present invention includes a smoothing unit that smoothes curved waveform data, a minimum point detection unit that detects a minimum point of the curved waveform data smoothed by the smoothing unit, and the minimum point detection. A swell section detecting unit that detects a section having a maximum value within a section between the minimum points as a swell section based on the minimum point detected by the unit, and the minimum point detecting unit includes the smoothing unit. If the minimum point of the curved waveform data smoothed by the above is detected and the detected minimum point is larger than the threshold ratio with respect to the maximum point before or after the detected minimum point, the minimum point is excluded. It is characterized by detecting a minimum point.
Further, the swell detection device of the present invention includes a smoothing unit that smoothes curved waveform data, a minimum point detection unit that detects a minimum point of the curved waveform data smoothed by the smoothing unit, and the minimum point detection. And a swell section detecting unit that detects a section having a maximum value in a section between the minimum points as a swell section based on the minimum point detected by the section, and the swell section detecting unit detects the detected 2 When the value of a section between two rising sections is larger than a threshold ratio with respect to the value of the two detected rising sections, the two detected rising sections are connected to one rising section.
Further, the swell detection device of the present invention includes a smoothing unit that smoothes curved waveform data, a minimum point detection unit that detects a minimum point of the curved waveform data smoothed by the smoothing unit, and the minimum point detection. A climax section detection unit that detects a climax section that has a maximum value in a section between the maximal points based on the minimal point detected by the section as a climax section. If the length of the detected climax section is equal to or less than the threshold ratio with respect to the climax section having the maximum length among the sections, the climax section is detected by excluding the climax section.

The program of the present invention includes a smoothing step for smoothing the curved waveform data, a minimal point detecting step for detecting a minimal point of the curved waveform data smoothed by the smoothing step, and a minimal point detecting step. A program for causing a computer to execute a rising section detecting step for detecting a section having a maximum value in a section between minimum points as a rising section based on the detected minimum points , and the smoothing step A first smoothing step for smoothing the curved waveform data; and a second smoothing step for smoothing the curved waveform data smoothed by the first smoothing step, wherein the minimum inspection is performed. In the output step, the minimum point of the curved waveform data smoothed by the second smoothing step is detected. Based on the change amount of the curved waveform data smoothed by the first smoothing step, the start point and the end point of the section between the minimum points detected by the minimum point detection step are corrected, and the rising section is detected. Features .
The program of the present invention includes a smoothing step for smoothing the curved waveform data, a minimal point detecting step for detecting a minimal point of the curved waveform data smoothed by the smoothing step, and a minimal point detecting step. A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section in which a maximum value in a section between the minimum points is greater than or equal to a threshold value based on the detected maximal point, the minimum point detecting step Then, the local minimum point of the curved waveform data smoothed by the smoothing step is detected, and if the detected local minimum point is larger than a threshold ratio with respect to the local maximum point before or after the detected local minimum point, It is characterized in that a minimum point is detected by excluding the minimum point.
The program of the present invention includes a smoothing step for smoothing the curved waveform data, a minimal point detecting step for detecting a minimal point of the curved waveform data smoothed by the smoothing step, and a minimal point detecting step. A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section whose maximum value in a section between the minimum points is greater than or equal to a threshold value based on the detected maximal point, the climax section detecting step Then, when the value of the section between the two detected rising sections is larger than the threshold ratio with respect to the value of the two detected rising sections, the two detected rising sections are connected to one rising section. It is characterized by that.
The program of the present invention includes a smoothing step for smoothing the curved waveform data, a minimal point detecting step for detecting a minimal point of the curved waveform data smoothed by the smoothing step, and a minimal point detecting step. A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section whose maximum value in a section between the minimum points is greater than or equal to a threshold value based on the detected maximal point, the climax section detecting step Then, with respect to the climax section having the maximum length among the climax sections detected, if the length of the climax section detected is equal to or less than the threshold ratio, the climax section is detected by excluding the climax section. Features.

  According to the present invention, a rising section of music waveform data can be detected.

It is a flowchart which shows the swelling detection method of the swelling detection apparatus by the 1st Embodiment of this invention. It is a figure which shows music waveform data. It is a figure which shows the absolute value of the amplitude of music waveform data. It is a figure which shows the 1st moving average. It is a figure which shows downsampling. It is a figure which shows a 2nd moving average. 7A to 7C are diagrams illustrating correction of the minimum point. It is a figure which shows correction | amendment of a minimum point. FIGS. 9A and 9B are diagrams showing candidates for the exciting section. FIGS. 10A and 10B are diagrams showing connections in the rising section. FIGS. 11A and 11B are diagrams showing exclusion of candidates for a short climax section. 6 is a flowchart illustrating an intro and ending detection method according to a second embodiment of the present invention. It is a flowchart which shows the detail of a fade-in (out) detection process. It is a flowchart which shows the detail of a fade-in (out) detection process. It is a flowchart which shows the detail of detection processing without a drum (a cappella etc.). It is a flowchart which shows the detail of a loud sound (intro) detection process. It is a flowchart which shows the detail of a loud sound (ending) detection process. It is a figure which shows music waveform data. It is a figure which shows the arrangement | sequence data of a 1 second space | interval. It is a figure which shows the end point detection process of fade-in. It is a figure which shows the increase amount from a front frame. It is a figure which shows the detection process of a rapid increase. It is a figure which shows the change process of the end point of fade-in. It is a figure which shows the detection process of the starting point of fade-in. It is a figure which shows the amplitude detection process before a fade-in area. FIGS. 26A to 26C are diagrams showing the fast Fourier transform process. It is a figure which shows the power detection process of a low sound range. It is a figure which shows the starting point detection process of a drum no area. It is a figure which shows the power detection process of a drum no area. It is a figure which shows the end point detection process of a loud sound area. It is a figure which shows the starting point detection process of a loud sound area. It is a figure which shows a fast Fourier-transform process. It is a block diagram which shows the hardware structural example of a swell detector (computer). It is a block diagram for demonstrating the function of the program of a swell detector (computer).

(First embodiment)
FIG. 33 is a block diagram illustrating a hardware configuration example of the swell detection device (computer) according to the first embodiment of the present invention. The swell detection device is composed of, for example, a personal computer. A central processing unit (CPU) 3302, a ROM 3303, a RAM 3304, a network interface 3305, an input device 3306, an output device 3307, and an external storage device 3308 are connected to the bus 3301.

  The CPU 3302 processes or calculates data and controls various components connected via the bus 3301. The ROM 3303 stores a control procedure (computer program) of the CPU 3302 in advance, and is activated when the CPU 3302 executes the computer program. A computer program is stored in the external storage device 3308, and the computer program is copied to the RAM 3304 and executed. The RAM 3304 is used as a work memory for data input / output, transmission / reception, and temporary storage for control of each component. The external storage device 3308 is, for example, a hard disk storage device or a CD-ROM, and the stored content does not disappear even when the power is turned off. The CPU 3302 executes the computer program in the RAM 3304 to perform the processes of FIGS. 1 and 12 to 17.

  A network interface 3305 is an interface for connecting to a network such as the Internet. The input device 3306 is, for example, a keyboard and a mouse, and can give various instructions or inputs. The output device 3307 is a display device (display), a printing device (printer), a speaker, and the like.

  FIG. 1 is a flowchart showing a method for detecting the swell of the swell detector according to this embodiment. The processing shown in FIG. 1 is performed by the CPU 3302 executing the program.

  In step S101, the CPU 3302 loads music waveform data (WAV data) from the external storage device 3308 to the RAM 3304, and performs monaural processing and normalization processing. Specifically, if the music waveform data is stereo, the CPU 3302 converts the stereo music waveform data into monaural music waveform data. Next, the CPU 3302 normalizes the music waveform data to match the maximum amplitude of the music waveform data with the maximum value of the dynamic range, and generates the music waveform data of FIG.

  Next, in step S102, the CPU 3302 smoothes the absolute value of the waveform amplitude using a moving average or the like in order to digitize the swell. Note that the climax may be quantified based on the sum of power as a result of fast Fourier transform (FFT), the sum of log of power, or the like. Specifically, as shown in FIG. 3, the CPU 3302 converts the amplitude of the music waveform data into an absolute value. Next, the CPU 3302 takes a moving average for 1 second before and after the data of FIG. 3 to generate the smooth data of FIG. In FIG. 3, since the horizontal axis is reduced, there is actually a small value between the peaks of the curved waveform data, so when taking the average, the overall shape is lowered as shown in FIG.

  Next, in step S103, the CPU 3302 performs further smoothing again using a moving average or the like. Since the desired smoothness cannot be obtained by only one moving average in step S102, the second moving average is performed in step S103. Specifically, as shown in FIG. 5, the CPU 3302 reduces the number of samples by thinning out the data in FIG. For example, the number of samples at 44.1 kHz is reduced to the number of samples at 100 Hz. Even if the number of samples is reduced, since FIG. 4 and FIG. 5 are plotted at intervals of 0.1 second, the difference does not appear clearly. Next, the CPU 3302 takes a moving average in 1 second before and after the data of FIG. 5 to generate smooth data of FIG.

  Next, in step S104, the CPU 3302 searches for the minimum point in the data of FIG. 6, and if there is not much difference compared to the one with the smaller value of the front and rear maximum points, the minimum value between the maximum points is determined. Correct the value as the smaller of and eliminate the local minimum. This is to facilitate recognition of the moving average result as a section. Specifically, as shown in FIG. 7A, the CPU 3302 corrects data around a local minimum point 702 with a small change for the purpose of deleting fine irregularities. The CPU 3302 searches for the local minimum point 702 and compares it with the previous local maximum point 701 and the subsequent local maximum point 703, and the value of the local minimum point 702 is larger than 90% of the smaller ones of the front and rear local maximum points 701 and 703. Since the unevenness is small, the data between the maximum points 701 and 703 is raised to the smaller value of the maximum point as shown in FIG. The data of (C) is generated. FIG. 8 shows data obtained as a result of the processing in step S104, and fine irregularities are removed from the data in FIG.

  Next, in step S105, the CPU 3302 extracts all the points between the minimum points as candidates for the climax section based on the data in FIG. Specifically, the CPU 3302 first records the interval between the minimum points as a climax section 901 as shown in FIG.

  Next, in step S106, the CPU 3302 uses the first smoothing data shown in FIG. 4, and the starting point is the position where the numerical value changes the most until the maximum value first appears from the starting point of the rising section. To correct. Specifically, since the CPU 3302 adjusts the position of the start point for each recorded section 901 in FIG. 9A, as shown in FIG. 9B, the first maximum point 902 from the beginning of the section 901 is displayed. The position 903 having the largest change amount from the previous data is corrected to the start point of the section.

  Next, in step S107, the CPU 3302 adjusts the position of the end point for each of the recorded sections 901 in FIG. 9A in the same manner as the start point from the last maximum point of the section 901 to the section 901. The position with the largest change amount to the next data until the end is corrected to the end point of the section 901.

  Next, in step S108, the CPU 3302 obtains the maximum value of the climax in order to select the climax section, and only those having the section maximum value of a certain ratio or more are set as the climax section. The sections may be selected using the section average value, the section total, a data ratio equal to or higher than the threshold value in the section, and the like.

  Specifically, the CPU 3302 obtains the maximum value among all candidates for the climax section 901. Next, when the maximum value of each of the climax sections is less than N% (N = 65 for the first time) of all the above candidates, the CPU 3302 is not a climax section (a small peak on the data waveform). ) And remove it from the candidate for the exciting section. Next, the CPU 3302 calculates the total length of those remaining as climax sections candidates with respect to the overall length, and if it is 33% or more, the CPU 3302 determines that there are still too many climax sections candidates, and the above N Is increased by 5 and the above process is repeated to reduce the number of candidates for the exciting section.

  Next, as shown in FIG. 10A, when the candidates for the climax section are narrowed to some extent, the CPU 3302 determines that the value of the section 1002 between the climax section 1001 and the next climax section 1003 is large to some extent. As shown in FIG. 10B, the two rising sections 1001 and 1003 are connected to one rising section 1004. For example, the CPU 3302 determines that the value of the section 1002 between the climax section 1001 and the next climax section 1003 is 75 smaller than the smaller value of the end value of the climax section 1001 and the first value of the next climax section 1003. If it is equal to or greater than%, it is determined that the section 1002 between the rising sections 1001 and 1003 is also rising, and is combined into one rising section 1004.

  Next, the CPU 3302 obtains the maximum length among the climax sections, and removes candidates that are 30% or less of the maximum length in each climax section. For example, in FIG. 11A, the lengths of the sections 1101 and 1102 are 30% or less of the maximum length, so they are excluded from the candidates. As shown in FIG. 11B, the sections 1103 and 1104 are finally raised sections. Detected as

  As described above, according to the present embodiment, it is possible to detect a rising section of music waveform data. As a result, by reproducing the image (still image or moving image) data in synchronization with the rising section of the music waveform data, the music and the image can be reproduced while being synchronized. For example, it is possible to perform a climax reproduction process such as moving a still image in a climax section of music waveform data. Thereby, it can synchronize with the climax section of the music waveform data and can give the climax effect to the image.

  FIG. 34 is a block diagram for explaining the function of the program of the computer of FIG. When the computer executes the program, a swell detection device (method) is realized. The swell detection device (method) 3400 includes a smoothing unit (step) 3401, a minimum point detection unit (step) 3402, and a swell section detection unit (step) 3403. The smoothing unit 3401 includes a first smoothing unit (step) 3411 and a second smoothing unit (step) 3412.

  A smoothing unit 3401 smoothes the waveform data corresponding to steps S102 and S103. The minimum point detection unit 3402 detects the minimum point of the curved waveform data smoothed by the smoothing unit 3401 corresponding to step S104. In response to steps S <b> 105 to S <b> 108, the climax section detection unit 3403 detects a section in which the maximum value in the section between the minimum points is greater than or equal to the threshold based on the minimum points detected by the minimum point detection unit 3402 as a climax section.

  Specifically, the first smoothing unit 3411 corresponds to step S102 and smoothes the curved waveform data. The second smoothing unit 3412 corresponds to step S103 and smoothes the curved waveform data smoothed by the first smoothing unit 3411. The minimum point detection unit 3402 detects the minimum point of the curved waveform data smoothed by the second smoothing unit 3412. The climax section detection unit 3403 corresponds to steps S106 and S107, and is based on the amount of change in the curved waveform data smoothed by the first smoothing unit 3411, between the minimum points detected by the minimum point detection unit 3402. Correct the start point and end point of the section, and detect the rising section.

  As shown in FIGS. 7A to 7C, the local minimum point detection unit 3402 detects the local minimum point of the curved waveform data smoothed by the smoothing unit 3401, and the local minimum point from which the detected local minimum point is detected is detected. If the maximum point before or after the point is larger than the threshold ratio, the minimum point is excluded and the minimum point is detected.

  As shown in FIGS. 10 (A) and 10 (B), the climax section detection unit 3403 uses a threshold ratio for the values of the two climax sections in which the value of the section between the two climax sections detected is detected. When it is larger, the two detected rising sections are connected to one rising section.

  In addition, as shown in FIGS. 11A and 11B, the climax section detection unit 3403 detects the length of the detected climax section with respect to the climax section having the maximum length among the detected climax sections. If is less than or equal to the threshold ratio, the rising section is excluded and the rising section is detected.

(Second Embodiment)
FIG. 12 is a flowchart illustrating an intro and ending detection method according to the second embodiment of the present invention. In step S1201, the CPU 3302 loads music waveform data (WAV data) from the external storage device 3308 to the RAM 3304, and performs monaural processing and normalization processing. Specifically, if the music waveform data is stereo, the CPU 3302 converts the stereo music waveform data into monaural music waveform data. Next, the CPU 3302 normalizes the music waveform data to match the maximum amplitude of the music waveform data with the maximum value of the dynamic range, and generates the music waveform data of FIG.

  Next, in step S1202, the CPU 3302 performs a fade-in (out) detection process. Details of the fade-in (out) detection process will be described later with reference to FIGS. 13 and 14. When fade-in (out) is detected, the process proceeds to step S1205. When fade-in (out) is not detected, the process proceeds to step S1203. In step S1205, the CPU 3302 determines that a fade-in (out) type intro (ending) has been detected, and sets the detected fade-in (out) interval as an intro (ending) interval.

  In step S <b> 1203, the CPU 3302 performs a detection process for detecting no drum (such as a cappella) at the beginning or end. If there is a drumless section at the beginning or end of the song waveform data, it can be determined that the section is an intro or ending such as a cappella. Details of the drumless (a cappella etc.) detection process will be described later with reference to FIG. If no drum (such as a cappella) is detected, the process proceeds to step S1206. If no drum (such as a cappella) is detected, the process proceeds to step S1204. In step S1206, the CPU 3302 determines that an intro (ending) type of drumless (a cappella etc.) has been detected, and sets the detected no drum (a cappella etc.) section as an intro (ending) period.

  In step S1204, the CPU 3302 performs a loud sound detection process at the beginning or end. Details of the loud sound detection process will be described later with reference to FIGS. 16 and 17. When a loud sound is detected, the process proceeds to step S1207, and when a loud sound is not detected, the process proceeds to step S1208. In step S1207, the CPU 3302 determines that an intro (ending) of a loud sound type has been detected, and sets the detected loud sound section as an intro (ending) section.

  In step S1208, the CPU 3302 determines that there is no intro (ending) feature, and records that no intro (ending) has been detected.

  As described above, according to the present embodiment, intro and / or ending of music waveform data can be detected. As a result, by reproducing the image (still image or moving image) data in synchronization with the intro and / or ending of the music waveform data, the music and the image can be reproduced while being synchronized. For example, it is possible to perform image processing in which an image gradually appears in the intro of the music waveform data and gradually disappears in the ending of the music waveform data.

  13 and 14 are flowcharts showing details of the fade-in (out) detection process in step S1202 of FIG. Fade-in is an intro that begins with the volume increasing gradually. The fade-out is an ending in which the song ends while the volume gradually decreases.

  In step S1301, as shown in FIG. 19, the CPU 3302 creates array data 1902 of the maximum absolute value of the amplitude at intervals of 1 second for the music waveform data 1901. As in FIG. 1 (first embodiment), a moving average or power obtained by fast Fourier transform may be used.

  Next, in step S1302, the CPU 3302 searches for the first large maximum point 1903 of the array data 1902 as shown in FIG. Here, the large maximum point indicates a maximum point having a value that is half or more of the maximum value of the array data 1902. As for the ending, as in the intro, scan from the end to find a large maximum point, and tentatively determine the starting point of the fade-out.

  Next, in step S1303, as shown in FIG. 21, the CPU 3302 creates array data 2101 of the amount of increase from the previous frame from the beginning for the intro to the end point of the temporary fade-in, and for the ending the start point of the fade-out. From the end to the end, the array data of the reduced amount from the previous frame is created. At this time, when the increase or decrease is negative, it is set to zero.

  Next, in step S1304, the CPU 3302 checks whether there is an excessive increase before the first local maximum point (provisional fade-in end point) 1903 in the intro. If there is, the process proceeds to step S1305, and if not, the process proceeds to step S1306. Similarly, the CPU 3302 checks whether there is a decrease that is too rapid after the last maximum point (a temporary fade-out start point) in the ending. If there is, the process proceeds to step S1305, and if not, the process proceeds to step S1306. In step S1305, the CPU 3302 does not regard the fade-in or fade-out, and determines that no fade-in or fade-out has been detected.

  Specifically, in the case of the intro, as shown in FIG. 21, the CPU 3302 checks whether there is an excessive increase using the array data 2101 created in step S1303. Here, abrupt means a value that is half or more of the maximum value of the array data 1902 in FIG. If there is an excessive increase, it is not appropriate to make a fade-in, and the process advances to step S1305. For example, the array data 2201 in FIG. 22 indicates that the data 2202 is an excessive increase, and the volume suddenly increases from that position, so that it is determined as an intro of a loud sound type in step S1207 in FIG. It should be determined that this intro is not a fade-in.

  Next, in step S1306, the CPU 3302 determines the position of the intro when there is a rapid increase before the first maximum point (provisional fade-in end point), and if not, the first maximum point is the end point of the fade-in. Determine as. Similarly, for the ending, the CPU 3302 determines the position when there is a rapid decrease after the last maximum point (provisional fade-out start point), and if not, the final maximum point is determined as the fade-out start point. That is, the CPU 3302 checks forward from the temporary fade-in end point position, and if there is a portion that is rapidly increased as compared with the previous frame even if it is not sudden, the position is set as the fade-in end point. This is because only a portion where the volume gradually increases between the beginning and the end point of the temporary fade-in is desired to be a fade-in section. In FIG. 23, since the data 2301 has increased rapidly, the end point of the fade-in is changed. As in the case of the swell detection in FIG. 1 (first embodiment), when smoothing is performed using a moving average or the like, it is preferable to determine the actual start point and / or end point with data as close to the original as possible.

  Next, in the case of an intro, the CPU 3302 can specify the position of the actual music waveform data 1901 from the frame at the end point of the fade-in, and therefore the value of the frame at the position at which the fade-in end point was within that range (1 second) Search for a place with the same value and determine the end of the fade-in. The same applies to the ending.

  Next, in step S1307, the CPU 3302 searches for the local minimum point before the end point of the fade-in in the case of the intro based on the array data 1902 in FIG. 19, and determines the local minimum point after the start point of the fade-out in the case of the ending. look for. In the case of FIG. 24, the intro local minimum point 2402 is searched based on the array data 2401.

  Next, in step S1308, in the case of an intro, the CPU 3302 uses the position where a certain amount of amplitude first appears backward from the minimum point as the start point of the fade-in, and in the case of ending, the CPU 3302 initially applies a certain amount from the minimum point toward the front. The position where the amplitude of appears is the end point of the fade-out.

  Specifically, the CPU 3302 creates frame data at intervals smaller than 1 second between the times of the frames corresponding to the minimum point 2402 in the same manner as in FIG. 19, and sets the last minimum point of the frame data. look for. Next, the CPU 3302 creates frame data at a finer interval in the same manner as in FIG. 19 between the times of the frames to which the searched local minimum corresponds, and searches for the final local minimum of the frame data. The CPU 3302 repeats the above processing, specifies the position of the minimum point, and uses this as the temporary fade-in start point. Next, the CPU 3302 narrows the range by shifting the start point of the fade-in until a waveform appears to some extent from the temporary fade-in start point to the end point. The purpose of this is to omit from the fade-in the part of the sound that is so small that it cannot be heard.

  Next, in step S1309, if the length of the fade-in section from the start point to the end point of the fade-in is less than 3 seconds, the CPU 3302 proceeds to step S1310 because the fade-in is too short and unsuitable. In the above case, the process proceeds to step S1311. Similarly, if the length of the fade-out section from the start point to the end point of the fade-out is less than 3 seconds, the CPU 3302 proceeds to step S1310 because the fade-out is too short and is not suitable. move on. In step S1310, the CPU 3302 does not regard the fade-in or fade-out, and determines that no fade-in or fade-out has been detected.

  Similarly to step S1309, even after step S1306, if the length from the top of the waveform (0 seconds) to the end point of the fade-in is less than 3 seconds, the CPU 3302 is too short as a fade-in, Since it is not appropriate, the process may proceed to step S1310. The same applies to the ending.

  In step S1311, the CPU 3302 checks whether there is a waveform having a certain amplitude before the fade-in interval in the case of an intro. If there is a waveform having a certain amplitude, there is an intro portion with a small sound such as narrative before fading in, and the process proceeds to step S1312, and it is determined that the fading is not performed. For example, as shown in FIG. 25, if there is a data area 2501, the data area 2501 is also another intro part, so it is determined that it is not a fade-in intro. On the other hand, if there is no waveform having a certain amplitude, the process proceeds to step S1313, where the detection interval is set as a fade-in interval, and it is determined that a fade-in intro has been detected. Similarly, the CPU 3302 checks whether there is a waveform having a certain amplitude after the fade-out section in the case of ending. If there is a waveform having a certain amplitude, the process proceeds to step S1312, and it is determined that the ending is not a fade-out. On the other hand, if there is no waveform having a certain amplitude, the process advances to step S1313 to determine that the fade-out ending has been detected by setting the detection interval as a fade-out interval.

  FIG. 15 is a flowchart showing details of the drumless (a cappella etc.) detection process in step S1203 of FIG. When the intro has no drum, it can be determined that the intro is an a cappella or the like.

  In step S1501, the CPU 3302 performs fast Fourier transform on the music waveform data at intervals of several tens of milliseconds. For example, fast Fourier transform is performed with the number of samples 2048 and overlap 1/2, and the power of each frame is obtained.

  Next, in step S1502, the CPU 3302 creates power total arrangement data in the low sound range and other areas for each frame. For example, as shown in FIG. 26 (A), the CPU 3302 sets the low frequency range of 250 Hz or less as the low frequency range in which the drum comes out, and sums the power 2601 of 250 Hz or less for each frame to add up the power value of the low frequency range in FIG. Then, the other power total value in FIG. 26C obtained by summing the powers 2602 above 250 Hz is obtained.

  Next, in step S1503, in the case of the intro, in the first half, the CPU 3302 sets the position where the power of the low frequency range first becomes the end point without drums. Similarly, in the case of ending, the CPU 3302 sets the position where the low-frequency power finally decreases in the second half as the starting point without drums.

  FIG. 27 shows music waveform data 2701 and bass sound power total value 2702. For example, in the case of an intro, the CPU 3302 finds a position 2703 where the bass power is first increased between the beginning and the entire quarter (in the first half of the song) with the total bass power 2702 of each frame. The end point without drum. Here, increasing refers to a value that is 1/6 or more of the maximum value of the total power 2702 in the low frequency range.

  Next, in step S1504, the CPU 3302 sets the position where a certain amount of amplitude appears at the beginning in the case of an intro as a starting point without drums, and the position where a certain amount of amplitude appears at the end in the case of endings as an end point without drums. And

  FIG. 28 shows the music waveform data 2701 and the bass power total value 2702. For example, in the case of an intro, the CPU 3302 sets a position 2801 where a waveform appears to some extent from the head (0 seconds) toward the end point without drum as the start point without drum. The purpose of this is to omit the part of the sound that cannot be heard even if it is actually heard from the section without the drum.

  In step S1505, it is checked whether the drumless section from the start point to the end point without drum is shorter than the threshold value. If it is shorter, the process proceeds to step S1506, and if it is longer, the process proceeds to step S1607. In step S <b> 1506, the CPU 3302 does not consider that there is no drum, and determines that no drum (a cappella or the like) has been detected.

  Similarly to step S1505, in step S1503, if the end point without drum is not found, or if the length from the beginning (0 seconds) to the end point without drum is less than 5 seconds, it is too short as no drum. Since it is not suitable, the process may proceed to step S1506.

  In step S1507, the CPU 3302 checks whether or not the power total value other than the low sound range is output in the same manner as in the non-drum section. When it is out, the process proceeds to step S1509, and the CPU 3302 determines that the intro or ending without the drum is detected by setting the detection section as the no drum section. If not, the process proceeds to step S1508, where it is determined that no intro or ending without a drum has been detected since it is a section where power is not output as a whole.

  That is, the CPU 3302 checks whether the power is output in the section without the drum and the section after that with the power values other than the bass range (above 250 Hz) of each frame in FIG. Specifically, first, the CPU 3302 obtains an average value of the total power in the non-drum section other than the low sound range. Next, the CPU 3302 obtains the average value of the total power values other than the low frequency range (in the first half of the song) between the end point of the drumless section and the entire quarter. Next, the CPU 3302 checks whether the average of the powers other than the low sound range in the drumless section is smaller than 1/5 of the average of the powers other than the subsequent low sound range. In other words, it is confirmed whether or not the change is severe only in the low frequency range in the drumless section and thereafter.

  If it is smaller, the CPU 3302 determines that the drumless section has a lower low frequency range, but the power other than the lower frequency range is low, so that the overall power is not output and is not suitable for the section without the drum. move on. If it is larger, the CPU 3302 proceeds to step S1509, determines from the drum-less start point to the drum-less end point as a drum-free section, and determines that an intro or ending without a drum has been detected.

  FIG. 29 is a diagram showing the music waveform data 2901 and the power total value 2902 other than the bass range. The CPU 3302 compares the average value other than the low frequency range in the drumless section A and the subsequent section B. In section A, the total power value 2902 is small, but exceeds the threshold value, so section A is a drumless section. On the other hand, when the total power value 2902 of the section A is smaller, the section A is not detected as a section without a drum.

  FIG. 16 is a flowchart showing details of the loud sound (intro) detection process in step S1204 of FIG. In the loud sound (intro) detection process, it is checked whether the music starts with a loud sound such as a cymbal.

  In step S1601, as shown in FIG. 30, the CPU 3302 creates frame array data 3002 having a maximum absolute value of amplitude at intervals of 0.2 seconds for the music waveform data 3001. As in FIG. 1 (first embodiment), a moving average or power obtained by fast Fourier transform may be used.

  Next, in step S1602, the CPU 3302 searches for a position 3003 where the first absolute amplitude value is large in the first half of the music waveform data 3001, and tentatively determines the end point of a loud sound. Here, a large amplitude absolute value indicates a value that is 2/5 or more of the maximum value of the frame array data 3002.

  Next, in step S1603, as shown in FIG. 31, the CPU 3302 searches for the first small absolute amplitude value 3101 from the end point of the loud sound to the front, and tentatively determines it as the starting point of the loud sound. Here, the small amplitude absolute value refers to a value that is 1/6 or less of the maximum value of the frame array data 3002.

  In step S1604, the CPU 3302 proceeds to step S1605 when the interval between the position 3101 having a small amplitude (starting point of loud sound) 3101 and the position having a large position (ending point of loud sound) 3003 is wide, and proceeds to step S1606 when narrow. Specifically, when the loud sound range is wider than 0.8 seconds, the CPU 3302 considers that the sound does not start sharply and therefore does not start with a cymbal feeling, and proceeds to step S1605. The process proceeds to S1606. In step S1605, the CPU 3302 does not rise sharply and is not regarded as a loud sound intro, and proceeds to step S1208 in FIG.

  In step S <b> 1606, the CPU 3302 checks whether a certain amount of amplitude is present before the position 3101 having a small amplitude (starting point of a loud sound). When it is out, the process proceeds to step S1607, and when it is not out, the process proceeds to step S1608. In step S1607, the CPU 3302 does not regard it as a loud sound intro, and proceeds to step S1208 in FIG.

  In step S1608, the CPU 3302 performs fast Fourier transform at intervals of several tens of milliseconds as shown in FIG. For example, the CPU 3302 performs fast Fourier transform while overlapping so that the number of samples is 2048 and 0.2 seconds per frame, and obtains the power of each frame.

  Next, in step S1609, the CPU 3302 sets the end point of the loud sound immediately before the power peak changes from the position where the amplitude is large. In other words, the CPU 3302 searches for a position where the power at the end point of the provisional loud sound ends (searches for a position until the loud sound grows and the sound is no longer characteristic), and immediately before the end of the loud sound ends. Determine as.

  Specifically, first, the CPU 3302 searches for two locations 3201 and 3202 in which the power is the largest from the power of the end point frame of the temporary loud sound in FIG.

  Next, the CPU 3302 checks whether or not the power is characteristic at the same location in each frame from the end point frame of the provisional loud sound, and determines the point immediately before finishing as the end point of the loud sound. Specifically, the CPU 3302 searches for five places where the power is the largest among the powers of the frames to be examined. Next, the CPU 3302 checks whether there is the same place as one of the two places 3201 and 3202 having the maximum power among the five places. Since the number of frames in which the same portion does not appear continuously is counted, this counter is reset to 0 for the first time.

  If there is, the CPU 3302 determines that the power is characteristically output at the same location, resets the counter to 0, sets the frame to be examined as the next frame, and returns to the process of searching for the above 5 locations.

  If not, the CPU 3302 increments the counter by one. If the counter is less than 3, the CPU 3302 sets the frame to be examined as the next frame, and returns to the process of searching for the above five locations. When the counter is 3, the CPU 3302 determines that the sound has become inconspicuous because there is no strong power at the same location continuously, and it is three times before the currently examined frame (the power is continuously present). Set the end point of the loud sound (just before it starts to be judged).

  Next, in step S1610, the CPU 3302 sets the maximum position within the range of the provisional loud sound as the starting point of the loud sound. That is, the CPU 3302 sets the place where the absolute value of the amplitude is the maximum between the start point of the temporary loud sound and the end point of the loud sound as the start point of the loud sound. This is because the start point is not the start of loud sound but the loud position of the start of the video effect.

  Next, in step S1611, the CPU 3302 determines that the detected section is a loud sound section, a loud sound intro is detected, and the process proceeds to step S1207 in FIG.

  FIG. 17 is a flowchart showing details of the loud sound (ending) detection process in step S1204 of FIG. In the loud sound (ending) detection process, as in the loud sound (intro) detection process of FIG. 16, it is checked whether the music ends with a loud sound such as a cymbal.

  In step S1701, the CPU 3302 creates frame arrangement data having a maximum absolute value of amplitude at intervals of 0.2 seconds for the music waveform data. As in FIG. 1 (first embodiment), a moving average or power obtained by fast Fourier transform may be used.

  Next, in step S1702, the CPU 3302 searches for a position where the last absolute amplitude value is large in the latter half of the music waveform data, and tentatively determines the end point of a loud sound.

  Next, in step S1703, the CPU 3302 performs fast Fourier transform at intervals of several tens of milliseconds.

  Next, in step S1704, the CPU 3302 searches for a position immediately before the power peak changes from the end point of a loud sound toward the front. That is, the CPU 3302 searches for a position where the loud sound starts.

  Next, in step S1705, the CPU 3302 searches for a position with a large amplitude first from immediately before to after the power peak changes.

  Next, in step S1706, the CPU 3302 checks whether the interval from immediately before the peak changes to a position with a large amplitude is wide. If it is wide, the CPU 3302 proceeds to step S1707, and since the sound is not sharply raised, the CPU 3302 does not regard it as a loud sound ending and proceeds to step S1208 in FIG. If narrow, the process proceeds to step S1708.

  In step S1708, the CPU 3302 sets the maximum position within the tentative loud sound range as the starting point of the loud sound.

  Next, in step S1709, the CPU 3302 determines that the detected section is a loud sound section, and that a loud sound ending has been detected, and proceeds to step S1207 in FIG.

  This embodiment can be realized by a computer executing a program. Also, means for supplying a program to a computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium such as the Internet for transmitting such a program is also applied as an embodiment of the present invention. Can do. A computer program product such as a computer-readable recording medium in which the above program is recorded can also be applied as an embodiment of the present invention. The above program, recording medium, transmission medium, and computer program product are included in the scope of the present invention. As the recording medium, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

3400 Swell detection device 3401 Smoothing unit 3402 Minimum point detection unit 3403 Swell section detection unit 3411 First smoothing unit 3412 Second smoothing unit

Claims (8)

A smoothing unit for smoothing the curved waveform data;
A minimum point detection unit for detecting a minimum point of the curved waveform data smoothed by the smoothing unit;
A climax section detection unit that detects a section where the maximum value in the section between the minimum points is equal to or more than a threshold based on the minimum point detected by the minimum point detection section;
The smoothing unit
A first smoothing unit for smoothing the curved waveform data;
A second smoothing unit that smoothes the curved waveform data smoothed by the first smoothing unit;
The minimum point detection unit detects a minimum point of the curved waveform data smoothed by the second smoothing unit,
The rising section detection unit corrects the start point and end point of the section between the minimum points detected by the minimum point detection unit based on the change amount of the curved waveform data smoothed by the first smoothing unit. , up detector Ri Sheng you and detecting the climax section. A smoothing unit for smoothing the curved waveform data;
A minimum point detection unit for detecting a minimum point of the curved waveform data smoothed by the smoothing unit;
A climax section detection unit that detects a section where the maximum value in the section between the minimum points is equal to or more than a threshold based on the minimum point detected by the minimum point detection section;
The minimum point detection unit detects a minimum point of the curved waveform data smoothed by the smoothing unit, and the detected minimum point is a threshold value with respect to a maximum point before or after the detected minimum point greater than the proportion excluded to rise detector Ri Sheng you and detects the minimum point and the minimum point. A smoothing unit for smoothing the curved waveform data;
A minimum point detection unit for detecting a minimum point of the curved waveform data smoothed by the smoothing unit;
A climax section detection unit that detects a section where the maximum value in the section between the minimum points is equal to or more than a threshold based on the minimum point detected by the minimum point detection section;
When the value of the section between the two detected rising sections is larger than a threshold ratio with respect to the value of the two detected rising sections, the rising section detecting unit sets the detected two rising sections to 1 One of Sheng Ri rise detection device shall be the features that lead to climax section. A smoothing unit for smoothing the curved waveform data;
A minimum point detection unit for detecting a minimum point of the curved waveform data smoothed by the smoothing unit;
A climax section detection unit that detects a section where the maximum value in the section between the minimum points is equal to or more than a threshold based on the minimum point detected by the minimum point detection section;
The climax section detection unit excludes the climax section if the length of the climax section detected is equal to or less than a threshold ratio with respect to the climax section having the maximum length among the detected climax sections. Sheng you and detects the Ri-up detection device. A smoothing step for smoothing the curved waveform data;
A minimum point detecting step for detecting a minimum point of the curved waveform data smoothed by the smoothing step;
A program for executing the excitement section detecting step of detecting a section excitement the interval maximum value is not less than the threshold value in the interval between the minimum point on the basis of the minimum point detected by the minimum point detecting step to the computer ,
The smoothing step includes
A first smoothing step for smoothing the curved waveform data;
A second smoothing step for smoothing the curved waveform data smoothed by the first smoothing step;
In the minimum point detection step, the minimum point of the curved waveform data smoothed by the second smoothing step is detected,
In the rising section detecting step, the start point and end point of the section between the minimum points detected by the minimum point detecting step are corrected based on the change amount of the curved waveform data smoothed by the first smoothing step. , A program characterized by detecting a rising section . A smoothing step for smoothing the curved waveform data;
A minimum point detecting step for detecting a minimum point of the curved waveform data smoothed by the smoothing step;
A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section in which a maximum value in a section between minimum points is greater than or equal to a threshold value based on the minimum point detected by the minimum point detection step. ,
In the minimum point detection step, a minimum point of the curved waveform data smoothed by the smoothing step is detected, and the detected minimum point is a threshold value with respect to a maximum point before or after the detected minimum point. A program characterized by detecting a minimum point by excluding the minimum point if it is larger than a ratio. A smoothing step for smoothing the curved waveform data;
A minimum point detecting step for detecting a minimum point of the curved waveform data smoothed by the smoothing step;
A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section in which a maximum value in a section between minimum points is greater than or equal to a threshold value based on the minimum point detected by the minimum point detection step. ,
In the climax section detecting step, when the value of the section between the two climax sections detected is larger than the threshold ratio with respect to the value of the two climax sections detected, the two climax sections detected as 1 A program characterized by connecting to two climax sections. A smoothing step for smoothing the curved waveform data;
A minimum point detecting step for detecting a minimum point of the curved waveform data smoothed by the smoothing step;
A program for causing a computer to execute a climax section detecting step for detecting, as a climax section, a section in which a maximum value in a section between minimum points is greater than or equal to a threshold value based on the minimum point detected by the minimum point detection step. ,
In the climax section detecting step, if the length of the detected climax section is equal to or less than the threshold ratio with respect to the climax section having the maximum length among the climax sections detected, the climax section is excluded if the length of the climax section is equal to or less than the threshold ratio. Detecting a program.

Images