JP6690445B2 - SOUND INFORMATION SETTING METHOD AND SOUND INFORMATION SETTING DEVICE - Google Patents

Description

  The present invention relates to a technique of setting sound information such as a chord name in a sound represented by an acoustic signal.

  For example, in order for the user to specify the sound information such as the chord name for each period of the music while listening to the reproduced sound of the music, very complicated and specialized work is required. Therefore, a technique for estimating a code by analyzing an acoustic signal is disclosed in Patent Document 1, for example.

JP, 2014-235328, A

  However, it is not easy to analyze sound information such as chords with high accuracy by analyzing acoustic signals. For example, it is actually difficult to accurately estimate a code from a mixed sound of a plurality of acoustic components such as a singing voice and a percussion instrument sound. In consideration of the above circumstances, an object of the present invention is to enable a user to efficiently instruct sound information.

  In order to solve the above problems, in the sound information setting method according to the first aspect of the present invention, a computer system extracts a characteristic amount of an acoustic signal for each of a plurality of unit periods, The analysis image on which the corresponding unit image is arranged is displayed on the display device so that the display mode of the unit image of each unit period becomes closer as the feature amount of each unit period is similar, and the user selects from the analysis image. The sound information is set for a set period including one or more unit periods corresponding to the selected unit image. In the above aspect, the analysis image in which the display aspect of each unit image is closer to each other as the feature amount of each unit period is similar is displayed on the display device, and the setting including one or more unit periods selected from the analysis image by the user is performed. Sound information is set for the period. Therefore, the user can efficiently instruct the sound information.

  In the sound information setting method according to the second aspect of the present invention, the computer system extracts the characteristic amount of the acoustic signal for each of the plurality of unit periods, and sets the first unit image corresponding to each of the plurality of unit periods on the time axis. And a distribution image in which the second unit images corresponding to each of the plurality of unit periods are arranged on the coordinate plane at intervals according to the distance of the feature amount of each unit period. The more similar the feature amount is, the closer the display mode of the first unit image in each unit period is approximated, and the closer the display mode of the second unit image in each unit period is approximated. The sound information is set for a set period including one or more unit periods corresponding to the first unit image selected by the user from the image or the second unit image selected by the user from the distribution image. In the above aspect, the more similar the feature amount of each unit period is, the more similar the display aspect of the first unit image of each unit period is, and the closer the display aspect of the second unit image of each unit period is is displayed. Sound information is set for a set period displayed on the device and including one or more unit periods selected by the user. Therefore, the user can efficiently instruct the sound information.

  In the preferred example of the second aspect, when the first unit image in the time-series image is selected, the display aspect of the first unit image is changed and the unit period common to the first unit image in the distribution image. The display mode of the second unit image corresponding to is changed. Further, in the preferred example of the second mode, when the second unit image in the distribution image is selected, the display mode of the second unit image is changed, and the same time unit image as the second unit image is shared. The display mode of the first unit image corresponding to the unit period is changed.

  In a preferred example of the second aspect, when the user selects the first unit image of the time-series image, a set period including the unit period represented by the first unit image, and a unit whose feature amount is similar to the unit period. Other set period including the period is set. In the above aspect, the set period including the unit period represented by the first unit image selected by the user and the other set period including the unit period in which the feature amount is similar to the unit period are collectively set. Therefore, it is possible to reduce the load for the user to specify a plurality of setting periods, as compared with a configuration in which the user needs to specify each setting period individually.

  A sound information setting device according to a preferred aspect of the present invention is a feature extraction unit that extracts a feature amount of an acoustic signal for each of a plurality of unit periods, and an analysis image in which unit images corresponding to each of the plurality of unit periods are arranged. Corresponding to the unit image selected by the user from the display control unit and the analysis image so that the display mode of the unit image in each unit period becomes closer as the feature amount of each unit period becomes closer. And an information setting unit that sets sound information for a set period including one or more unit periods.

It is a block diagram of the sound information setting device which concerns on 1st Embodiment of this invention. It is explanatory drawing of a time series image. It is explanatory drawing of a distribution image. It is an explanatory view of a hue circle for selecting a display mode of each unit image in a distribution image. It is explanatory drawing of a setting period. It is explanatory drawing of the procedure which sets a setting period and sound information. It is a flow chart of sound information setting processing. It is explanatory drawing of the procedure which sets a setting period and sound information in 2nd Embodiment. It is explanatory drawing of the coloring image in a modification.

<First Embodiment>
FIG. 1 is a configuration diagram of a sound information setting device 100 according to the first embodiment of the present invention. The sound information setting device 100 of the first embodiment is realized by a computer system including a control device 10, a storage device 12, a display device 14, and an input device 16. For example, various information processing devices such as mobile phones, smartphones, and personal computers can be used as the sound information setting device 100.

  The storage device 12 stores a program executed by the control device 10 and various data used by the control device 10. For example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium can be used as the storage device 12. It is also possible to use, as the storage device 12, a combination of a plurality of recording media of the same type or different types that are configured separately from each other. A storage device 12 (for example, cloud storage) separate from the sound information setting device 100 may be installed, and the control device 10 may perform writing and reading to and from the storage device 12 via a communication network.

  The storage device 12 of the first embodiment stores acoustic signals X representing various sounds such as musical sounds and voices. The acoustic signal X is, for example, a signal in the time domain (a series of a plurality of samples) that represents the waveforms of performance sounds of a plurality of musical instruments. It is also possible to supply the acoustic signal X to the sound information setting device 100 from a reproducing device that reproduces the acoustic signal X recorded on a recording medium such as an optical disc. The acoustic signal X is stored in the storage device 12 as a file of any format.

  The sound information setting device 100 of the first embodiment is an information processing device that sets the sound information Z for a specific period of the acoustic signal X (hereinafter referred to as “setting period”). The sound information Z is set for each of a plurality of setting periods in which the acoustic signal X is divided on the time axis. The sound information Z is information regarding the characteristics of the sound represented by the acoustic signal X. In the first embodiment, the chord name of the sound represented by the acoustic signal X is exemplified as the sound information Z.

  The display device 14 of FIG. 1 is composed of, for example, a liquid crystal display panel, and displays various images under the control of the control device 10. The input device 16 is an operation device that receives an instruction from a user. For example, a plurality of operators operated by the user or a touch panel that detects contact with the display surface of the display device 14 is preferably used as the input device 16.

  The control device 10 is configured by a general-purpose processing circuit such as a CPU (Central Processing Unit), and controls each element of the sound information setting device 100 as a whole. The control device 10 of the first embodiment executes a program stored in the storage device 12 to execute a plurality of functions for setting the sound information Z in each setting period of the acoustic signal X (feature extraction unit 22, display). The control unit 24 and the information setting unit 26) are realized. A configuration in which the functions of the control device 10 are distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit realizes some or all of the functions of the control device 10 may be employed.

  The feature extraction unit 22 extracts the feature amount F of the acoustic signal X. The feature extraction unit 22 of the first embodiment calculates the feature amount F for each of a plurality of periods (hereinafter referred to as “unit period”) in which the acoustic signal X is divided on the time axis. Each unit period is, for example, a period (frame) having a time length of about several tens of milliseconds to several hundreds of milliseconds. The sound information Z is set for each set period including a plurality of continuous unit periods. In addition, two adjacent unit periods on the time axis may partially overlap each other.

  The feature amount F of the first embodiment is an index representing the feature of the chord (chord) of the sound represented by the acoustic signal X. In the first embodiment, the chroma vector is exemplified as the feature amount F. The chroma vector is a vector (PCP: Pitch Class Profile) including a plurality of elements corresponding to different tones (for example, 12 semitones of equal temperament). An element corresponding to any one scale tone of the chroma vector is set to a numerical value obtained by adding or averaging the intensities of band components corresponding to the scale tone of the acoustic signal X over a plurality of octaves. Therefore, among the plurality of elements of the chroma vector, the element corresponding to each constituent sound of the chord within the unit period is set to a larger numerical value.

  The display control unit 24 causes the display device 14 to display an image. The display control unit 24 of the first embodiment causes the display device 14 to display an analysis image in which images (hereinafter referred to as “unit images”) corresponding to each of the plurality of unit periods of the acoustic signal X are arranged. In the first embodiment, the time series image GA of FIG. 2 and the distribution image GB of FIG. 3 are displayed on the display device 14 as analysis images. The time-series image GA and the distribution image GB are, for example, displayed in parallel with each other (for example, displayed on the same screen).

  As illustrated in FIG. 2, the time-series image GA is an image in which unit images UA (exemplification of the first unit image) corresponding to each of the plurality of unit periods of the acoustic signal X are arranged along the time axis. Each unit image UA is an image representing one unit period of the acoustic signal X. FIG. 2 exemplifies a case where each unit image UA is a rectangular figure.

  As illustrated in FIG. 3, the distribution image GB is an image in which the unit image UB (illustration of the second unit image) corresponding to each of the plurality of unit periods of the acoustic signal X is arranged on the coordinate plane. Each unit image UB is an image representing one unit period of the acoustic signal X. In FIG. 3, the case where each unit image is a circular figure is illustrated. As understood from the above description, the unit image UA and the unit image UB are displayed on the display device 14 for each of the plurality of unit periods of the acoustic signal X. That is, each unit image UA of the time-series image GA and each unit image UB of the distribution image GB have a one-to-one correspondence.

  The plurality of unit images UB in the distribution image GB of FIG. 3 are arranged on the coordinate plane at intervals according to the similarity (that is, the degree of similarity) of the feature amount F of each unit period. Specifically, the display control unit 24 calculates the distance of the feature amount F between the two unit periods for all combinations that select two unit periods from the plurality of unit periods of the acoustic signal X. The distance is an index of the similarity of the feature amount F between two unit periods, and for example, a Euclidean distance or a cosine distance is a suitable example. The unit images UB of each unit period in which the feature amounts F are similar (that is, the distance is small) are arranged at close positions, and the unit images UB of each unit period in which the feature amounts F deviate are arranged at distant positions. Therefore, for example, the unit images UB of each unit period in which similar or common chords are played are concentrated in a specific area in the coordinate plane. For arranging the plurality of unit images UB with respect to the coordinate plane, a known analysis technique such as a multidimensional scaling method (MDS) can be arbitrarily used.

  As illustrated in FIG. 2 and FIG. 3, the display control unit 24 of the first embodiment sets the display mode of each unit image UA in the time-series image GA and the display mode of each unit image UB in the distribution image GB as follows. It is variably set according to the feature amount F of the unit period. The display mode means a property of an image that can be visually discriminated. For example, the three attributes of color, saturation, lightness (gradation), or hue (hue) are typical examples of display modes.

  Specifically, with respect to the distribution image GB of FIG. 3, the display control unit 24 makes each unit such that the display mode of the unit image UB of each unit period becomes closer as the feature amount F of each unit period becomes similar to each other. The display mode of the image UB is selected. That is, as the feature amount F is similar between any two unit periods of the acoustic signal X, the display modes of the two unit images UB corresponding to each unit period are similar to each other, and the two unit periods UB are similar to each other. The difference in the feature amount F between the two becomes more remarkable the difference in the display mode of the two unit images UB corresponding to each unit period.

  Specifically, the display control unit 24 changes the hue along the circumferential direction around the center of gravity P0 of the plurality of unit images UB, and the saturation along the radial direction around the center of gravity P0. The display mode of each unit image UB is selected so as to change. For example, as illustrated in FIG. 4, assume a hue circle in which the hue continuously changes in the circumferential direction and the saturation continuously changes in the radial direction. The display control unit 24 arranges the hue circles so as to overlap the distribution image GB so that the center of the hue circle is located at the center of gravity P0 of the plurality of unit images UB, and the hue at the point where each unit image UB is located in the hue circle. And saturation are selected as the hue and saturation of the unit image UB. Therefore, as the feature amount F becomes more similar between any two unit periods of the acoustic signal X, the hue and saturation of the two unit images UB corresponding to each unit period are closer to each other. The user who visually recognizes the distribution image GB illustrated above can check the approximate number and frequency of types of codes in the acoustic signal X.

  In addition, as illustrated in FIG. 2, the display control unit 24 displays the unit image UA corresponding to any one unit period in the time series image GA and the unit image corresponding to the unit period in the distribution image GB. It is displayed in the same display mode as UB. That is, the unit image UA and the unit image UB corresponding to the common unit period of the acoustic signal X are displayed in the same display mode. Therefore, similarly to the distribution image GB in the time-series image GA, the more similar the feature amount F is between the arbitrary two unit periods of the acoustic signal X, the more the two unit images UA corresponding to each unit period. The display modes are similar to each other, and the more the feature amounts F deviate between the two unit periods, the more remarkable the display modes of the two unit images UA corresponding to the respective unit periods become.

  As is understood from the above description, the display control unit 24 of the first embodiment approximates the display mode of each unit image UA in the time-series image GA and the distribution as the feature amount F of each unit period is similar. The time series image GA and the distribution image GB are displayed on the display device 14 so that the display modes of the respective unit images UB in the image GB are similar. Therefore, the user who visually recognizes the time-series image GA displayed on the display device 14 can confirm whether the feature amount F is similar or different in each unit period on the time axis of the acoustic signal X. Since the feature amount F of the first embodiment is a chroma vector reflecting the feature of the code, the user who visually recognizes the time series image GA changes the code in the acoustic signal X (for example, any one code continues). It is possible to understand the period).

  The information setting unit 26 of FIG. 1 sets the sound information Z for each setting period of the acoustic signal X. The user can arbitrarily specify the set period of the acoustic signal X and the sound information Z of the set period by appropriately operating the input device 16. Specifically, one or more set periods are designated by the user's instruction for the time-series image GA. The information setting unit 26 of the first embodiment sets, for each setting period specified by the user, the sound information Z specified by the user for the setting period. Specifically, the information setting unit 26 generates and updates the setting data indicating the sound information Z for each setting period according to the instruction from the user. The setting data is data that defines the correspondence between the start and end points of each setting period and the sound information Z set in the setting period, and is stored in the storage device 12.

  The display control unit 24 displays each set period of the acoustic signal X and the content (specifically, the chord name) of the sound information Z for each set period on the analysis image. Specifically, as illustrated in FIG. 5, the display control unit 24 displays each set period on the time-series image GA (specifically, it is highlighted by a frame line), and the sound information is displayed near the set period. Z is displayed, and the sound information Z is displayed in the vicinity of one or more unit images UB corresponding to the set period in the distribution image GB. In the first embodiment, as described in detail below, the user operates the analysis image (time series image GA, distribution image GB) displayed on the display device 14 with the input device 16 to select an arbitrary one. It is possible to specify the above unit period as the set period and also specify the sound information Z of the set period.

  First, the user can specify one or more unit images UA of the plurality of unit images UA of the time-series image GA as the setting period and also specify the sound information Z of the setting period. FIG. 6 is an explanatory diagram of a procedure for designating the set period and the sound information Z for the time-series image GA. As in the procedure PA1 illustrated in FIG. 6, the user specifies any one unit image UA of the plurality of unit images UA of the time-series image GA. The information setting unit 26 designates the unit period represented by the one unit image UA selected by the user as the provisional setting period, and the display control unit 24 highlights the setting period. Specifically, a frame line surrounding the set period is displayed.

  As in the procedure PA2 illustrated in FIG. 6, the user appropriately operates the input device 16 so that the display mode is similar (front or rear) around the one unit image UA specified in the procedure PA1. It is possible to extend the set period to another unit image UA that matches. That is, before and after the one unit period selected in the procedure PA1, the set period is extended within the range where the unit period and the feature amount F are similar. The information setting unit 26 updates the set period according to the user's instruction for expansion, and the display control unit 24 highlights the expanded set period. By the above operation, one set period is defined. It is also possible to update the unit image UA or the unit image UB of each unit period belonging to one unit section to a common display mode.

  As in the procedure PA3 illustrated in FIG. 6, the user appropriately operates the input device 16 to generate arbitrary sound information Z (specifically, a chord name) for the set period designated in the procedure PA2. It is possible to specify. The information setting unit 26 sets the sound information Z designated by the user in the setting period, and the display control unit 24 displays the sound information Z near the setting period. When the set period and the sound information Z are designated for the time-series image GA, the display control unit 24 reflects the contents of the designation in the distribution image GB. Specifically, as illustrated in FIG. 5, a plurality of unit images UB corresponding to each unit period within the set period is highlighted in the distribution image GB (specifically, the plurality of unit images UB are surrounded. The frame information is displayed) and the sound information Z of the set period is displayed near the unit image UB. As can be understood from the above description, the display control unit 24 changes the display mode of the unit image UA when the unit image UA in the time-series image GA is selected, and the unit in the distribution image GB is changed. The display mode of the unit image UB corresponding to the unit period common to the image UA is changed. In the above description, the sound information Z is set (PA3) after the setting period is extended (PA2), but the setting period can be extended after the setting of the sound information Z.

  In addition, the user can arbitrarily specify one or more unit images UB in the distribution image GB and the unit image UB by an instruction to the input device 16. When the unit image UB in the distribution image GB is designated, the display control unit 24 changes the display mode of the unit image UB, and a unit common to the unit image UB designated by the user in the time series image GA. The display mode of the unit image UA corresponding to the period is changed. As can be understood from the above description, designation of one of the unit image UA in the time-series image GA and the unit image UB in the distribution image GB is reflected in the display of the other. That is, for each unit period designated by the user by selecting the unit image UA or the unit image UB, both the unit image UA and the unit image UB are displayed (that is, highlighted) in a display mode different from that in the non-selected state. .

  As will be understood from the above description, in the first embodiment, the display mode of the unit image UA and the unit image UB corresponding to the common unit period of the acoustic signal X is determined by one of the unit image UA and the unit image UB. They are changed (that is, highlighted) in conjunction with each other. Therefore, the user can establish a correspondence relationship between each unit image UA of the time-series image GA and each unit image UB of the distribution image GB (that is, a combination of the unit image UA and the unit image UB corresponding to a common unit period). It is possible to understand visually and intuitively.

  FIG. 7 is a flowchart of a process (hereinafter, referred to as “sound information setting process”) for the sound information setting device 100 to set the sound information Z in each setting period of the acoustic signal X. The control device 10 starts the sound information setting process of FIG. 7 triggered by an instruction to activate the program stored in the storage device 12.

  When the sound information setting process is started, the feature extraction unit 22 extracts the feature amount F for each unit period of the acoustic signal X (SA1). The display control unit 24 includes a time-series image GA in which a plurality of unit images UA in a display manner corresponding to the feature amount F are arranged, and a distribution image GB in which a plurality of unit images UB in a display manner according to the feature amount F are arranged. Is displayed on the display device 14 (SA2).

  The information setting unit 26 determines whether or not an instruction of the set period has been received from the user (SA3). When the instruction of the set period is received (SA3: YES), the information setting unit 26 sets the set period according to the instruction from the user (SA4), and the display control unit 24 sets the set period selected by the user. The display mode of each unit image UA and each unit image UB corresponding to is changed (SA5). On the other hand, when the set period is not instructed (SA3: NO), the information setting unit 26 determines whether the designation of the set period has been canceled (SA6). When the designation of the set period is released (SA6: YES), the information setting unit 26 releases the designation of the set period (SA7), and the display control unit 24 causes the unit image UA and the unit corresponding to the set period. The display mode of the image UB is changed to the initial mode (SA8).

  When the designation or cancellation of the set period is not instructed (SA6: NO), the information setting unit 26 determines whether the instruction of the sound information Z has been received from the user (SA9). When the sound information Z is instructed (SA9: YES), the sound information setting unit 26 sets the sound information Z instructed by the user in the setting period of the current selection state (SA10), and the display control unit 24 The sound information Z designated by the user is displayed on the time series image GA and the distribution image GB (SA11). On the other hand, if the sound information Z is not designated (SA9: NO), the setting (SA10) and the display (SA11) of the sound information Z are not executed.

  When the above process is completed, the control device 10 determines whether or not the user has instructed the end of the sound information setting process (SA12). When the end of the sound information setting process is not instructed (SA12: NO), the control device 10 shifts the process to step SA3. That is, setting (SA4, SA5) and cancellation (SA7, SA8) of the setting period and setting of the sound information Z (SA10, SA11) according to the instruction from the user are repeated. Therefore, the sound information Z is set for each of the plurality of setting periods of the acoustic signal X. On the other hand, when the user instructs the end (SA10: YES), the control device 10 ends the sound information setting process of FIG. 7.

  As illustrated above, in the first embodiment, the analysis image in which the display mode of each unit image (UA, UB) is closer to each other as the feature amount F of each unit period is similar is displayed on the display device 14, and the user is The sound information Z is set for a set period including one or more unit periods selected from the analysis image. Therefore, the user can efficiently specify the sound information Z. Particularly in the first embodiment, the time-series image GA in which the unit images UA corresponding to the respective unit periods are arranged along the time axis is displayed. Therefore, there is an advantage that the user can select a desired set period while confirming the temporal transition of the similarity of the feature amount F of each unit period. Further, in the first embodiment, the distribution image GB in which the unit images UB corresponding to the respective unit periods are arranged on the coordinate plane at intervals according to the distance of the feature amount F is displayed. Therefore, there is also an advantage that the user can select a desired set period while visually and intuitively grasping the distribution of the feature amount F in each unit period.

  By the way, as a configuration for setting the sound information Z in each period of the acoustic signal X, for example, a configuration using a plurality of reference values prepared in advance for each sound information Z (for example, for each chord) (hereinafter referred to as “comparative”) Is also assumed. Each reference value is a template to be compared with the feature amount F. In contrast, the feature amount F of any one unit period is compared with each of a plurality of reference values, and the sound information Z corresponding to the reference value most similar to the feature amount F of the unit period is set in the unit period. To be done. However, in practice, the sound information Z of the reference value similar to the feature amount F of the unit period is not necessarily the proper content, and in some cases, other sound information Z may be proper. In contrast to the proportionality illustrated above, in the first embodiment, the sound information Z corresponding to the instruction from the user is set for the setting period designated by the user, and therefore the acoustic signal X It is possible to suppress the problem of the proportionality that inappropriate sound information Z is set in each period.

<Second Embodiment>
A second embodiment of the present invention will be described. In the first embodiment, each set period of the acoustic signal X is individually designated, and the sound information Z is set individually for each set period. In the second embodiment, when the user specifies one setting period, other setting periods in which the feature amount F is similar to the setting period are also collectively specified. It should be noted that, in each of the following exemplary embodiments, the elements having the same operation or function as those in the first embodiment are given the same reference numerals as those used in the description of the first embodiment, and the detailed description thereof will be appropriately omitted.

  FIG. 8 is an explanatory diagram of selection of a set period in the second embodiment. In the procedure PB1 of FIG. 8, when the user designates one unit image UA in the time-series image GA, the information setting unit 26 causes the unit period corresponding to the unit image UA and the feature amount F in the unit period. And other unit periods having similar values are collectively designated as the set period. Specifically, each unit period in which the feature amount F within the allowable range is extracted with respect to the feature amount F in one unit period designated by the user (for example, one unit period designated by the user Unit image UA for which the difference from the feature amount F is less than the threshold value. That is, each of the plurality of unit periods in which the feature amount F is similar to each other is collectively selected as the set period. The display control unit 24 highlights each unit image UA selected by the information setting unit 26.

  In the procedure PB2 of FIG. 8, the user can extend any one of the plurality of setting periods designated in the procedure PB1 forward or backward like the procedure PA2 of the first embodiment. It is possible. Since the feature amount F may be different between the unit periods adjacent to each other on the time axis, when the set period is extended (the total number of unit periods in the set period increases), the feature amount of each unit period in the set period is increased. The numerical range of F is expanded. The sound information setting unit 26 expands the allowable range up to the numerical value range of the feature amount F of the plurality of unit periods included in the extended set period, and after the feature amount F is enlarged around the unit period designated in the procedure PB1. The range of the plurality of unit periods included in the permissible range is defined as the updated set period. That is, a plurality of set periods each including a plurality of unit periods are collectively designated.

  In procedure PB2 of FIG. 8, the user can specify the sound information Z for any one of the plurality of setting periods specified in the procedure PB2. The sound information setting unit 26 sets the sound information Z designated by the user in common for the plurality of setting periods designated in the procedure PB1 and the procedure PB2. That is, common sound information Z is collectively set for a plurality of setting periods having similar feature amounts F.

  Also in the second embodiment, the same effect as that of the first embodiment is realized. Further, in the second embodiment, when the unit image UA of the time-series image GA is designated, a set period including the unit period represented by the unit image UA and another feature amount F similar to the unit period are displayed. The set period and the set period are collectively set. Therefore, it is possible to reduce the load for the user to specify a plurality of setting periods as compared with the configuration in which each setting period needs to be specified individually.

<Modification>
Each aspect illustrated above can be variously modified. Specific modes of modification will be exemplified below. Two or more aspects arbitrarily selected from the following exemplifications can be appropriately merged within a range not inconsistent with each other.

(1) The time length of the unit period which is the unit for extracting the feature amount F is arbitrary. For example, a configuration may be adopted in which the feature amount F is extracted for each unit period in which the acoustic signal X is divided with the beat point of the music represented by the acoustic signal X as a boundary. A unit of time may be a period of time length corresponding to a plurality of beats. It is also possible to extract the feature amount for a plurality of periods within the unit period and calculate a representative value (for example, an average value) of the feature amounts within the unit period as the feature amount F for the unit period.

(2) In each of the above-described embodiments, the hue and the saturation are illustrated as the display modes of the unit image UA and the unit image UB, but the display modes of the unit image UA or the unit image UB are not limited to the above examples. For example, it may be at least one of hue, saturation, and lightness, or any combination thereof. Further, it is also possible to express the similarity of the feature amount F in a display mode such as a pattern or light and dark (shade). As understood from the above description, the display mode of the unit image UA or the unit image UB means the property of the image that can be visually discriminated.

(3) In each of the above-described embodiments, the configuration in which the user specifies the sound information Z set in each setting period has been illustrated, but the method for specifying the sound information Z is not limited to the above example. For example, it is possible to specify the sound information Z by analyzing the acoustic signal X. For example, it is possible to set the chord name automatically estimated from the acoustic signal X as the sound information Z in each set period by using a known chord identification technique. A configuration in which the user can arbitrarily change the sound information Z automatically specified by the analysis of the acoustic signal X as an initial value is also preferable.

(4) In each of the above embodiments, the chord name is illustrated as the sound information Z, but the content of the sound information Z is not limited to the above example. For example, it is possible to set, as the sound information Z, the type of musical instrument (instrument name) that is predominantly sounded within the set period or the note name that is predominantly sounded within the set period as the sound information Z for each set period. Further, assuming an acoustic signal X representing a human uttered sound, it is possible to set the phoneme (pronounced content) produced within the set period as the sound information Z for each set period. As understood from the above description, the sound information Z includes various information (labels) set in each set period.

(5) In each of the above-described embodiments, the distribution image GB is generated using the multidimensional scaling method (MDS), but the method of generating the distribution image GB is not limited to the above examples. For example, in each of the above-described embodiments, the hue circle is used to select the display mode of each unit image UB of the distribution image GB, but as illustrated in FIG. It is also possible to use an image whose saturation changes (hereinafter referred to as "colored image") for selecting the display mode of each unit image UB. Specifically, the display control unit 24 arranges the colored image so as to overlap the distribution image GB, and determines the hue and the saturation of the point where each unit image UB is located in the colored image as the hue and the saturation of the unit image UB. Select as degree.

  Further, for example, it is also possible to classify (cluster) the plurality of feature amounts F extracted by the feature extraction unit 22 into a plurality of clusters by a known technique such as k-means and select the display mode for each cluster. For example, the hue circle is divided into a plurality of regions corresponding to different clusters, and the unit image UA and the unit image UB representing the unit period in which the feature amount F belonging to each cluster is extracted It is also possible to assign a color within the area corresponding to. With the above configuration, the unit images (UA, UB) in each unit period in which the feature amount F is classified into the common cluster are displayed in a display mode that is similar to each other.

  As can be understood from the above example, the distribution image GB is configured so that the display modes of the unit images UB corresponding to each of the plurality of unit periods are similar between the unit images UB of the unit periods having similar feature amounts F. And is comprehensively represented as an image arranged on the coordinate plane at intervals according to the distance of the feature amount F in each unit period.

(6) The sound information setting device 100 illustrated in each of the above-described modes is realized by the cooperation of the control device 10 and a program as illustrated in each of the above-described modes. A program according to a preferred aspect of the present invention includes a feature extraction unit 22 that extracts a feature amount F of an acoustic signal X for each of a plurality of unit periods, and unit images (UA, UB) corresponding to each of the plurality of unit periods. A display control unit 24 for displaying the arranged analysis images (GA, GB) on the display device 14 so that the display mode of the unit image of each unit period becomes closer as the feature amount F of each unit period becomes similar, and The computer is caused to function as the information setting unit 26 that sets the sound information Z for a set period including one or more unit periods corresponding to the unit image selected by the user from the analysis image.

  The programs exemplified above may be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary recording medium such as a semiconductor recording medium or a magnetic recording medium is used. Recording media of the form It is also possible to distribute the program to the computer in the form of distribution via a communication network.

100 ... Sound information setting device, 10 ... Control device, 12 ... Storage device, 14 ... Display device, 16 ... Input device, 22 ... Feature extraction part, 24 ... Display control part, 26 ... Information setting part.

Claims (8)

Computer system
The feature amount of the acoustic signal is extracted for each of the plurality of unit periods,
A display device such that the analysis image in which unit images corresponding to each of the plurality of unit periods are arranged is closer to the display mode of the unit image in each unit period as the feature amount of each unit period is similar. To display
A sound information setting method for setting sound information for a set period including one or more unit periods corresponding to the unit image selected by the user from the analysis image. Computer system
The feature amount of the acoustic signal is extracted for each of the plurality of unit periods,
The time-series image in which the first unit image corresponding to each of the plurality of unit periods is arranged along the time axis, and the second unit image corresponding to each of the plurality of unit periods include the feature amount of each unit period. The distribution image arranged on the coordinate plane at intervals according to the distance, the more similar the feature amount of each unit period is, the more similar the display mode of the first unit image of each unit period is, and each unit unit Display on the display device so that the display mode of the second unit image during the period is approximate,
Sound information that sets sound information for a set period including one or more unit periods corresponding to the first unit image selected by the user from the time-series image or the second unit image selected by the user from the distribution image Setting method. When the first unit image in the time-series image is selected, the display mode of the first unit image is changed, and the first unit image corresponding to the unit period common to the first unit image in the distribution image is selected. The sound information setting method according to claim 2, wherein the display mode of the two-unit image is changed. When the second unit image in the distribution image is selected, the display mode of the second unit image is changed, and the second unit image corresponding to the unit period common to the second unit image is selected. The sound information setting method according to claim 2, wherein the display mode of the one-unit image is changed. When the user selects the first unit image of the time-series image, a setting period including a unit period represented by the first unit image, and another setting including a unit period having a feature amount similar to the unit period The period is set, and the sound information setting method according to claim 2. The sound information setting method according to claim 1, wherein the sound information includes a chord name. A feature extraction unit that extracts the feature amount of the acoustic signal for each of the plurality of unit periods,
A display device such that the analysis image in which unit images corresponding to each of the plurality of unit periods are arranged is closer to the display mode of the unit image in each unit period as the feature amount of each unit period is similar. A display control unit to be displayed on
A sound information setting device, comprising: an information setting unit that sets sound information for a set period including one or more unit periods corresponding to the unit image selected by the user from the analysis image. A feature extraction unit that extracts the feature amount of the acoustic signal for each of the plurality of unit periods,
The time-series image in which the first unit image corresponding to each of the plurality of unit periods is arranged along the time axis, and the second unit image corresponding to each of the plurality of unit periods include the feature amount of each unit period. The distribution image arranged on the coordinate plane at intervals according to the distance, the more similar the feature amount of each unit period is, the more similar the display mode of the first unit image of each unit period is, and each unit unit A display control unit that causes the display device to display such that the display mode of the second unit image during the period is approximated;
Information setting for setting sound information for a set period including one or more unit periods corresponding to the first unit image selected by the user from the time-series image or the second unit image selected by the user from the distribution image Information setting device including a section.

Images