JP7635786B2 - AUDIO PROCESSING SYSTEM, AUDIO PROCESSING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an audio processing system, an audio processing method, and a program , and more particularly to a method for acquiring a head related impulse response (HRIR) for reproducing a sound with a three-dimensional effect using, for example, headphones, earphones, or the like.

Conventionally, methods for reproducing sound with a three-dimensional effect have been proposed. For example, Patent Document 1 discloses a method of convolving the transfer characteristics of sound waves from a sound source to the left and right ears of a listener as head-related transfer functions with source sounds to synthesize two-channel acoustic signals. By presenting sounds based on the acoustic signals obtained by this method to the left and right ears of a listener, sound image localization in the direction of the sound source is realized. HRIR corresponds to a representation of the head-related transfer function as an impulse response in the time domain.
On the other hand, it is known that even if an audio signal is synthesized using a common HRIR in advance, the desired stereoscopic effect cannot be obtained for more than 30% of listeners. Therefore, it is important to obtain HRIRs for each listener.

Patent Document 1 describes a measurement device that measures the head-related transfer function by placing microphones at the ears of a subject and receiving signals from a pseudo sound source emanating from around the subject's head. This measurement device has a sound source memory section, an output control section, an output section, an input section, a key input section that supports the start of measurement, an input control section that records measurement data in the data memory section, and a reflector. This measurement device is characterized in that, when measuring in a measurement room that has not been subjected to anechoic processing, it starts input in synchronization with the output of the output control section, finishes data input before reverberation reaches the input section, and a reflector is placed in the path of the reflected sound that reaches the input section within the measurement time.

Japanese Patent Application Publication No. 8-307988

However, a special environment is required to obtain the HRIR. A special environment refers to an environment without reflection, such as an anechoic chamber, or an environment with so little reflection that its influence can be ignored. If the HRIR includes the influence of reflection, it will not be possible to reproduce the sound waves arriving at each of the left and right ears from the target direction targeted by the sound image localization. In this regard, the measuring device described in Patent Document 1 reduces the influence of reflection by placing a reflector in the path of the reflected sound, thereby blocking reflections to the microphone. For this reason, placing a reflector in the path of the reflected sound is difficult for anyone other than an acoustic engineer with knowledge of the path of the reflected sound.

The present invention has been made to solve the above-mentioned problems. It is an object of the present invention to provide an audio processing system, an audio processing method, and a program that can more easily obtain an HRIR.

According to a first aspect of the present invention, an acoustic processing system includes a sound source that emits a sound based on a predetermined sound source signal, a sound receiving unit that acquires a received signal of the arriving sound, a calculation unit that acquires an impulse response from the received signal from the sound source to the sound receiving unit, a sound-absorbing material that covers an object on the path of a reflected sound that arrives at the sound receiving unit at least within a predetermined observation period from the generation of the sound, a measurement environment information acquisition unit that performs image recognition processing on an image of a measurement environment of the impulse response to determine the material of a subject that corresponds to the sound-absorbing material in the measurement environment and the distribution of the material, a reflectance storage unit that stores reflection characteristic related data including information on the sound reflectance for each material, and a measurement environment determination unit that refers to the reflection characteristic related data, identifies a reflectance corresponding to the identified material, and determines whether to measure the impulse response based on the average reflectance within an evaluation area calculated from the distribution of the material.

According to a second aspect of the present invention, an acoustic processing method is an acoustic processing method for an acoustic processing system including a sound-absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving unit within at least a predetermined observation period from the generation of sound from a sound source, the method comprising: a first step in which the sound source emits a sound based on a predetermined sound source signal; a second step in which the sound receiving unit acquires a received signal of the arriving sound; a third step in which an arithmetic processing unit acquires an impulse response from the sound source to the sound receiving unit from the received signal; a fourth step in which a measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine the material of the subject that corresponds to the sound-absorbing material in the measurement environment and the distribution of the material; and a fifth step in which a measurement environment determination unit refers to reflection characteristic related data including information on the sound reflectance for each material stored in a reflectance storage unit, identifies a reflectance corresponding to the identified material, and determines whether or not to measure the impulse response based on an average reflectance within the evaluation area calculated from the distribution of the material.

According to a third aspect of the present invention, a program causes a computer of an acoustic processing system including a sound-absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving unit within at least a predetermined observation period from the generation of sound from a sound source to execute the following steps: a first step in which the sound source emits a sound based on a predetermined sound source signal; a second step in which the sound receiving unit acquires a received signal of the arriving sound; a third step in which an arithmetic processing unit acquires an impulse response from the sound source to the sound receiving unit from the received signal; a fourth step in which a measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine the material of the subject that corresponds to the sound-absorbing material in the measurement environment and the distribution of the material; and a fifth step in which a measurement environment determination unit refers to reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, identifies the reflectance corresponding to the identified material, and determines whether or not to measure the impulse response based on the average reflectance within the evaluation area calculated from the distribution of the material.

According to one aspect of the present invention, HRIR can be obtained more easily.

1 is a plan view illustrating an example of the configuration of a sound processing system according to a first embodiment. 1 is a side view illustrating an example of the configuration of a sound processing system according to a first embodiment. 1 is a schematic block diagram illustrating an example of a hardware configuration of a sound processing device according to a first embodiment. 1 is a schematic block diagram illustrating an example of a functional configuration of a sound processing device according to a first embodiment. 5 is a flowchart illustrating an example of an HRIR measurement process according to the first embodiment. FIG. 11 is a side view illustrating an example of the configuration of a sound processing system according to a second embodiment. FIG. 11 is a schematic block diagram illustrating an example of a functional configuration of a control unit according to a second embodiment. 13 is a flowchart illustrating an example of a measurement environment determination process according to the second embodiment. FIG. 2 is an explanatory diagram showing an example of a configuration of the embodiment.

Hereinafter, sound processing systems according to multiple embodiments of the present invention will be described with reference to the drawings.
First Embodiment
The sound processing system according to the first embodiment includes a sound source for emitting sound, a sound receiving unit for acquiring a received signal of the arriving sound, a sound absorbing material for absorbing the arriving sound, and a sound processing device. An example of the functional configuration of the sound processing device 10 will be described later.
Fig. 1 is a plan view showing a configuration example of a sound processing system 1 according to this embodiment. Fig. 2 is a side view showing a configuration example of a sound processing system 1 according to this embodiment. Fig. 2 shows a side surface passing through line AA' in Fig. 1 and parallel to the vertical direction.

1 and 2, the sound processing system 1 is used in an indoor space Rm. The space Rm may be a room that is not provided with any particular soundproofing or reverberation equipment as a whole. In other words, the space Rm is not limited to a soundproof room or a laboratory, but may be any of an office, a classroom, a meeting room, a living room, etc. The space Rm has an approximately rectangular parallelepiped shape and a rectangular shape in a plan view. The space Rm is surrounded by four side surfaces, one bottom surface, and one top surface. The top surface is a surface facing the bottom surface, and is shown above the bottom surface in Figs. 1 and 2. The side surfaces, bottom surface, and top surface are collectively referred to as inner surfaces. The side surfaces, bottom surface, and top surface form the wall surface, floor surface, and ceiling, respectively. In the space Rm, two speakers Sp, sound absorbing material As, and a seat Pf are arranged as sound sources for emitting sound.

The two speakers Sp are installed at positions close to each other within a predetermined range from one side (hereinafter referred to as the speaker installation side) that is the intersection line between two adjacent side surfaces (hereinafter referred to as the adjacent side) among the four side surfaces of the space Rm. However, the two speakers Sp are installed at different heights in the vertical direction as shown in FIG. 2. The two speakers Sp may be distinguished from each other by calling them speakers Sp01, Sp02, etc. In FIG. 1, of the two speakers Sp, speaker Sp01 is shown, and speaker Sp02, which is installed at a lower position than speaker Sp01, is not shown. A playback signal is input from the sound processing device 10 to one of the speakers Sp (speaker Sp01 in the example shown in FIG. 1). The speaker Sp radiates (plays) sound based on the sound source signal input from the sound processing device 10.

The seat Pf is installed so that its center faces each speaker Sp at a predetermined distance (e.g., 1 to 2 m) on the plane bisecting the angle formed by the adjacent planes. The seat Pf has a seat plate and a backrest, which enable the subject Sb to be supported stationary in a seated position. The subject Sb is the person for whom the HRIR from the speaker Sp to each of the left and right ears is to be measured.

The sound-absorbing material As is installed so as to cover at least a part of the inner surface of the space Rm. In the example shown in Figures 1 and 2, the sound-absorbing material As is installed so as to cover a part of the bottom surface and a part of the adjacent surface. Although not shown, a sound-absorbing material may be installed to cover the seat Pf. The sound-absorbing material As includes a material with a sufficiently low reflectance of the incoming sound. The sound-absorbing material As may be any material that converts most of the energy of the vibration of the incoming sound into heat, thereby absorbing the energy of the sound and dissipating it as heat. For example, glass wool, rock wool, urethane sponge, etc. can be used as the sound-absorbing material As. The sound-absorbing material may also be called a sound-insulating material, a sound-proofing material, etc. The area in which the sound-absorbing material As is installed will be described later.

A microphone Mic is attached to each of the left and right ears of the subject Sb as a sound receiving unit. The microphones attached to the left and right ears are distinguished by being called microphones MicL and MicR, respectively. The microphones MicL and MicR each receive sounds that reach them and convert the received sounds into received signals. The microphones MicL and MicR each output the converted received signals to the sound processing device 10. In the following description, the received signals output from the microphones MicL and MicR may be referred to as the left received signal and the right received signal, respectively.

Sound waves emitted from the speaker Sp propagate along various paths to the microphones MicL and MicR. When no object is placed between the speaker Sp and the microphones MicL and MicR as illustrated in Figure 1, a portion of the sound waves emitted from the speaker Sp arrives at the microphones MicL and MicR as direct sound. Another portion of these sound waves is reflected by the surface of an object placed on the path and arrives at the microphones MicL and MicR as reflected sound.

Here, it is assumed that a sound wave S emitted by a speaker Sp has a propagation path including a certain reflection point on the surface of an object and is reflected at the reflection point. In this case, a reflected wave _Srn observed at a sound receiving point where any microphone Mic is installed is expressed by the following formula (1).

In formula (1), R and L respectively indicate the reflectance of the sound wave at the reflection point and the attenuation rate according to the distance.
Generally, there are an infinite number of reflection points, so the reflected wave Sr arriving at the sound receiving point corresponds to a combination of the reflected waves _Srn from each reflection point, as shown in formula (2). In formula (2), Σ indicates the sum between the reflection points. A collection of these reflection points corresponds to the surfaces of various reflectors that reflect the sound arriving from the speaker Sp.

Therefore, the sound wave S' arriving at the receiving point is a combination of the direct wave S component from the speaker Sp and the reflected wave Sr component, as shown in equation (3).

On the other hand, the head impulse response indicates the propagation characteristics of sound waves from a sound source to each ear of the subject Sb. In other words, the head impulse response includes the propagation characteristics due to the reflection and diffraction of sound waves arriving from the sound source on the surface of the subject Sb's head (e.g., the auricle, etc.). To obtain a pure head impulse response, it is expected that the influence of reflected waves will be removed. Conventionally, it has been common to measure the head impulse response in an environment where no reflection of sound waves occurs, such as an anechoic chamber. This is because the contribution of reflected sound from objects installed on the propagation path of the sound waves is not included.

The sound processing system 1 according to the present embodiment reduces the components of reflected sound from other objects during a period in which the head impulse response includes at least components due to reflection and diffraction of sound waves at the head (hereinafter referred to as the head transfer period). Therefore, a period including at least the head transfer period is preset as the observation period of the head impulse response in the sound processing device 10. The head transfer period, depending on the size of the human head, is typically a period of about 1 to 3 ms starting from the time when the direct wave arriving from the speaker Sp first arrives at the head of the subject Sb. Therefore, by covering the area of the reflector that brings about the reflected sound arriving at the microphones MicL and MicR at least during the observation period with the sound absorbing material As, the components of the reflected sound can be reduced. Even when the sound absorbing material As is installed, it does not interfere with the use of the space Rm as much as a reflector, and the work load associated with its installation is light. Therefore, as illustrated in FIG. 1, the HRIR can be obtained even in an ordinary environment such as a corner of the space Rm. However, the positional relationship between the sound source and the seat Pf or the subject Sb is not limited to that illustrated in Figures 1 and 2. For example, the seat Pf may be installed in the center of the space Rm. The shape of the space Rm is also not limited to that illustrated in Figures 1 and 2. For example, the shape of the space Rm may be a cylinder.

1 and 2, the area where the sound absorbing material As is installed includes a square partial area of the bottom surface from the speaker installation side to the seat Pf, and an area extending from the part of each of the two adjacent surfaces that contacts the partial area of the bottom surface to a height that is a predetermined height higher than the height of the higher of at least two speakers Sp01 and Sp02. The time from the time when the speaker emits sound to the start point of the observation period (hereinafter, the observation start time) is calculated based on the distance from the speaker Sp to the subject Sb and the sound speed. The observation start time may be set in advance in the sound processing device 10, or may be calculated based on the position of the speaker Sp and the position of the seat Pf or the subject Sb. In addition, the area where the sound absorbing material As is installed may include the part where the reflected sound arrives at the microphones MicL and MicR during the observation period, and the size and shape of the area are not limited to those exemplified in FIG. 1 and FIG. 2. For example, the shape of the area where the floor surface is covered with the sound absorbing material As may be a sector shape as exemplified in FIG. 9, and does not necessarily have to include the position of the seat Pf.

The intensity of the reflected sound arriving at each of the microphones MicL and MicR from the sound-absorbing material As is not completely zero, but is sufficiently low compared to the intensity of the direct sound. In general, when two sounds of different intensities are presented to a subject, the weaker sound is drowned out by the stronger sound, and a phenomenon (masking) is known in which the weaker sound cannot be heard. With the sound-absorbing material As, the S/N ratio when the direct sound component is the signal component and the reflected sound component is the noise component can be reduced to a level where the reflected sound cannot be perceived. Therefore, the head impulse response acquired by the sound processing system 1 according to this embodiment can also achieve a three-dimensional effect on the auditory sense that is the same as the head impulse response measured in an anechoic chamber.

Next, the configuration of the sound processing device 10 according to this embodiment will be described.
FIG. 3 is a schematic block diagram showing an example of the hardware configuration of the sound processing device 10 according to the present embodiment.
The sound processing device 10 includes a control unit 110, a storage unit 130, an input/output unit 140, a display unit 150, and an operation unit 160. The sound processing device 10 may be configured as a dedicated device, or may be realized by a general-purpose computer.

The control unit 110 performs processing to realize and control various functions of the sound processing device 10. The control unit 110 is capable of executing processing instructed by instructions (commands) written in various programs such as application programs, and executes and controls the operation of each unit constituting the sound processing device 10. The control unit 110 is configured to include one or more processors, such as a CPU (Central Processing Unit), for example.

The memory unit 130 stores various data used by the control unit 110, various data acquired by the control unit 110, various programs, etc. The memory unit 130 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), etc.

The input/output unit 140 is connected to a device separate from the sound processing device 10 via a wired or wireless connection, and inputs and outputs various types of data. The input/output unit 140 may be connected to a communication network via a wired or wireless connection, and transmits and receives various types of data to and from the separate device connected to the communication network. The input/output unit 140 is configured to include input/output devices such as an input/output interface and a communication interface, for example.

Under the control of the control unit 110, the display unit 150 displays information indicated by various display data input from the control unit 110 in a visually recognizable manner. Such display data includes image data, text data, and the like. The display unit 150 is configured to include any type of display device, such as a liquid crystal display (LCD) or an organic electro-luminescence display (OLED).

The operation unit 160 accepts a user operation, generates an operation signal corresponding to the accepted operation, and outputs the generated operation signal to the control unit 110. The operation signal indicates various information such as a user's instructions or commands to the sound processing device 10. The operation unit 160 includes an operation device in any form, such as a push button or a stick key. The operation unit 160 may be configured to include a receiver that receives wirelessly (including infrared) from a physically separated operation device (e.g., a remote controller).

Next, an example of the functional configuration of the sound processing device 10 according to the present embodiment will be described.
4 is a schematic block diagram showing an example of the functional configuration of the sound processing device 10 according to this embodiment. The sound processing device 10 includes a control unit 110 including a measurement control unit 112, a recording unit 114, and a calculation unit 116.

The measurement control unit 112 performs various controls related to the measurement of the head impulse response. When an operation signal indicating the start of measurement is input from the operation unit 160, the measurement control unit 112 generates a predetermined sound source signal to be used for measurement. The sound source signal is, for example, an impulse signal. An impulse signal is a signal whose signal value at a certain sample time is significantly different from zero (non-zero value) and whose signal value at other times is zero. The measurement control unit 112 outputs the generated sound source signal to the speaker Sp via the input/output unit 140. The speaker Sp emits sound according to the input sound source signal. The sample time at which the signal value is non-zero corresponds to the time at which the impulse sound is emitted from the speaker Sp.

2, when there are multiple speakers Sp, the measurement control unit 112 may specify the speaker Sp that emits the sound as the output destination of the sound source signal. The speaker Sp to be the output destination may be specified by an operation signal input from the operation unit 160, or may be selected according to a predetermined order.

The seat Pf may have a horizontal plane parallel to the floor surface and a rotatable seat plate. The subject Sb seated on the seat plate can rotate to change the direction in the horizontal plane relative to the speaker Sp. The subject Sb may operate the operation unit 160 to instruct the start of measurement each time the direction is changed by a predetermined angle (e.g., 3 to 60°). This makes it easy to measure HRIRs with different directions in the horizontal plane. The measurement control unit 112 may acquire directional information indicating the direction of the seat Pf, output the acquired directional information to the calculation unit 116, and store the directional information in the memory unit 130 in association with the measured HRIR for the calculation unit 116.

The recording unit 114 stores (records) the left and right received audio signals input from the microphones MicL and MicR via the input/output unit 140, respectively.
The microphones MicL and MicR receive the sounds that are directly or indirectly transmitted from the speaker Sp. Thus, the left and right received signals are input from the microphones MicL and MicR to the recording unit 114. When an impulse signal is used as the sound source signal, the left and right received signals indicate the HRIR from the speaker Sp to the microphones MicL and MicR. Thus, the recording unit 114 may start recording the left and right received signals when an operation signal indicating the start of measurement is input from the operation unit 160, and stop recording after a predetermined recording period (for example, 0.2 to 2 seconds) has elapsed since the start of recording. The recording period may be at least as long as the observation period.

After an operation signal indicating the start of measurement is input from the operation unit 160 and the above-mentioned recording period has elapsed, the calculation unit 116 reads the newly recorded received sound signal from the recording unit 114. The calculation unit 116 obtains the HRIR from the read received sound signal. As described above, the reflected sound contained within the observation period is significantly attenuated by the sound absorbing material As. However, components of the reflected sound that arrive after the above-mentioned observation period may remain in the received sound signal.

Therefore, the calculation unit 116 may gradually attenuate (fade out) the signal values of the impulse response obtained from the received signal for a period after the observation period. For example, the calculation unit 116 attenuates the signal value of the received signal from the end point of the observation period over a predetermined attenuation time (for example, 10 to 100 ms) depending on the passage of time until it reaches zero, and discards the signal value at the time thereafter. Thus, the influence of the reflected sound that appears in the received signal and arrives after the observation period (hereinafter, the late reflected sound) is removed, and the calculation unit 116 can obtain a significant portion as the HRIR. The components of the late reflected sound include, for example, reflected sound from the ceiling, the back of the subject Sb, that is, reflected sound from the opposite direction from the seat Pf to the speaker Sp, and the like. The calculation unit 116 stores the HRIR signal indicating the obtained HRIR in the storage unit 130.

Next, an example of the HRIR measurement process according to this embodiment will be described.
5 is a flowchart showing an example of the HRIR measurement process according to this embodiment. However, before starting the following process, the reflecting objects (i.e., the floor, wall, seat Pf, etc.) at the locations where the reflected sound arrives in response to the impulse arriving at the microphones MicL and MicR attached to the left and right ears of the subject Sb, respectively, within at least an observation period from the generation of the impulse from the speaker Sp are covered with a sound absorbing material As.

(Step S102) The measurement control unit 112 outputs an impulse signal to the speaker Sp. The speaker Sp emits an impulse as a test sound based on the impulse signal.
(Step S104) The microphones MicL and MicR receive the sounds that have arrived at them, and obtain left and right received signals that indicate the received sounds. The microphones MicL and MicR output the left and right received signals to the recording unit 114, respectively.
(Step S106) The recording unit 114 records the left and right received signals each including the HRIR.
(Step S108) The calculation unit 116 removes late reflections by attenuating and discarding signal values for the period after the observation period from each of the left and right received signals newly recorded in the recording unit 114, thereby obtaining the HRIR.

Note that the received signal may contain noise arriving at the microphone Mic from the surroundings. Therefore, after the measurement start is instructed, the measurement control unit 112 may repeat the HRIR measurement process from step S102 to step S108 a predetermined number of times or more. As the number of repetitions, a predetermined number of times of two or more is set in advance in the measurement control unit 112. Then, after each HRIR measurement process is completed, the calculation unit 116 outputs an HRIR acquisition completion notification to the measurement control unit 112. Each time an HRIR acquisition completion notification is input, the measurement control unit 112 counts the number of measurements by adding (incrementing) one by one. However, the initial value of the number of measurements is set to 0. Then, the measurement control unit 112 determines whether the number of measurements has reached the number of repetitions, and when it is determined that the number has been reached, it outputs a measurement completion notification to the calculation unit 116. When the measurement control unit 112 determines that the number has not been reached, it starts the next HRIR measurement process.

When the measurement end notification is input from the measurement control unit 112, the calculation unit 116 generates a new HRIR signal including, as a signal value, the sum obtained by adding the signal values for each time included in the HRIR signal for the number of repetitions acquired between repetitions of the HRIR measurement process for each time (synchronous addition). The calculation unit 116 may also generate an HRIR signal including, as a new signal value instead of the original signal value, the quotient obtained by dividing the signal value for each time by the number of repetitions (normalization). The calculation unit 116 stores the generated HRIR in the storage unit 130. The synchronous addition reduces the noise components that randomly overlap between repetitions of the HRIR measurement process relatively to the signal components of the HRIR. Therefore, the signal-to-noise ratio (SNR) is improved, and the influence of noise can be reduced.

Second Embodiment
Next, the second embodiment will be described, focusing mainly on the differences from the first embodiment. Unless otherwise specified, the same components and processes as those in the first embodiment are denoted by the same reference numerals and the description thereof will be repeated.

FIG. 6 is a side view showing an example of the configuration of the sound processing system 1 according to this embodiment.
The sound processing system 1 according to the present embodiment includes a speaker Sp as a sound source, microphones Mic L and Mic R (however, Mic R is not shown in FIG. 6) as sound receiving units, a sound absorbing material As, and a sound processing device 10, as well as an image acquisition unit Cm as an example of a measurement environment information acquisition unit. The image acquisition unit Cm captures an image of the measurement environment of the HRIR as an example of measurement environment information. The image acquisition unit Cm is, for example, a digital still camera capable of capturing still images. The measurement environment may include an area (hereinafter, evaluation area) in which reflectors that bring about reflected sounds that arrive at the microphones Mic L and Mic R within an observation period, among the reflected sounds of the sound emitted from the speaker Sp, are distributed or arranged. In other words, the image acquisition unit Cm may be positioned and oriented so as to include the evaluation area in the field of view in the space Rm, and the field of view angle may be set.

The sound processing device 10 judges the measurement environment, particularly the arrangement of the sound absorbing material in the evaluation area, based on the images captured by the image acquisition unit Cm. The sound processing device 10 estimates the reflection characteristics from the arrangement by referring to reflection characteristic related data indicating the relationship between the arrangement and the reflection characteristics. The sound processing device 10 then judges whether or not to measure the HRIR based on whether or not the estimated reflection characteristics are greater than a predetermined reflection intensity. This identifies whether or not it is possible to measure the HRIR that can be used to achieve a three-dimensional effect.

Next, a functional configuration example of the sound processing device 10 according to the present embodiment will be described below. Fig. 7 is a schematic block diagram showing a functional configuration example of the control unit 110 provided in the sound processing system 1 according to the present embodiment.
The control unit 110 includes a measurement control unit 112 , a sound recording unit 114 , a calculation unit 116 , a reflectance storage unit 124 , and a measurement environment determination unit 126 .

The reflection characteristic related data is stored in advance in the reflectance storage unit 124. The reflection characteristic related data is data that includes information indicating, for each material, the correspondence between the material and the reflectance of the sound waves arriving at that material. The reflectance is the ratio of the intensity of the reflected waves of the incoming sound waves to the intensity of the incoming sound waves. The reflection characteristic related data needs to include information relating to at least one type of material that can be a sound absorbing material. The reflection characteristic related data may or may not include information relating to materials that are not sound absorbing materials. Sound absorbing materials correspond to materials whose reflectance is sufficiently lower than a predetermined reflectance (e.g., -30 to -50 dB).

The measurement environment determination unit 126 receives image data representing an image captured by the image acquisition unit Cm. The measurement environment determination unit 126 executes a predetermined image recognition process on the input image data to determine the placement state of the sound-absorbing material in the measurement environment. More specifically, the measurement environment determination unit 126 identifies the material of the object placed in each block obtained by subdividing the image shown in the image data into a predetermined size by the image recognition process. For example, image recognition data indicating the relationship between the image feature amount and the material is set in advance in the measurement environment determination unit 126. In the image recognition process, for example, the measurement environment determination unit 126 calculates the image feature amount for each block based on the pixel value of each pixel placed in the block, and can specify the material of the subject corresponding to the calculated image feature amount by referring to the image recognition data. Here, a collection of blocks in which the specified material is a material corresponding to the sound-absorbing material corresponds to the sound-absorbing material area, and the placement state of the sound-absorbing material is indicated by the distribution of the sound-absorbing material area.

The measurement environment determination unit 126 determines whether or not to measure the HRIR based on the placement information of the sound-absorbing material. More specifically, the measurement environment determination unit 126 refers to the reflection characteristic related data to determine the reflectance of the material specified for each block included in the evaluation area, and calculates the average value of the specified reflectance among the blocks in the evaluation area as the average reflectance. The measurement environment determination unit 126 determines whether or not to measure the HRIR depending on whether the calculated average reflectance is equal to or lower than a predetermined upper limit value of the reflectance (e.g., -25 to -40 dB). The measurement environment determination unit 126 has evaluation area information indicating the evaluation area in the captured image set in advance. As described above, the evaluation area is determined based on the positional relationship between the speaker Sp and the seat Pf, the distribution of reflectors (walls, sides, seats Pf, etc.) in the space Rm, the position, orientation, and size of the viewing angle of the image acquisition unit Cm.

The measurement environment determination unit 126 generates display data including determination information corresponding to a determination result indicating, for example, whether or not to measure the HRIR, and outputs the generated display data to the display unit 150. The display unit 150 notifies the user as a determination result whether the measurement environment is an appropriate environment, that is, whether or not it is an environment in which an HRIR that allows the listener to fully perceive the stereoscopic effect can be measured. Here, the user may be an operator involved in measuring the HRIR, or may be the subject Sb himself/herself.

The measurement environment determination unit 126 may output determination information indicating the determination result to the measurement control unit 112. When the determination information input from the measurement environment determination unit 126 indicates that the HRIR is to be measured, the measurement control unit 112 executes the above-mentioned HRIR measurement process, and when the determination information indicates that the measurement environment does not measure the HRIR, the measurement control unit 112 may not execute the above-mentioned HRIR measurement process. As a result, when the measurement environment is not appropriate, the measurement of the HRIR is restricted, and when the measurement environment is appropriate, the HRIR is measured. Therefore, the quality of the measured HRIR is ensured.

Next, an example of the measurement environment determination process according to this embodiment will be described. FIG. 8 is a flowchart showing an example of the measurement environment determination process according to this embodiment. However, before starting execution of the process shown in FIG. 8, reflection characteristic related data is saved in advance. The reflection characteristic related data includes information indicating the reflectance of each material.

(Step S202) The image acquisition unit Cm captures an image of the measurement environment, and outputs image data indicating the captured image to the sound processing device 10.
(Step S204) The measurement environment determination unit 126 performs image recognition processing on the image data input from the image acquisition unit Cm to identify the material of the subject in the measurement environment represented in the field of view for each block into which the image is divided. The measurement environment determination unit 126 determines the reflectance of the material identified for each block by referring to the reflection characteristic related data.

(Step S206) The measurement environment determination unit 126 determines an average reflectance as an example of a representative value of the reflectance in the evaluation area based on the reflectance determined for each block in the predetermined evaluation area. The measurement environment determination unit 126 determines whether the determined average reflectance is equal to or less than a predetermined upper limit of the reflectance. When the measurement environment determination unit 126 determines that the average reflectance is equal to or less than the upper limit, it determines that the measurement environment measures the HRIR, and when the measurement environment determination unit 126 determines that the average reflectance is greater than the upper limit, it determines that the measurement environment does not measure the HRIR.
The measurement environment determination section 126 may output determination information indicating the determination result to the display section 150 or to the measurement control section 112 .

In addition, when the measurement environment determination unit 126 determines whether or not the measurement environment is an environment for measuring HRIR, the measurement environment determination unit 126 has determined the reflectance in the evaluation area as an example of an index of the placement state of the sound absorbing material, but this is not limited to this. For example, the measurement environment determination unit 126 may determine the coverage rate of a sound absorbing material of a predetermined material in the evaluation area as an example of an index of the placement state of the sound absorbing material. Here, the measurement environment determination unit 126 may preset sound absorbing material information for each component indicating whether the component is a sound absorbing material or not. In this case, it is not necessary to previously store reflection characteristic related data in the reflectance storage unit 124.

The measurement environment determination unit 126 performs image recognition processing on the image data as described above, and identifies the material of the subject for each block. The measurement environment determination unit 126 can identify whether the identified material is a sound absorbing material by referring to the sound absorbing material information set in the measurement environment determination unit 126. The measurement environment determination unit 126 determines a set of blocks in which the identified material is determined to be a sound absorbing material as a sound absorbing area. The measurement environment determination unit 126 can calculate the ratio of the area of the evaluation area to the area of the sound absorbing area included in the evaluation area as the sound absorbing area ratio. Then, the measurement environment determination unit 126 determines whether the sound absorbing area ratio is equal to or greater than a lower limit of a predetermined ratio (for example, 0.8 to 0.98) as an index value indicating the arrangement state of the sound absorbing material. When the measurement environment determination unit 126 determines that the sound absorbing area ratio is equal to or greater than the predetermined lower limit, it determines that the measurement environment is an environment in which the HRIR is measured, and when the sound absorbing area ratio is equal to or less than the predetermined lower limit, it determines that the measurement environment is an environment in which the HRIR is not measured.

Next, an example of the configuration of the above embodiment will be described. FIG. 9 is an explanatory diagram showing an example of the configuration. The sound processing system 1 according to the above embodiment includes a sound source (e.g., a speaker Sp), a sound receiving unit (e.g., a microphone Mic), a calculation unit 116, and a sound absorbing material As. The sound source emits a sound based on a predetermined sound source signal. The sound receiving unit acquires a received sound signal of the arriving sound. The calculation unit 116 acquires an impulse response from the sound source to the sound receiving unit from the acquired received sound signal. The sound absorbing material covers an object on the path of a reflected sound that arrives at least within a predetermined observation period from the generation of the sound.
According to this configuration, the installation of the sound absorbing material reduces the intensity of the reflected sound component that arrives at least within the observation period in the impulse response compared to when the sound absorbing material is not installed. Also, the sound absorbing material is sufficient as long as it covers the object related to the propagation path of the reflected sound that arrives at least within the observation period among the propagation paths of the sound waves from the sound source to the sound receiving unit. Therefore, even in an ordinary space that does not have special equipment such as an anechoic chamber, it is possible to easily obtain the impulse response to be used as the HRIR. Here, the load related to the installation of the sound absorbing material is relatively small, and the space can be effectively utilized.

Furthermore, the calculation section 116 may attenuate the signal value of the acquired impulse response after the observation period has elapsed.
According to this configuration, the part of the acquired impulse response within the observation period is extracted, and the part after the observation period is removed, so that the part important for the perception of the three-dimensional effect of the sound as the HRIR within the observation period is maintained, and the influence of the late reflection sound arriving after the observation period can be avoided.

The sound processing system 1 may further include a measurement control unit 112 that repeats the generation of sound from a sound source, the acquisition of a received sound signal, and the acquisition of an impulse response multiple times. The calculation unit 116 may synchronously add up the multiple acquired impulse responses.
According to this configuration, the noise components added are relatively reduced compared to the components of the impulse response added by synchronous addition, so that a more accurate impulse response can be obtained.

The sound processing system 1 may further include a measurement environment information acquisition unit (e.g., an image acquisition unit Cm) and a measurement environment determination unit 126. The measurement environment information acquisition unit acquires information on the measurement environment of the impulse response. The measurement environment determination unit refers to preset reflection characteristic related data, determines the arrangement state of the sound absorbing material based on the acquired information, and determines whether the measurement environment is suitable for measuring the impulse response based on the determined arrangement state of the sound absorbing material.
According to this configuration, it is judged whether the measurement environment is appropriate for obtaining an impulse response that allows the listener to fully perceive the three-dimensional effect based on the arrangement of the sound absorbing material, and whether or not it is necessary to measure the impulse response is judged according to the judgment result. Therefore, the acquisition of an inappropriate impulse response is restricted, and the acquisition of an appropriate impulse response is encouraged.

The sound processing system 1 may also include a reflectance storage unit 124. The reflectance storage unit 124 stores reflection characteristic related data including information associating information on materials and sound reflectances. The measurement environment determination unit 126 identifies the material of the subject identified from the image as the acquired information, refers to the reflection characteristic related data to identify the reflectance corresponding to the identified material, and determines whether or not to measure an impulse response based on the identified reflectance.
According to this configuration, the material of the subject shown in the acquired image is identified, and whether or not to measure the impulse response is determined based on the reflectance determined from the identified material. Therefore, the reflectance in the measurement environment of the impulse response is estimated without performing actual measurement, and therefore it is possible to determine whether or not to measure the impulse response without changing the acoustic environment due to the actual measurement.

Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their variations are included in the scope of the invention and its equivalents as set forth in the claims, as well as in the scope and gist of the invention.

This application claims priority based on Japanese Patent Application No. 2020-156368, filed on September 17, 2020, the disclosure of which is incorporated herein in its entirety.

The present invention may be applied to an acoustic processing system, an acoustic processing method, and a recording medium.

Reference Signs List 1... Sound processing system 10... Sound processing device 110... Control unit 112... Measurement control unit 114... Sound recording unit 116... Calculation unit 124... Reflectance storage unit 126... Measurement environment determination unit 130... Memory unit 140... Input/output unit 150... Display unit 160... Operation unit As... Sound absorbing material Cm... Image acquisition unit Mic (Mic L, Mic R)... Microphone Sp... Speaker

Claims (5)

a sound source that emits a sound based on a predetermined sound source signal;
A sound receiving unit for acquiring a received signal of an incoming sound;
A calculation unit that obtains an impulse response from the sound source to the sound receiving unit from the received sound signal; a sound absorbing material that covers an object on a path of a reflected sound that arrives at the sound receiving unit within at least a predetermined observation period from the generation of the sound;
a measurement environment information acquisition unit that performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of a subject corresponding to a sound absorbing material in the measurement environment and a distribution of the material ;
A reflectance storage unit that stores data related to reflection characteristics including information on the sound reflectance for each material;
By referring to the reflection characteristic related data, a reflectance corresponding to the identified material is identified;
a measurement environment determination unit that determines whether or not to measure the impulse response based on an average reflectance within an evaluation area calculated from a distribution of the material. The calculation unit is
The sound processing system of claim 1 , wherein the signal value of the impulse response after the observation period has elapsed is attenuated. a measurement control unit that repeats the generation of sound from the sound source, the acquisition of the received sound signal, and the acquisition of the impulse response a plurality of times;
The calculation unit is
The sound processing system according to claim 1 or 2, wherein a plurality of the impulse responses are synchronously added. A sound processing method for a sound processing system including a sound absorbing material that covers an object on a path of a reflected sound that arrives at a sound receiving section within at least a predetermined observation period from the generation of the sound from a sound source, the method comprising: a first step in which the sound source emits a sound based on a predetermined sound source signal;
A second step in which the sound receiving unit acquires a received signal of the arriving sound;
a third step of acquiring an impulse response from the sound source to the sound receiving unit from the received sound signal by a calculation processing unit;
a fourth step in which the measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of the object corresponding to the sound absorbing material in the measurement environment and a distribution of the material ;
The measurement environment determination unit refers to the reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, and identifies the reflectance corresponding to the identified material;
and a fifth step of determining whether or not to measure the impulse response based on an average reflectance within the evaluation area calculated from the distribution of the material. a sound absorbing material that covers an object on a path of a reflected sound that arrives at the sound receiving unit within at least a predetermined observation period from the generation of the sound from the sound source,
A first step in which the sound source emits a sound based on a predetermined sound source signal;
A second step in which the sound receiving unit acquires a received signal of the arriving sound;
a third step of acquiring an impulse response from the sound source to the sound receiving unit from the received sound signal by a calculation processing unit;
a fourth step in which the measurement environment information acquisition unit performs image recognition processing on an image of the measurement environment of the impulse response to determine a material of the object corresponding to the sound absorbing material in the measurement environment and a distribution of the material ;
The measurement environment determination unit refers to the reflection characteristic related data including information on the sound reflectance for each material stored in the reflectance storage unit, and identifies the reflectance corresponding to the identified material;
a fifth step of determining whether or not to measure the impulse response based on an average reflectance within an evaluation area calculated from the distribution of the material;
A program that executes the following.

Images