JP7760261B2 - Acoustic sonification device and acoustic sonification program - Google Patents

Description

The present invention relates to an audio sonification device and an audio sonification program.

In the acoustic design of buildings, reverberation time and sound pressure distribution obtained using statistical acoustics theory and numerical calculations are evaluated to determine the specifications of components, plans, etc. However, a certain level of experience and knowledge is required to properly evaluate the results.

On the other hand, there is technology that uses the results of numerical calculations and model experiments to reproduce the sounds inside a building under design so that they can be heard, allowing intuitive evaluation even for those with little experience or knowledge. This technology is called acoustic auralization technology.

Previously, the following technologies were available for acoustic auralization:

Patent Document 1 discloses a spatial sound generation device that aims to generate a sound field with a realistic, three-dimensional acoustic wavefront.

This spatial audio generating device is equipped with a memory unit and a control unit connected to multiple speakers. The control unit, based on information indicating a moving sound source, applies an inverse system while changing the transfer characteristics over time in accordance with the movement of the sound source, and calculates multiple input signals to each speaker from a sound source signal indicating the sound emitted by the sound source. The inverse system also outputs the input signals in accordance with the transfer characteristics in the space in which the multiple speakers are located, so as to cause the speakers to form a three-dimensional acoustic wavefront based on the input signals in boundary sound field control.

Republished Publication No. 2018-070487

However, the technology disclosed in Patent Document 1 does not take into account the movement of the sound receiving point, and has the problem of not being able to reproduce the sound heard by a listener who virtually moves within the target room.

This disclosure has been made in consideration of the above circumstances, and aims to provide an acoustic auralization device and an acoustic auralization program that can reproduce the sound heard by a listener when the listener virtually moves.

The acoustic auralization device of the present invention as recited in claim 1 includes an acquisition unit that acquires shape information that indicates the shape of a target room, acoustic-related information related to the acoustic characteristics of the interior of the room, and sound source information that indicates the sound emitted from a target sound source; an identification unit that identifies the position of a sound receiving point, which is a location of a listener who checks the acoustic design effects of the room, as the sound receiving point moves; a derivation unit that derives acoustic auralization information that indicates how the sound emitted from the sound source will be heard at the sound receiving point, using the shape information, the acoustic-related information, and the sound source information acquired by the acquisition unit and the position of the sound receiving point identified by the identification unit; and a reproduction control unit that controls a reproduction unit to reproduce the sound indicated by the acoustic auralization information derived by the derivation unit, wherein the derivation unit derives the acoustic auralization information using a transfer function and assuming that the sound receiving point and the sound source are at the same position , and sets the initial height of the sound receiving point to the height of the listener's ears .

The acoustic auralization device of the present invention as described in claim 1 acquires shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the room's interior, and sound source information indicating the sound emitted from the target sound source, identifies the position of a sound receiving point, which is the location of a listener who will confirm the acoustic design effects of the room, as the point moves, and derives acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point using the acquired shape information, acoustic-related information, and sound source information and the identified position of the sound receiving point. By controlling a playback unit to play the sound indicated by the derived acoustic auralization information, it is possible to reproduce the sound heard by the listener when the listener virtually moves.

According to the acoustic auralization device of the present invention described in claim 1 , by deriving the acoustic auralization information using a transfer function, it is possible to reproduce the sound emitted from the sound source more easily than when a transfer function is not used.

The acoustic sonification device of the present invention as described in claim 2 is the acoustic sonification device as described in claim 1 , further comprising a generation unit that uses the shape information to generate video information showing an image of the room as seen by the listener in accordance with the listener's movement, and a display control unit that controls the display unit to virtually display the image shown by the video information generated by the generation unit.

According to the acoustic sonification device of the present invention described in claim 2 , the shape information is used to generate video information showing an image of the room as seen by the listener as the listener moves, and the image shown by the generated video information is virtually displayed, thereby making it possible to reproduce the sound emitted from the sound source with a more realistic feel than when the image is not displayed.

The acoustic auralization device of the present invention described in claim 3 is the acoustic auralization device described in claim 2 , in which the derivation of the acoustic auralization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer.

According to the acoustic auralization device of the present invention described in claim 3 , by deriving the acoustic auralization information and generating the video information using a single computer, it is possible to more easily synchronize the playback of the sound indicated by the acoustic auralization information with the display of the video indicated by the video information compared to when the derivation of the acoustic auralization information and the generation of the video information are performed using different computers.

The acoustic auralization device of the present invention described in claim 4 is an acoustic auralization device described in any one of claims 1 to 3 , wherein the derivation unit derives the acoustic auralization information as information that reflects the influence of the Doppler effect due to the movement of the sound source.

According to the acoustic sonification device of the present invention described in claim 4 , by making the acoustic sonification information information that reflects the influence of the Doppler effect due to the movement of the sound source, it is possible to reproduce the sound emitted from the sound source with higher accuracy than when the influence of the Doppler effect is not reflected.

A sound auralization program according to the present invention as set forth in claim 5 is a process of acquiring shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from a target sound source, identifying the position of a sound receiving point, which is a location of a listener who confirms the acoustic design effects of the room, as the sound receiving point moves, deriving acoustic auralization information indicating how the sound emitted from the sound source will be heard at the sound receiving point using the acquired shape information, acoustic-related information, and sound source information, and the identified position of the sound receiving point, and controlling a playback unit to play the sound indicated by the derived acoustic auralization information , in which the sound receiving point and the sound source are positioned identically, and the computer is caused to execute the process of deriving the acoustic auralization information using a transfer function and assuming that the sound receiving point and the sound source are in the same position , and setting the initial height of the sound receiving point to the height of the listener's ears .

According to the acoustic auralization program of the present invention as set forth in claim 5 , shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from the target sound source are acquired, the position of a sound receiving point, which is a location of a listener who confirms the acoustic design effects of the room, is identified as the sound receiving point moves, and acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point is derived using the acquired shape information, acoustic-related information, and sound source information, as well as the identified position of the sound receiving point.By controlling the playback unit to play the sound indicated by the derived acoustic auralization information, it is possible to reproduce the sound that would be heard by the listener when the listener virtually moves.

As described above, the present invention makes it possible to reproduce the sound heard by a listener when the listener virtually moves.

1 is a block diagram showing an example of a hardware configuration of a sound auralization device according to an embodiment. 1 is a block diagram showing an example of a functional configuration of a sound auralization device according to an embodiment. FIG. 2 is a front view showing an example of the configuration of a controller according to the embodiment. FIG. 2 is a schematic diagram illustrating an example of a configuration of a building-related information database according to the embodiment. FIG. 2 is a schematic diagram illustrating an example of a configuration of a transfer function related information database according to the embodiment. FIG. 10 is a perspective view showing an example of a setting target for calculating a transfer function according to the embodiment. FIG. 10 is a side view showing an example of a setting target for calculating a transfer function when wave properties are not taken into consideration according to the embodiment. FIG. 10 is a side view showing an example of a setting target for calculating a transfer function when wave properties are taken into consideration according to the embodiment. 10 is a flowchart illustrating an example of a setting process according to the embodiment. FIG. 10 is a front view showing an example of a configuration of a setting target input screen according to the embodiment. FIG. 10 is a front view showing an example of the configuration of a setting information input screen according to the embodiment. 10 is a flowchart illustrating an example of a sound auralization process according to the embodiment. FIG. 10 is a front view showing an example of a configuration of a processing target input screen according to the embodiment. FIG. 2 is a front view showing an example of the configuration of a video screen according to the embodiment. FIG. 10 is a front view showing another example of the configuration of the video screen according to the embodiment.

Below, examples of embodiments of the present invention will be described in detail with reference to the drawings.

First, the configuration of the sound auralizer 10 according to this embodiment will be described with reference to Figures 1 to 3. Figure 1 is a block diagram showing an example of the hardware configuration of the sound auralizer 10 according to this embodiment. Figure 2 is a block diagram showing an example of the functional configuration of the sound auralizer 10 according to this embodiment. Figure 3 is a front view showing an example of the configuration of the controller 21 according to this embodiment. Examples of the sound auralizer 10 include information processing devices such as personal computers and server computers.

As shown in FIG. 1, the acoustic sonification device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, memory 12 as a temporary storage area, non-volatile storage unit 13, input unit 14 such as a keyboard and mouse, display unit 15 such as an LCD display, medium read/write device (R/W) 16, and communication interface (I/F) unit 18. The CPU 11, memory 12, storage unit 13, input unit 14, display unit 15, medium read/write device 16, and communication interface (I/F) unit 18 are connected to one another via bus B. The medium read/write device 16 reads information written on a recording medium 17 and writes information to the recording medium 17.

The storage unit 13 is realized by a hard disk drive (HDD), solid state drive (SSD), flash memory, etc. A setting program 13A and an acoustic auralization program 13B are stored in the storage unit 13 as a storage medium. The setting program 13A is stored in the storage unit 13 when the recording medium 17 on which the setting program 13A is written is set in the medium reading and writing device 16 and the medium reading and writing device 16 reads the setting program 13A from the recording medium 17. The acoustic auralization program 13B is also stored in the storage unit 13 when the recording medium 17 on which the acoustic auralization program 13B is written is set in the medium reading and writing device 16 and the medium reading and writing device 16 reads the acoustic auralization program 13B from the recording medium 17. The CPU 11 reads each of the setting program 13A and the acoustic auralization program 13B from the storage unit 13 as appropriate, expands them in the memory 12, and sequentially executes the processes contained in each program.

The memory unit 13 also stores a building-related information database 13C and a transfer function-related information database 13D. Details of the building-related information database 13C and the transfer function-related information database 13D will be described later.

Furthermore, as shown in FIG. 1, a microphone (hereinafter referred to as "mic") 20, a controller 21, and a speaker 22 are connected to the communication I/F unit 18 according to this embodiment.

The microphone 20 according to this embodiment is used to collect voices emitted by the user of the sound auralization device 10 (corresponding to the "listener" of the present invention) and sound source information indicating sounds emitted from a sound source, which will be described later. In this embodiment, a unidirectional, condenser-type microphone is used as the microphone 20, but this is not limited to this. For example, a bidirectional, omnidirectional, or other dynamic microphone may also be used as the microphone 20.

The controller 21 according to this embodiment is used to input various information in response to user operations. As an example, as shown in FIG. 3, in this embodiment, a general-purpose controller for a game console is used as the controller 21, but this is not limited to this, and a controller dedicated to the sound auralization device 10 may also be used as the controller 21.

As shown in FIG. 3, the controller 21 according to this embodiment is equipped with a pair of gripping portions 21A and 21B that are held by the user's left and right hands, respectively. Grip portions 21A and 21B are connected by an intermediate portion 21C, and are provided with buttons 21A1 and 21B1, respectively, that can be operated with the thumb of the holding hand.

The controller 21 according to this embodiment is also provided with sticks 21C1 and 21C2 at the lower end of the intermediate portion 21C in FIG. 3, near the grip portions 21A and 21B, respectively, that can be operated with the thumb of the hand holding the corresponding grip portion. In the sound auralization device 10 according to this embodiment, stick 21C1 accepts instruction input for moving the sound receiving point (described below), and stick 21C2 accepts instruction input for the orientation of the sound receiving point.

Furthermore, the speaker 22 according to this embodiment is for reproducing the sound generated by the acoustic auralization device 10. In this embodiment, a dynamic, stereo speaker is used as the speaker 22, but this is not limited to this. For example, a mono speaker of a capacitor type, piezoelectric type, or the like may also be used as the speaker 22.

Next, with reference to Figure 2, the functional configuration of the acoustic auralization device 10 according to this embodiment will be described.

As shown in FIG. 2, the acoustic auralization device 10 according to this embodiment includes an acquisition unit 11A, a detection unit 11B, a derivation unit 11C, a playback control unit 11D, a generation unit 11E, a display control unit 11F, and an identification unit 11G. The CPU 11 of the acoustic auralization device 10 executes the setting program 13A and the acoustic auralization program 13B, thereby functioning as the acquisition unit 11A, the detection unit 11B, the derivation unit 11C, the playback control unit 11D, the generation unit 11E, the display control unit 11F, and the identification unit 11G.

The acquisition unit 11A in this embodiment acquires shape information indicating the shape of the target room (hereinafter also referred to as the "target room") and acoustic-related information relating to the acoustic characteristics of the interior of the room.

In this embodiment, information based on 3D CAD (Computer Aided Design) information, which will be described in detail later, is used as the shape information, but this is not limited to this. For example, information based on BIM (Building Information Modeling) information may also be used as the shape information. Furthermore, in this embodiment, information including the sound absorption coefficient, scattering coefficient, and transmittance of surfaces such as wall surfaces, ceiling surfaces, and floor surfaces in the target room (hereinafter referred to as "constituent surfaces") is used as the acoustic-related information, but this is not limited to this. For example, information including any one or a combination of two of the sound absorption coefficient, scattering coefficient, and transmittance may also be used as the acoustic-related information.

The sound absorption coefficient is an index that indicates how much sound the material absorbs; at its maximum value, all incident sound is absorbed, while at its minimum value (usually 0), the incident sound is reflected without being absorbed. The scattering coefficient is an index that indicates the proportion of sound incident on the material that is reflected in a direction other than specular reflection; at a high scattering coefficient, sound is reflected randomly in all directions, while at a low scattering coefficient, sound is specularly reflected like a mirror. The transmittance is an index that indicates how much sound passes through the material; at its maximum value, all incident sound is transmitted in the opposite direction to the incident direction, while at its minimum value (usually 0), the incident sound is completely blocked without being transmitted.

In addition, the detection unit 11B according to this embodiment detects sound source information indicating at least one of the sound (hereinafter also referred to as "sound source sound") emitted from a target sound source (hereinafter also referred to as "target sound source") and the sound at a sound receiving point (hereinafter also referred to as "sound receiving point sound"), which is the location of a listener who checks the acoustic design effects in the target room.

Note that, in this embodiment, a case where the detection unit 11B detects sound source information indicating both the sound source sound and the sound receiving point sound is described, but this is not limited to this. For example, the detection unit 11B may detect only the sound source information indicating the sound receiving point sound, or only the sound source information indicating the sound source sound. Furthermore, in this embodiment, a case where the detection unit 11B detects the sound source information indicating the sound source sound indirectly by detecting sound emitted from the target sound source using the microphone 20 and storing it in advance in the storage unit 13 as sound source information, and then reading out the sound source information from the storage unit 13 is described. In this case, the sound source information is acquired by reading it out from the storage unit 13. In other words, the sound source information is acquired by the acquisition unit 11A. However, this is not limited to this embodiment, and for example, the detection or acquisition of sound source information indicating the sound source sound may be performed directly via the microphone 20, without going through the storage unit 13.

In addition, the derivation unit 11C according to this embodiment uses the shape information and sound-related information acquired by the acquisition unit 11A and the sound source information detected by the detection unit 11B to derive acoustic auralization information that indicates how a sound emitted from at least one of the target sound source and the sound receiving point sounds at the sound receiving point.

In this embodiment, a case is described in which the derivation unit 11C derives acoustic auralization information corresponding to both the sound source sound and the sound receiving point sound, but this is not limited to this. For example, the derivation unit 11C may derive acoustic auralization information corresponding only to the sound receiving point sound, or the derivation unit 11C may derive acoustic auralization information corresponding only to the sound source sound.

In addition, in this embodiment, the derivation unit 11C derives the acoustic auralization information using a transfer function.

The playback control unit 11D in this embodiment then controls the playback unit (in this embodiment, speaker 22) to play the sound indicated by the acoustic auralization information derived by the derivation unit 11C.

On the other hand, the generation unit 11E according to this embodiment uses the shape information described above to generate video information showing an image of the target room that can be seen by the listener as the listener moves. The display control unit 11F according to this embodiment then controls the display unit 15 to virtually display the image shown by the video information generated by the generation unit 11E.

The identification unit 11G according to this embodiment identifies the position of the sound receiving point as the sound receiving point moves. The derivation unit 11C according to this embodiment then derives the acoustic auralization information using the position of the sound receiving point identified by the identification unit 11G.

In this embodiment, the derivation of the acoustic auralization information by the derivation unit 11C and the generation of the video information by the generation unit 11E are performed by a single computer (in this embodiment, CPU 11). This makes it easier to synchronize the playback of the sound indicated by the acoustic auralization information with the display of the video indicated by the video information. However, this is not limited to this, and for example, the processing by the derivation unit 11C and the processing by the generation unit 11E may be performed by different computers. In this embodiment, the processing load by the derivation unit 11C and the processing load by the generation unit 11E can be distributed, allowing for more accurate and faster playback of sound and display of video.

Next, the building-related information database 13C according to this embodiment will be described with reference to Figure 4. Figure 4 is a schematic diagram showing an example of the configuration of the building-related information database 13C according to this embodiment. The building-related information database 13C is a database that stores information about buildings that are the subject of handling by the acoustic auralization device 10 according to this embodiment.

As shown in FIG. 4, the building-related information database 13C according to this embodiment stores, in association with each other, the building name, 3D CAD information, and sound source-related information for each building handled by the sound auralization device 10.

The building name is information indicating the name of the corresponding building, and the 3D CAD information is information indicating a model (hereinafter referred to as a "building-related model") that includes building shape information indicating the shape of the corresponding building and specific information for identifying each room in the building.

In this embodiment, the building-related model is created using predetermined 3D CAD software. In this embodiment, Rhinoceros (registered trademark) is used as the 3D CAD software, but this is not limited to this. For example, other software such as Revit (registered trademark) may also be used as the 3D CAD software.

The sound source-related information is information related to the target sound source described above, and includes room information, sound source information, position information, and directivity information. The room information is information for identifying the corresponding room (target room) in the corresponding building, and in this embodiment, the above-mentioned identification information is applied. As described above, the sound source information is information indicating the sound itself emitted from the corresponding target sound source and picked up by microphone 20. The position information is information indicating the three-dimensional position of the corresponding target sound source, and the directivity information is information indicating the directivity of the corresponding target sound source.

For example, if the target room is a school classroom, a specific example of the target sound source would be the teacher's voice. In this case, the target sound source position would be the front of the blackboard or a position on the podium where the teacher is likely to stand, the directionality would be that of a human voice, and the sound source information would be time-series information indicating the voice emitted by the teacher when teaching.

Next, the transfer function-related information database 13D according to this embodiment will be described with reference to FIG. 5. FIG. 5 is a schematic diagram showing an example of the configuration of the transfer function-related information database 13D according to this embodiment. Note that the transfer function-related information database 13D is a database that stores information related to the calculation of transfer functions, which are generated by the acoustic auralization device 10 according to this embodiment and will be described in detail below.

As shown in FIG. 5, the transfer function-related information database 13D according to this embodiment stores, in association with each piece of information, the building name and the transfer function-related information for each building handled by the acoustic auralization device 10.

The building name is the same information as the building name in the building-related information database 13C, and the transfer function-related information is information related to the transfer functions described below that correspond to each room in the corresponding building, and includes room information and transfer function information. The room information is the same information as the room information in the building-related information database 13C, and the transfer function information is information related to the calculation of the transfer function between the position of the target sound source and the position of the sound receiving point, which is set for the corresponding room by the setting process described below.

Here, settings related to the calculation of the transfer function applied in this embodiment will be described with reference to Figures 6 to 8. Figure 6 is a perspective view showing an example of a setting target related to the calculation of the transfer function in this embodiment. Figure 7 is a side view showing an example of a setting target related to the calculation of the transfer function when wave properties are not taken into account in this embodiment. Figure 8 is a side view showing an example of a setting target related to the calculation of the transfer function when wave properties are taken into account in this embodiment.

As an example, as shown in Figure 6, in this embodiment, a transfer function is calculated based on the ray tracing method, which does not take into account the wave nature of sound. With the ray tracing method, results vary depending on the number of sound rays generated from a single target sound source, the maximum number of reflections, etc., so appropriate values are set taking into account the computational load on the CPU 11, etc.

As an example, as shown in Figure 7, when wave properties are not taken into account, sound flows in a straight line, so if there is a soundproof wall or the like between the target sound source and the sound receiving point, the sound will not reach the sound receiving point. On the other hand, as an example, as shown in Figure 8, when wave properties are taken into account, sound from the target sound source will go around the soundproof wall or the like and reach the sound receiving point as diffracted sound. In this embodiment, taking into account the processing power of current computers, the sound ray method that does not take wave properties into account is used as the calculation means, but it goes without saying that this is not limited to this and a calculation method that takes wave properties into account may also be used.

Next, the operation of the sound auralizer 10 according to this embodiment will be described with reference to Figures 9 to 15. First, the operation of the sound auralizer 10 when executing a setting process for setting information related to the calculation of a transfer function will be described with reference to Figures 9 to 11. When a user inputs an instruction to start the execution of the setting process via the input unit 14, the CPU 11 of the sound auralizer 10 executes the setting program 13A, thereby executing the setting process shown in Figure 9. Figure 9 is a flowchart showing an example of the setting process according to this embodiment. Note that, to avoid confusion, the case where a building-related information database 13C has already been constructed will be described here.

In step 100 of FIG. 9, the CPU 11 controls the display unit 15 to display a setting target input screen with a predetermined configuration, and in step 102, the CPU 11 waits until the specified information is input.

Figure 10 shows an example of a setting target input screen according to this embodiment. As shown in Figure 10, the setting target input screen according to this embodiment displays a message prompting input of the room (hereinafter referred to as the "setting target room") for which information regarding the calculation of the transfer function is to be set. The setting target input screen according to this embodiment also displays an input area 15A for inputting the building in which the setting target room is located and various pieces of information about the setting target room.

When the setting target input screen shown in FIG. 10 is displayed on the display unit 15 as an example, the user inputs the corresponding information into the corresponding input field 15A via the input unit 14 and then presses the end button 15D. In response, step 102 is judged as positive, and the process proceeds to step 104.

In step 104, the CPU 11 reads from the building-related information database 13C the 3D CAD information corresponding to the building entered on the setting target input screen and the sound source-related information corresponding to the setting target room entered on the setting target input screen (hereinafter, this information will be collectively referred to as "building-related information").

In step 106, the CPU 11 controls the display unit 15 to display a setting information input screen with a predetermined configuration, and in step 108, the CPU 11 waits until the specified information is input.

Figure 11 shows an example of a setting information input screen according to this embodiment. As shown in Figure 11, the setting information input screen according to this embodiment displays a message prompting the user to enter information about the acoustic characteristics of the room to be set. The setting information input screen according to this embodiment also displays an input area 15B for entering values for acoustic-related information such as the sound absorption coefficient, scattering coefficient, and transmittance of the constituent surfaces in the room to be set.

As shown in FIG. 11, in this embodiment, a straight line of a predetermined length extending in the left-right direction is displayed as input area 15B for each corresponding piece of sound-related information. In this embodiment, the value of the corresponding sound-related information is input by specifying the position of a marker using input unit 14, with one end of the line (in this embodiment, the left end) as the minimum value and the other end as the maximum value, but this is not limited to this. For example, a rectangular input area for directly inputting the corresponding value, similar to input area 15A shown in FIG. 10, may be used as the input area for inputting the value of the sound-related information.

Furthermore, in this embodiment, to avoid confusion, all constituent surfaces in the target room are made of the same material, and values of acoustic-related information corresponding only to that material are input, but this is not limited to this. For example, each constituent surface in the target room, such as the wall, ceiling, and floor, may be made of a different material, and values of acoustic-related information corresponding to each type of material may be input.

As an example, when the setting information input screen shown in FIG. 11 is displayed on the display unit 15, the user specifies the position of the marker via the input unit 14 to be the position corresponding to the value of the corresponding sound-related information, and then presses the end button 15D. In response to this, step 108 is judged as positive, and the process proceeds to step 110.

In step 110, the CPU 11 uses the acoustic-related information entered on the setting information input screen to perform settings related to the calculation of the transfer function between the position of the target sound source in the setting target room indicated by the position information in the read building-related information and the position of the sound receiving point (i.e., the position of the listener) by applying the sound ray method described above.

In step 112, the CPU 11 stores (registers) information indicating the setting results for the calculation of the transfer function for the target room obtained through the above processing as transfer function information in the transfer function related information database 13D, together with the building name indicating the target building and the room information indicating the target room. Then, once this storage is complete, the CPU 11 terminates this setting process.

Next, the operation of the sound auralization device 10 when executing sound auralization processing will be described with reference to Figures 12 to 15. When a user inputs an instruction to start execution of sound auralization processing via the input unit 14, the CPU 11 of the sound auralization device 10 executes the sound auralization program 13B, thereby executing the sound auralization processing shown in Figure 12. Figure 12 is a flowchart showing an example of sound auralization processing according to this embodiment. Note that, in order to avoid confusion, the case will be described where information about the target room to be subjected to sound auralization processing has already been registered in the transfer function related information database 13D.

In step 200 of FIG. 12, the CPU 11 controls the display unit 15 to display a processing target input screen with a predetermined configuration, and in step 202, the CPU 11 waits until the specified information is input.

Figure 13 shows an example of a processing target input screen according to this embodiment. As shown in Figure 13, the processing target input screen according to this embodiment displays a message prompting the user to input the room that is the target of sound auralization processing (hereinafter referred to as the "sound auralization target room"). The processing target input screen according to this embodiment also displays an input area 15C for inputting the building in which the sound auralization target room is located and various pieces of information about the sound auralization target room.

As an example, when the processing target input screen shown in FIG. 13 is displayed on the display unit 15, the user inputs the corresponding information into the corresponding input field 15C via the input unit 14 and then presses the end button 15D. In response, step 202 is judged as positive, and the process proceeds to step 204.

In step 204, the CPU 11 reads building-related information corresponding to the building entered on the processing target input screen from the building-related information database 13C. Also in step 204, the CPU 11 reads transfer function information corresponding to the acoustic auralization target room entered on the processing target input screen from the transfer function-related information database 13D.

In step 206, the CPU 11 uses the read building-related information and transfer function information to calculate how direct sound and reflected sound arrive from the target sound source to the sound receiving point using the sound ray method described above, and calculates the transfer function by performing a Fourier transform on the direct sound and reflected sound group.

In step 208, the CPU 11 uses the results of the transfer function calculation to begin generating acoustic auralization information that indicates how sound will be heard when looking in a predetermined direction from the sound receiving point in the acoustic auralization target room. Also in step 208, the CPU 11 uses information about the acoustic auralization target room in the read 3D CAD information to begin generating video information that indicates a 3D video of the acoustic auralization target room when looking in the predetermined direction from the sound receiving point.

In this embodiment, the initial position of the sound receiving point is set to a predetermined height (1.6 m in this embodiment) at the entrance/exit of the room to be auralized, but this is not limited to this. For example, the predetermined height may be a height corresponding to the user's actual height, or the center (center of gravity) of the room to be auralized may be used as the initial position of the sound receiving point.

In addition, in this embodiment, the above-mentioned initially predetermined direction is the direction from the entrance of the sound auralization target room facing the center of the room, but this is not limited to this. For example, a direction preset by the user or the like depending on the purpose of the sound auralization device 10 may be applied as the above-mentioned initially predetermined direction.

In step 210, the CPU 11 starts playing sound from the speaker 22 using the generated acoustic auralization information, and starts displaying the video screen represented by the generated video information on the display unit 15. From this point on, the CPU 11 synchronizes the playback of the sound with the display of the video screen.

Figure 14 shows an example of a video screen according to this embodiment. As shown in Figure 14, the video screen according to this embodiment displays a message that sound and video are being played, as well as a three-dimensional image of the sound auralization target room when viewed from the sound receiving point in the predetermined direction. The video screen according to this embodiment also displays a method for moving the position of the sound receiving point and changing the facing direction (hereinafter simply referred to as "orientation") using the controller 21. Therefore, by referring to the video screen, the user can virtually view the sound auralization target room from the line of sight that they would have if they were located at the sound receiving point, and can virtually hear from the speaker 22 the sound that they would hear if they were facing the line of sight.

When the video screen is displayed, if the user wishes to change at least one of the position and orientation of the sound receiving point, they use the controller 21 to change the target in the manner displayed on the video screen. At this time, if the user wishes to have their own voice audible, they speak their voice into the microphone 20. Then, if the user wishes to end the display of the video screen and the playback of sound, they use the input unit 14 to select the end button 15D.

Therefore, in step 212, the CPU 11 determines whether the user has performed an operation to change at least one of the position and orientation of the sound receiving point, and if the determination is negative, the process proceeds to step 218, whereas if the determination is positive, the process proceeds to step 214.

In step 214, the CPU 11 identifies the position and orientation of the sound receiving point in response to the user's operation on the controller 21. In step 216, the CPU 11 uses the identified position and orientation of the sound receiving point to recalculate the transfer function between the target sound source and the sound receiving point, and executes sound/image update processing to update the sound being played by the speaker 22 and the image being displayed on the display unit 15.

In the sound/video update process according to this embodiment, when acoustic information indicating a user's voice is input via the microphone 20, the sound indicated by the acoustic information is assumed to be emanating from the sound-receiving point, and is reflected in the sound auralization information. That is, in this case, the sound/video update process convolves the sound collected by the microphone 20 with a transfer function obtained by performing a calculation assuming that the sound-receiving point and the sound source are in the same position. After executing the sound/video update process, the CPU 11 then proceeds to step 218. This sound/video update process updates the sound being played by the speaker 22 and the video screen being displayed on the display unit 15 in real time to a state corresponding to the user's operation of the controller 21.

In step 218, the CPU 11 determines whether the timing to end this acoustic auralization process has arrived by determining whether the user has pressed the end button 15D, and if the determination is negative, the process returns to step 212. If the determination is positive, the process proceeds to step 220.

Figure 15 shows an example of a video screen when the processing of steps 212 to 218 is repeatedly executed. Note that the example shown in Figure 15 shows an example in which the direction of the sound receiving point has moved to the left compared to the video screen shown in Figure 14.

In step 220, the CPU 11 stops the sound playback and video display that began in step 210, and in step 222, the CPU 11 stops the generation of the acoustic auralization information and video information that began in step 208, and then ends this acoustic auralization process.

As described above, this embodiment includes an acquisition unit 11A that acquires shape information indicating the shape of a target room, acoustic-related information related to the acoustic characteristics of the room's interior, and sound source information indicating the sound emitted from the target sound source; an identification unit 11G that identifies the position of a sound receiving point, which is the location of a listener who checks the acoustic design effects of the room, as the point moves; a derivation unit 11C that derives acoustic auralization information indicating how the sound emitted from the sound source will sound at the sound receiving point using the shape information, acoustic-related information, and sound source information acquired by the acquisition unit 11A and the position of the sound receiving point identified by the identification unit 11G; and a playback control unit 11D that controls the playback unit to play the sound indicated by the acoustic auralization information derived by the derivation unit 11C. Therefore, it is possible to reproduce the sound heard by a listener when the listener virtually moves.

Furthermore, according to this embodiment, acoustic auralization information is derived using a transfer function. Therefore, it is possible to reproduce the sound emitted from the sound source more easily than when a transfer function is not used.

Furthermore, according to this embodiment, the shape information is used to generate video information showing an image of the room as seen by the listener as the listener moves, and the image shown by the generated video information is virtually displayed. Therefore, the sound emitted from the sound source can be reproduced with a more realistic feel than when the image is not displayed.

Furthermore, according to this embodiment, the derivation of acoustic auralization information and the generation of video information are performed by a single computer. Therefore, compared to when the derivation of acoustic auralization information and the generation of video information are performed by different computers, it is easier to synchronize the playback of sound indicated by the acoustic auralization information with the display of video indicated by the video information.

In the above embodiment, we have described a case where there is only one sound source for each target room, but this is not limited to this. For example, there may be multiple sound sources in each target room. In this case, calculations related to the transfer function are performed for each combination of sound source and sound receiving point.

In addition, in the above embodiment, a case has been described in which the sound source does not move and only the sound receiving point moves, but this is not limiting. For example, only the sound source, or both the sound source and the sound receiving point, may move. In this case, the frequency of the sound emitted from the sound source is corrected using the Doppler effect based on the speed of the sound source and whether the sound source is moving toward or away from the sound receiving point, and then a transfer function is convolved with the sound. By correcting the frequency of the sound emitted by the sound source in this way, the accuracy of the reproduction of the sound can be improved. Furthermore, by convolving this transfer function, it is possible to reproduce a sound with reverberation in the reproduced sound field.

An example of a moving sound source is highway traffic noise when the target room is a room in a building adjacent to a highway. In this case, the position of the sound source is assumed to be the path of cars on the highway, the directivity is assumed to be a point sound source (omnidirectional) or the directivity when the car is moving, and the sound source information is assumed to be time-series information showing the sound of the car moving.

Furthermore, in the above embodiment, a case has been described in which the head-related transfer function at the sound receiving point is not taken into consideration, but this is not limiting. For example, a head-related transfer function according to the direction of the sound receiving point may be further convoluted.

In this case, head-related transfer function data for each direction, such as when the person is facing forward or to the side, is prepared in advance, and the head-related transfer function data to be convolved is selectively switched depending on the direction of the sound-receiving point. By applying this head-related transfer function, the direction of the sound source can be accurately reproduced.

In addition, in the above embodiment, a case where the controller 21 is used to change the position and orientation of the sound receiving point is described, but this is not limited to this. For example, the position and orientation of the sound receiving point may be changed using the input unit 14 according to this embodiment. Also, for example, the sound auralization device 10 may be provided with a voice recognition function, and the user may specify the position and orientation of the sound receiving point by voice.

Furthermore, in the above embodiment, a case has been described in which the image obtained by the acoustic auralization process is displayed on the display unit 15 provided in the acoustic auralization device 10, and the sound obtained by the acoustic auralization process is played back by the speaker 22 connected to the acoustic auralization device 10, but this is not limited to this. For example, the image and sound obtained by the acoustic auralization device 10 may be displayed and played back by a head-mounted display equipped with headphones. Although not mentioned in the above embodiment, when playing back sound by the speaker 22, it is preferable to prevent feedback caused by the closed loop between the microphone 20 and the speaker 22, for example by using a feedback canceller.

It goes without saying that the configurations of the various databases applied in the above embodiments are merely examples and are not limited to the examples shown.

In addition, in the above embodiment, for example, the hardware structure of the processing units that execute the processes of the acquisition unit 11A, detection unit 11B, derivation unit 11C, playback control unit 11D, generation unit 11E, display control unit 11F, and identification unit 11G can be the various processors listed below. As mentioned above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as a processing unit, as well as dedicated electrical circuits, such as programmable logic devices (PLDs) that are processors whose circuit configuration can be changed after manufacture, such as field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs), which are processors with a circuit configuration designed specifically to execute specific processes.

The processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The processing unit may also be configured with a single processor.

Examples of a processing unit being configured with a single processor include, first, a form in which a single processor is configured with a combination of one or more CPUs and software, as typified by client and server computers, and this processor functions as the processing unit. Second, a form in which a processor is used to realize the functions of the entire system, including the processing unit, on a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, the processing unit is configured as a hardware structure using one or more of the various processors listed above.

Furthermore, the hardware structure of these various processors can be, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

10 Acoustic sonification device 11 CPU
11A Acquisition unit 11B Detection unit 11C Derivation unit 11D Playback control unit 11E Generation unit 11F Display control unit 11G Identification unit 12 Memory 13 Storage unit 13A Setting program 13B Acoustic auralization program 13C Building-related information database 13D Transfer function-related information database 14 Input unit 15 Display unit 15A Input area 15B Input area 15C Input area 15D End button 16 Medium reading and writing device 17 Recording medium 18 Communication I/F unit 20 Microphone 21 Controller 22 Speaker

Claims (5)

an acquisition unit that acquires shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating sounds emitted from a target sound source;
an identification unit that identifies the position of a sound receiving point, which is a position where a listener who checks the acoustic design effect in the room is located, as the sound receiving point moves;
a derivation unit that derives acoustic aurization information indicating how the sound emitted from the sound source is heard at the sound receiving point, using the shape information, the acoustic-related information, and the sound source information acquired by the acquisition unit and the position of the sound receiving point identified by the identification unit; and
a playback control unit that controls a playback unit to play back the sound indicated by the acoustic auralization information derived by the derivation unit;
Equipped with
the derivation unit derives the acoustic auralization information by using a transfer function and assuming that the sound receiving point and the sound source are at the same position ;
The initial height of the sound receiving point is set to the height of the listener's ears .
Acoustic sonification device. a generating unit that generates video information showing a video of the room that can be seen by the listener according to the listener's movement using the shape information;
a display control unit that controls a display unit to virtually display the image indicated by the image information generated by the generation unit;
The acoustic sonification device of claim 1 further comprising: The derivation of the acoustic auralization information by the derivation unit and the generation of the video information by the generation unit are performed by a single computer.
3. The acoustic sonification device of claim 2. the derivation unit derives the acoustic auralization information as information that reflects the influence of the Doppler effect caused by movement of the sound source.
The acoustic sonification device according to any one of claims 1 to 3. Acquire shape information indicating the shape of a target room, acoustic-related information relating to the acoustic characteristics of the interior of the room, and sound source information indicating the sound emitted from the target sound source;
Identifying the position of a sound receiving point, which is a position where a listener who will confirm the acoustic design effect in the room is located, as the sound receiving point moves;
deriving acoustic auralization information indicating how the sound emitted from the sound source is heard at the sound receiving point using the acquired shape information, the acoustic-related information, and the sound source information, and the identified position of the sound receiving point;
a process of controlling a playback unit to play back a sound indicated by the derived acoustic auralization information;
deriving the acoustic auralization information using a transfer function and assuming that the sound receiving point and the sound source are at the same position ;
The initial height of the sound receiving point is set to the height of the listener's ears .
An acoustic sonification program that causes a computer to carry out the process.