JP6825818B2 - Viewing system that provides virtual space - Google Patents

Description

The present invention relates to a technique for providing a virtual space to a viewer.

There is a technology that provides a virtual space that gives the viewer a high sense of presence. For example, Patent Document 1 describes an image display unit designed by a user to display an image seen from a specific viewpoint in a closed space, and sound reproduction for reproducing a sound emitted from a sound source arranged in the closed space and transmitted to the viewpoint. A virtual environment generator including a unit is disclosed. According to the virtual environment generator described in Patent Document 1, when the user changes the shape and characteristics of the closed space, the sound reflecting the sound absorption coefficient, surface area, etc. of the wall surface constituting the closed space is reproduced from the sound reproduction unit. To.

Japanese Unexamined Patent Publication No. 6-161477

The user of the virtual environment generator described in Patent Document 1 described above needs to design a closed space. When designing a closed space, it is necessary to input data such as the shape and characteristics of the walls that make up the closed space. If the closed space has a complicated shape such as a concert hall, the data should be input. It takes a lot of effort.

On the other hand, the present invention provides the viewer with a virtual space that brings a high sense of presence without requiring a great deal of effort to input data.

In order to achieve the above object, the present invention includes member arrangement data indicating the arrangement of each of a plurality of members constituting a building arranged in the three-dimensional space in the three-dimensional space, and arrangement in the three-dimensional space. Sound source position data indicating the position of each of one or more sound sources, viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and member acoustic characteristics indicating the acoustic characteristics of each of the plurality of members. Based on the data, a sound propagation path from each of the one or more sound sources to the viewing point, which branches in a plurality of directions in the member, is provided in the member in the direction of the sound emitted from the sound source. A propagation path whose sound attenuation rate is less than a predetermined threshold is selected from the directions of a plurality of branching propagation paths, and a plurality of propagation paths branching in a member in the direction of each of the selected sound propagation paths. It is specified by repeating the process of selecting a propagation path in which the sound attenuation rate is less than a predetermined threshold from the directions of, and one or more members located on the specified propagation path are selected from the plurality of members. Selection means to be selected, the member arrangement data relating to one or more members selected by the selection means, the sound source position data, and the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources. Based on the member acoustic characteristic data regarding one or more members selected by the selection means and the viewing point position data, the reaching sound data representing the reaching sound, which is the sound heard by the viewer at the viewing point, is generated. Based on the sound data generation means, the viewing point position data, the line-of-sight direction data indicating the viewing direction of the viewer at the viewing point, and the member arrangement data, image data representing the image visible to the viewer is generated. At the same time as instructing the sounding device to sound the sound represented by the image data generation means and the arrival sound data generated by the sound data generation means, the display of the image represented by the image data generated by the image data generation means is displayed. A data processing device including an instruction means for instructing the device is proposed as the first aspect.
Further, in the first aspect of the present invention, the member arrangement data is BIM data created in the design of the building, and the member acoustic characteristic data is the member acoustic characteristic data for each of the plurality of members. The data is stored in association with the data indicating the physical properties of the material of the member created as the BIM data, and the image data generation means is performed by the viewer by the same processing as the processing performed by the BIM. As a second aspect, we propose a configuration in which image data representing a visible figure is generated .

Further, in the first or second aspect of the present invention, the sound data generation means is selected from a storage device that stores material acoustic characteristic data indicating the acoustic characteristics of the material with respect to each of the plurality of materials. As a third aspect, we propose a configuration in which material acoustic characteristic data corresponding to the material of the member is acquired as the member acoustic characteristic data regarding the member for each of the one or more members selected by the means.

Further, in the present invention, in any one of the first to third aspects described above, the selection means is from a storage device that stores material acoustic characteristic data indicating the acoustic characteristics of the material with respect to each of the plurality of materials. As a fourth aspect, a configuration is proposed in which material acoustic characteristic data corresponding to the material of the member is acquired as member acoustic characteristic data for each of the members, and one or more members are selected based on the member acoustic characteristic data. To do.

Further, in the first to fourth aspects of the present invention, at least one of the positions of the one or more sound sources indicated by the sound source position data is in the building, and the viewing point position. A configuration in which the position of the viewing point indicated by the data is outside the building is proposed as the fifth aspect.

Further, in the present invention, the computer is provided with member arrangement data indicating the arrangement of each of a plurality of members constituting a building arranged in the three-dimensional space in the three-dimensional space, and one or more members arranged in the three-dimensional space. Sound source position data indicating each position of the sound source, viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and member acoustic characteristic data indicating the acoustic characteristics of each of the plurality of members. Based on this, a plurality of sound propagation paths from each of the one or more sound sources to the viewing point, which are branched in a plurality of directions in the member, are branched in the member in the direction of the sound emitted from the sound source. A propagation path in which the sound attenuation rate is less than a predetermined threshold is selected from the directions of the propagation path of the sound, and the directions of the plurality of propagation paths branching in the member in the direction for each of the directions of the selected sound propagation path. A selection means for specifying by repeating a process of selecting a propagation path in which the sound attenuation rate is less than a predetermined threshold, and selecting one or more members located on the specified propagation path from the plurality of members. And the member arrangement data regarding one or more members selected by the selection means, the sound source position data, the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources, and the selection. Sound data for generating arrival sound data representing the arrival sound, which is the sound heard by the viewer at the viewing point, based on the member acoustic characteristic data for one or more members selected by the means and the viewing point position data. Image data that generates image data representing a figure visible to the viewer based on the generation means, the viewing point position data, the line-of-sight direction data indicating the viewing direction of the viewer at the viewing point, and the member arrangement data. At the same time as instructing the sounding device to sound the sound represented by the generation means and the arrival sound data generated by the sound data generating means, the display device is instructed to display the image represented by the image data generated by the image data generating means. A program for functioning as an instruction means is proposed as a sixth aspect.

Further, in the present invention, the computer is provided with member arrangement data indicating the arrangement of each of a plurality of members constituting a building arranged in the three-dimensional space in the three-dimensional space, and one or more members arranged in the three-dimensional space. Sound source position data indicating each position of the sound source, viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and member acoustic characteristic data indicating the acoustic characteristics of each of the plurality of members. Based on this, a plurality of sound propagation paths from each of the one or more sound sources to the viewing point, which are branched in a plurality of directions in the member, are branched in the member in the direction of the sound emitted from the sound source. A propagation path in which the sound attenuation rate is less than a predetermined threshold is selected from the directions of the propagation path of the sound, and the directions of the plurality of propagation paths branching in the member in the direction for each of the directions of the selected sound propagation path. A selection means for specifying by repeating a process of selecting a propagation path in which the sound attenuation rate is less than a predetermined threshold, and selecting one or more members located on the specified propagation path from the plurality of members. , The member arrangement data regarding one or more members selected by the selection means, the sound source position data, the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources, and the selection. Sound data that generates arrival sound data representing the arrival sound, which is the sound heard by the viewer at the viewing point, based on the member acoustic characteristic data for one or more members selected by the means and the viewing point position data. Image data that generates image data representing a figure visible to the viewer based on the generation means, the viewing point position data, the line-of-sight direction data indicating the viewing direction of the viewer at the viewing point, and the member arrangement data. At the same time as instructing the sounding device to sound the sound represented by the generation means and the arrival sound data generated by the sound data generating means, the display device is instructed to display the image represented by the image data generated by the image data generating means. A computer-readable recording medium for continuously recording a program for functioning as an instruction means is proposed as a seventh aspect.

According to the first aspect of the present invention, only the propagation path in which the attenuation rate reaches a predetermined threshold value is considered among the transmission paths in which the number of branches increases each time the member is reached. Therefore, useless processing is not performed to generate the arrival sound.
Further, according to the second aspect of the present invention, a virtual space that brings a high sense of presence to the viewer is realized by using the BIM (Building Information Modeling) data created in the design of the building. Therefore, if the material acoustic property data is stored in the material table, the user does not need to input any new data.

According to the third aspect of the present invention, the acoustic characteristics of the material are used as the acoustic characteristics of each of the members constituting the building. Therefore, it is not necessary to input the acoustic characteristics for each member.

According to the fourth aspect of the present invention, the member used for generating the reaching sound is appropriately selected according to the high and low of the sound transmittance, the high and low of the sound reflectance, and the like. Therefore, the viewer is provided with a sound that brings a high sense of presence without causing a large increase in the processing load to generate the reaching sound.

According to a fifth aspect of the present invention, the viewer is provided with sounds and images that provide a high sense of presence when outside the building.

According to the sixth or seventh aspect of the present invention, the data processing device according to the first aspect is realized by a computer.

The figure which showed the whole structure of the virtual space viewing system which concerns on one Embodiment. The figure which showed the structure of the computer which concerns on one Embodiment. The figure which showed the structure of the data processing apparatus which concerns on one Embodiment. The figure which illustrated the data structure of the member table which concerns on one Embodiment. The figure which illustrated the shape of the building represented by the member arrangement data which concerns on one Embodiment. The figure which illustrated the data structure of the material table which concerns on one Embodiment. The figure which showed the flow of the processing performed by the data processing apparatus which concerns on one Embodiment. The figure which showed the input screen which the display device which concerns on one Embodiment displays. The figure which showed a part of the sound propagation path specified by the selection means which concerns on one Embodiment. The figure which showed how the virtual space viewing system which concerns on one Embodiment provides a virtual space to a viewer. The figure which showed the input screen which the display device displays in one modification.

[Embodiment]
The virtual space viewing system 1 according to the embodiment of the present invention will be described below. When the viewer specifies an arbitrary viewing point and an arbitrary line-of-sight direction in a virtual three-dimensional space (hereinafter referred to as virtual space S) displayed on the display device, the virtual space viewing system 1 at the designated viewing point. It is a system that displays on the display device the image that can be seen when facing the specified line-of-sight direction, and at the same time emits the sound that can be heard when facing the specified line-of-sight direction at the specified viewing point from the sounding device.

FIG. 1 is a diagram showing the overall configuration of the virtual space viewing system 1. The virtual space viewing system 1 includes a computer 10, an operating device 12, a display device 13, and a sounding device 14. The operation device 12 is a device that receives the operation of the viewer U, and is, for example, a mouse, a keyboard, or the like. The display device 13 is a device that displays an image that can be seen in the virtual space S to the viewer U, and is, for example, a display device. The sounding device 14 is a device that sounds the sound heard in the virtual space S to the viewer U, and is, for example, a speaker, headphones, or the like.

The computer 10 is a general computer. FIG. 2 is a diagram showing the configuration of the computer 10. The computer 10 includes a memory 101 that stores various data, and a processor 102 that performs various data processing according to a program stored in the memory 101.

Further, the computer 10 has an operation input I / F 103, which is an interface for receiving input of operation data indicating the content of the operation performed by the viewer U on the operation device 12, and an interface for outputting image data to the display device 13. It includes a certain image output I / F 104 and a sound output I / F 105 which is an interface for outputting sound wave data to the sounding device 14.

A program according to the present embodiment is stored in the memory 101 of the computer 10, and when the processor 102 performs data processing according to the program, the computer 10 provides a virtual space S to the viewer U. 11 is realized. FIG. 3 is a diagram showing the configuration of the data processing device 11 thus realized by the computer 10.

The data processing device 11 includes an operation data acquisition unit 111 that acquires operation data from the operation device 12, and a storage device 112 that stores various data. Further, the data processing device 11 is used for generating arrival sound data representing a sound that can be heard by a viewer (hereinafter, referred to as arrival sound) from a plurality of members constituting a building arranged in the virtual space S. A selection means 113 for selecting a member and a sound data generation means 114 for generating arrival sound data based on the acoustic characteristics of the member selected by the selection means 113 are provided.

Further, the data processing device 11 includes an image data generation means 115 for generating image data representing a figure visible to the viewer based on the arrangement of a plurality of members constituting the building arranged in the virtual space S, and sound data generation. At the same time as instructing the sounding device 14 to pronounce the arrival sound represented by the arrival sound data generated by the means 114, the instruction means 116 instructing the display device 13 to display the image represented by the image data generated by the image data generation means 115. Be prepared.

The operation data acquisition means 111 is realized by the operation input I / F 103. The storage device 112 is realized by the memory 101. The selection means 113, the sound data generation means 114, the image data generation means 115, and the instruction means 116 are realized by the processor 102.

The storage device 112 stores a member table, a material table, generated sound data, and directivity data. The member table is a data table prepared according to each of one or more buildings arranged in the virtual space S. FIG. 4 is a diagram illustrating the data structure of the member table. A member table is a collection of data records corresponding to each of a plurality of members constituting a building. As data fields, the member table includes a [member ID] that stores a member ID (Identifier) that identifies a member, a [member name] that stores data that indicates a member name, and member arrangement data that indicates the arrangement of members in the virtual space S. [Arrangement] for storing the material, and [Material ID] for storing the material ID for identifying the material of the member.

In the present application, the arrangement of members indicates the shape and position of members in the virtual space S. The member arrangement data stored in the field [arrangement] of the member table is, for example, polygon data in which the position in the virtual space S is specified.

FIG. 5 is a diagram illustrating the shape of the building X represented by a plurality of member arrangement data stored in the member table. In the building X illustrated in FIG. 5, an outer wall having a rectangular shape when viewed from above is arranged on the floor, a ceiling is arranged on the outer wall, and an inner wall that divides the space surrounded by them into two spaces. Has a structure in which is arranged. The building X is composed of a floor member M1, an outer wall member M2 to M5, a ceiling member M6, and an inner wall member M7.

FIG. 6 is a diagram illustrating the data structure of the material table stored in the storage device 112. A material table is a collection of data records for each of a plurality of materials that may be used in any of the components that make up a building. The material table includes [material ID] for storing the material ID for identifying the material and [material name] for storing the data indicating the name of the material as data fields.

In addition, the material table includes [mass], [load capacity], and [thermal conductivity] as data fields for storing data indicating various physical properties (excluding acoustic characteristics) of the material. In addition, the material table includes [sound reflection characteristics], [sound transmission characteristics], and [sound transmission characteristics] as data fields for storing data indicating various acoustic characteristics of the material. Hereinafter, the data indicating the acoustic properties of the material stored in these data fields are collectively referred to as the material acoustic property data.

The data stored in the data field [sound reflection characteristics] indicates, for example, the ratio (reflectance) of the amplitude of the reflected sound to the amplitude of the emitted sound when the sound is radiated to the material for each of the plurality of frequency bands. .. The sound attenuation rate in reflection is (1-reflectance). The data stored in the data field [sound transmission characteristics] is, for example, the ratio (transmittance) of the amplitude of the transmitted sound to the amplitude of the radiated sound when the sound is radiated to a material having a predetermined thickness at a plurality of frequencies. Shown for each of the bands. The sound attenuation rate in transmission is (1-transmission rate). The data stored in the data field [sound transmission characteristics] is, for example, the ratio (transmission rate) of the amplitude of vibration in which the material is transmitted by a predetermined distance to the amplitude of the vibrating body when the vibrating body is brought into contact with the material. Is shown for each of the plurality of frequency bands. The attenuation rate in transmission is (1-transmission rate).

The generated sound data stored in the storage device 112 is data representing a sound (hereinafter, referred to as a generated sound) emitted from a sound source arranged in the virtual space S. The generated sound data is, for example, sound wave data, and the storage device 112 stores a plurality of generated sound data each showing a different sound wave shape.

The directivity data stored in the storage device 112 is data indicating the directivity of the sound emitted from the sound source. For example, if the sound source is a speaker, the amplitude of the sound toward the sounding direction (usually in front of the speaker) is the largest, and the amplitude becomes smaller as the direction of the sound deviates from the sounding direction. Moreover, the change in their amplitude differs for each frequency band. The directivity data is data showing the attenuation rate according to the direction of the sound going out from the sound source for each frequency band. A plurality of directional data corresponding to different types of sound sources are stored in the storage device 112.

FIG. 7 is a diagram showing a flow of processing performed by the data processing device 11 to provide the virtual space S to the viewer U. The image data generation means 115 of the data processing device 11 generates image data representing an input screen including a bird's-eye view of the virtual space S in which the building is arranged, based on the member arrangement data stored in the member table, and the image data. Instructs the display device 13 to display the input screen represented by (step S001).

FIG. 8 is a diagram schematically showing an input screen displayed by the display device 13 according to the instruction of the image data generation means 115 in step S001. On the input screen, the shape of the building X and the viewer icon A indicating the position and line-of-sight direction of the viewer in the virtual space S are displayed. The viewer icon A is accompanied by an arrow indicating the direction of the viewer's line of sight. The viewer U inputs the position of the viewer (position of the viewing point) and the direction of the line of sight by operating the viewer icon A with a mouse or the like on the input screen.

On the input screen, the viewer U performs a predetermined operation (for example, right-clicking the mouse at an arbitrary place and selecting "Create sound source" from the displayed menu) to arrange the sound source in the virtual space S. Can be done. In FIG. 8, a sound source icon B indicating the position and sounding direction of the sound source arranged in the virtual space S by the viewer U is displayed. The sound source icon B is accompanied by an arrow indicating the sounding direction. The viewer U can arrange a plurality of sound sources in the virtual space S. Further, the position of the sound source may be either inside or outside the building X.

The viewer U inputs the position and sounding direction of the sound source by operating the sound source icon B with a mouse or the like on the input screen. Further, the viewer U can perform a predetermined operation on the sound source icon B (for example, right-click the mouse on the sound source icon B) to pop up the sound source setting window C. In the sound source setting window C, the viewer U can select the type of sound source, select the generated sound to be produced from the sound source, and specify the volume.

When the viewer U performs the above-mentioned input operation on the input screen, the operation data acquisition means 111 acquires operation data indicating the content of the input operation of the viewer U from the operation device 12 (step S002). The operation data includes viewing point position data indicating the position of the viewing point according to the position of the viewer icon A, and line-of-sight direction data indicating the viewing direction of the viewer according to the direction of the arrow accompanying the viewer icon A. , Sound source position data indicating the position of the sound source according to the position of the sound source icon B, sound source data indicating the sound source direction according to the direction of the arrow accompanying the sound source icon B, and the sound source specified in the sound source setting window C. The sound source type identification data for identifying the type, the generated sound identification data for identifying the generated sound, and the volume data indicating the volume specified in the sound source setting window C are included.

Subsequently, the data processing device 11 represents a process for generating arrival sound data (steps S101 to S103) representing the arrival sound heard by the viewer U in the virtual space S, and represents an image visible to the viewer U in the virtual space S. Processes for generating image data (steps S201 to S202) are performed in parallel.

In the generation of the arrival sound data, first, the selection means 113 specifies the sound propagation path from the position of the sound source indicated by the sound source position data to the position of the viewing point indicated by the viewing point position data (step S101). FIG. 9 is a diagram showing a part of the sound propagation path specified by the selection means 113. In FIG. 9, the propagation path R1 indicates a propagation path in which the sound emitted from the position of the sound source indicated by the sound source icon B propagates in the air and directly reaches the position of the viewing point indicated by the viewer icon A. ing. The propagation path R2 indicates a propagation path in which the sound emitted from the position of the sound source propagates in the air, is reflected by the inner surface of the ceiling member M6, and then propagates in the air to reach the position of the viewing point. .. Further, the propagation path R3 indicates a propagation path in which the sound emitted from the position of the sound source propagates through the floor member M1, and the sound propagated to just below the viewing point propagates in the air and reaches the position of the viewing point. There is. Further, in the propagation path R4, the sound emitted from the position of the sound source propagates in the air, passes through the inner wall member M7, propagates in the air, is reflected by the inner surface of the outer wall member M4, and then passes through the air. It shows a propagation path that propagates, passes through the inner wall member M7 again, and then propagates in the air to reach the position of the viewing point.

The selection means 113 assumes a case where a sound is emitted in a certain direction from the position of the sound source, and if there is no viewing point in the direction of the sound, the member arrangement data in which the member in that direction is stored in the member table. Identify based on. Subsequently, the selection means 113 reads out the material acoustic characteristic data corresponding to the material ID of the specified member as the member acoustic characteristic data indicating the acoustic characteristic of the member. Subsequently, the selection means 113 determines the direction of the sound transmitted through the member, the direction of the sound reflected on the surface of the member, the direction of the sound transmitted inside the member, and the outside of the member based on the member arrangement data and the member acoustic characteristic data. From the directions of the sound diffracting, the direction of the sound to be considered as the propagation path is selected.

For example, when the sound transmittance of the member is low, or when the member is thick in the direction of sound transmission, the sound transmitted through the member is greatly attenuated. Therefore, if the attenuation rate of the sound transmitted through the member is equal to or higher than a predetermined threshold value in all frequency bands, the selection means 113 excludes the direction of the sound transmitted through the member from the target of the propagation path and transmits the member. If the sound attenuation rate is less than a predetermined threshold in any frequency band, the direction of the sound passing through the member is selected as the target of the propagation path. The selection means 113 makes similar selections for reflection, transmission within the member, and diffraction.

In the above selection by the selection means 113, the direction of the sound transmitted through the member, the direction of the sound reflected on the surface of the member, the direction of the sound transmitted inside the member, and the direction of the sound diffracted outside the member are selected. The propagation path is branched by the number of directions of the sound. The selection means 113 specifies a member in each of the directions of the propagation path branched in the member based on the member arrangement data stored in the member table. After that, the selection means 113 repeats the above-mentioned selection of the branching direction in the member and the identification of the member in the selected direction.

The selection means 113 selects a value obtained by multiplying the attenuation rate until the sound reaches the member for each frequency band by the attenuation rate of the member in selecting the branching direction of the second and subsequent members on the propagation path. Compare with threshold. Therefore, many propagation paths are attenuated above the threshold in all frequency bands before reaching the viewing point and are excluded from consideration. Then, some propagation paths reach the viewing point with an attenuation rate less than the threshold value in one or more frequency bands. The selection means 113 specifies a sound propagation path that reaches the viewing point with an attenuation rate less than the threshold value in such a frequency band of 1 or more as a propagation path used for generating the arrival sound data.

The selection means 113 repeats the above-mentioned specific processing of the propagation path for each of the cases where the sound is emitted from the position of the sound source in various directions. Propagation paths R1 to R4 shown in FIG. 9 exemplify the propagation paths thus identified by the selection means 113.

When the identification of the propagation path in step S101 of FIG. 7 is completed, the selection means 113 refers to the propagation path data indicating the propagation path and the member if there is a member on the propagation path for each of the identified propagation paths. The member ID of the above is handed over to the sound data generation means 114 (step S102). By the processing of steps S101 and S102 by the selection means 113, a member to be considered in the generation of the arrival sound data is selected from the plurality of members constituting the building X.

When a plurality of sound source position data are acquired by the operation data acquisition means 111 in step S002, the selection means 113 performs the processes of steps S101 and S102 with respect to the plurality of sound sources indicated by the sound source position data.

The sound data generation means 114 receives a plurality of propagation path data from the selection means 113 in step S102. A part of the propagation path data received by the sound data generation means 114 is associated with one or more member IDs that identify the members on the propagation path. The sound data generation means 114 reads out the member arrangement data corresponding to the member IDs from the member table for each of the member IDs received from the selection means 113. Further, the sound data generation means 114 refers to the material acoustic characteristic data corresponding to the member ID for each of the member IDs received from the selection means 113, and the member acoustic characteristic data indicating the acoustic characteristics of the member identified by the member ID. Read from the material table as.

The sound data generation means 114 acquired the propagation path data received from the selection means 113 in step S102, the member arrangement data read from the member table, the member acoustic characteristic data read from the material table, and the operation data acquisition means 111 in step S002. The arrival sound data is generated based on the sound generation direction data, the sound source type identification data, the generated sound identification data, the volume data, and the line-of-sight direction data (step S103). The start point of the propagation path indicated by the propagation path data is the position of the sound source indicated by the sound source position data, and the end points are the positions of the viewing points indicated by the viewing point position data.

Specifically, first, the sound data generation means 114 reads out the directivity data corresponding to the type of the sound source indicated by the sound source type identification data from the storage device 112. Subsequently, the image data generation means 115 converts the attenuation rate for each frequency band due to the directivity of the sound emitted from the sound source along the propagation path data into the read directivity data and the sounding direction data for each of the propagation path data. Identify based on.

Further, the sound data generation means 114 determines the attenuation rate for each frequency band when the sound propagates in the propagation path indicated by the propagation path data, and the member arrangement data of the members on the propagation path and the member arrangement data of the members on the propagation path. Specify based on the member acoustic characteristic data. Subsequently, the sound data generation means 114 reads the generated sound data identified by the generated sound identification data from the storage device 112, and specifies the generated sound represented by the read generated sound data for each of the propagation paths as described above. A sound is generated by multiplying the attenuation rate due to the directivity and the attenuation rate due to sound propagation for each frequency band.

Subsequently, the sound data generation means 114 mixes the sound for each propagation path generated as described above into, for example, the arrival sounds of the left and right channels. In the mixing, the sound data generation means 114 multiplies the line-of-sight direction indicated by the line-of-sight direction data for each of the propagation paths and the amplification factor according to the direction in which the propagation path heads toward the viewing point (sound arrival direction), and then obtains them. Add sounds. The sound data generation means 114 amplifies the sound of the two left and right channels obtained by mixing, as indicated by the volume data. The sound data generation means 114 generates data representing the sound of the two left and right channels after amplification as the arrival sound data.

As described above, in parallel with the process for generating the arrival sound data in steps S101 to S103, the data processing device 11 performs the process for generating image data (steps S201 to S202). First, the image data generation means 115 constitutes a plurality of buildings X based on the viewing point position data and the line-of-sight direction data acquired by the operation data acquisition means 111 in step S002, and the member arrangement data stored in the member table. From the members, a member required for generating image data is selected (step S201). The member selected by the image data generation means 115 in step S201 is within a predetermined viewing angle (angle of view) centered on the direction from the viewing point indicated by the viewing point position data to the line-of-sight direction indicated by the line-of-sight direction data. It is a member to be arranged.

Subsequently, the image data generation means 115 reads out the member arrangement data of the members selected in step S201 from the member table, and obtains data indicating the appearance according to the material IDs of those members (hereinafter, referred to as appearance data) from the material table. read out. The image data generation means 115 generates image data representing an image that can be seen by a viewer facing the line-of-sight direction at the viewing point, based on the read member arrangement data and appearance data, and the viewing point position data and the line-of-sight direction data (step). S202). Since the process in step S202 is a well-known process adopted in BIM and the like, the description thereof will be omitted.

The instruction means 116 instructs the sounding apparatus 14 to pronounce the sound represented by the arrival sound data generated by the sound data generation means 114 as described above, and at the same time, the image data generated by the image data generation means 115 as described above. Instructs the display device 13 to display the image represented by (step S301). The sounding device 14 sounds the arrival sound according to the instruction by the instruction means 116, and the display device 13 displays the figure according to the instruction of the instruction means 116. As a result, the viewer U can hear the sound heard when facing the line-of-sight direction at the viewing point, and can see the image seen when facing the line-of-sight direction. FIG. 10 is a diagram showing how the virtual space S is provided to the viewer U. As shown in FIG. 10, the display device 13 displays a screen (hereinafter, referred to as a graphic display screen) including the image of the virtual space S. Further, the sound in the virtual space S is sounded from the sounding device 14.

In addition, the input screen is reduced and displayed on the figurative display screen. The viewer U can perform operations such as changing the position and direction of the viewer icon A and the sound source icon B, adding a new sound source, changing the type of sound source, the generated sound, and the volume on this input screen. When the viewer U performs those operations, the data processing device 11 repeats the processing shown in FIG. 7 based on the newly acquired data in response to those operations. As a result, the viewer U can experience the change in shape and sound when moving in the virtual space S in substantially real time, and can experience the change in sound when the sound source is changed. it can.

According to the virtual space viewing system 1 described above, a virtual space that brings a high sense of presence is provided to the viewer. By the way, the member table (see FIG. 4) is data created as BIM data in the design of the building X. Therefore, in the virtual space viewing system 1, it is not necessary to newly create a member table.

Further, among the data stored in the material table (see FIG. 6), the data other than the material acoustic property data is the data created as BIM data if BIM is used in the design of the building X. is there. Therefore, it is not necessary to newly create the appearance data used for generating the image data in the virtual space viewing system 1.

Further, in the virtual space viewing system 1, when many members of the same material are included in the members constituting the building X, the material acoustic characteristic data corresponding to the materials of those members is read out from the material table. Used. Therefore, the user (for example, the administrator of the virtual space viewing system 1) can input the material acoustic property data into the material table once for each material, and then for each of the plurality of members constituting the building X, There is no need to enter acoustic characteristic data.

As described above, according to the virtual space viewing system 1, the user can provide the virtual space to the viewer without requiring a great deal of labor for inputting data.

Further, in the virtual space viewing system 1, among the plurality of members constituting the building X, the member to be considered in the generation of the arrival sound data is selected by the selection means 113. At that time, the selection means 113 excludes the member on the propagation path in which the sound is greatly attenuated from the members to be considered in the generation of the arrival sound data, based on the member acoustic characteristic data. Therefore, regarding the propagation path with little sound attenuation, even if a plurality of members are located on the propagation path, the acoustic characteristics of those members are reflected in the arrival sound data, but the sound attenuation is large. Regarding the propagation paths, the acoustic characteristics of the members on those propagation paths are not reflected in the arrival sound data. Therefore, according to the virtual space viewing system 1, a sound that brings a high sense of presence is provided to the viewer without a large increase in the processing load.

[Modification example]
The above-described embodiment is an embodiment of the present invention, and the present invention is not limited thereto, and can be variously modified within the scope of the technical idea. Examples of these modifications are shown below. In addition, two or more of the above-described embodiment and the modified examples illustrated below may be combined.

(1) In the description of the above-described embodiment, the case where the viewing point designated by the viewer is inside the building is illustrated, but as described above, the viewing point may be arranged outside the building. Good. FIG. 11 is a diagram illustrating an input screen when the viewing point is arranged outside the building X. In the example of FIG. 11, the sound source indicated by the sound source icon B1 and the sound source indicated by the sound source icon B2 are arranged inside the building X. Further, the sound source indicated by the sound source icon B3 is arranged outside the building X.

In this case, the viewer U receives the sound that reaches the viewing point from the inside of the building X through the outer wall member M5 at the viewing point indicated by the viewer icon A, and directly from the outside of the building X, or. A sound is provided in which the sound that reaches the viewing point after being reflected by the outer wall member M5 is mixed. As described above, according to the virtual space viewing system 1, the viewer is provided with a high sense of presence when the viewer is outside the building in the virtual space.

(2) In the above-described embodiment, the material acoustic characteristic data stored in the material table is used as the member acoustic characteristic data indicating the acoustic characteristics of the member without being converted or the like. When the sound data generation means 114 cannot read and use the material acoustic characteristic data stored in the material table as it is due to, for example, a difference in data format, the data processing device 11 uses the material acoustic characteristic data as sound data. A configuration may be adopted in which the generation means 114 includes conversion means for converting into available member acoustic characteristic data.

In this modification, for example, the storage device 112 stores conversion rule data indicating rules for conversion. The conversion rule data may be, for example, a correspondence table showing the correspondence of the data before and after the conversion. The conversion means reads the material acoustic characteristic data according to the material of the member from the material table, converts the read material acoustic characteristic data according to the rule indicated by the conversion rule data, and generates the member acoustic characteristic data. The member acoustic characteristic data converted by the conversion means in this way is used to generate the arrival sound data by the sound data generation means 114.

For example, if the material acoustic property data stored in the material table shows acoustic properties for eight frequency bands and the sound data generating means 114 can use the acoustic properties for ten frequency bands, the conversion means is material acoustic. The acoustic characteristics related to the eight frequency bands indicated by the characteristic data are converted into the acoustic characteristics related to the ten frequency bands by processing such as interpolation, and the data showing the converted acoustic characteristics is used as the member acoustic characteristic data in the sound data generation means 114. hand over.

(3) In the above-described embodiment, the sound data generation means 114 is assumed to generate arrival sound data of two left and right channels. The number of channels of the arrival sound data is not limited to two, and for example, arrival sound data indicating surround sounds of three or more channels may be generated.

(4) In the above-described embodiment, sound reflection characteristics, sound transmission characteristics, and sound transmission characteristics are adopted as the acoustic characteristics used for generating the arrival sound data. These are examples of acoustic characteristics, and other types of acoustic characteristics such as sound diffusivity may be used to generate the arrival sound data.

(5) In the above-described embodiment, the generated sound data is represented by a sound wave shape. The format of the generated sound data is not limited to the sound wave type. For example, the generated sound data may indicate parameters used in a synthesizer that processes a basic sound wave shape or the like to generate various sounds.

(6) In the above-described embodiment, the data processing device 11 is provided with the storage device 112 for storing the member table, the material table, and the like. Instead of this, the storage device 112 may be connected to the outside of the data processing device 11. Further, in the above-described embodiment, the display device 13 and the sounding device 14 are connected to the outside of the data processing device 11. At least one of these may be built into the data processing device 11.

(7) In the above-described embodiment, the member table is data created as BIM data in the design of the building X. Instead of this, for example, data of three-dimensional CAD (Computer Aided Design) may be used as a member table.

(8) In the above-described embodiment, the data processing device 11 is realized by the computer 10 performing processing according to the program according to the present embodiment. Instead of this, the data processing device 11 may be configured as a dedicated device.

(9) In the above-described embodiment, the program that causes the computer 10 to function as the data processing device 11 including the component shown in FIG. 3 continuously records the data, and the computer 10 can read the data. It may be recorded on a medium, distributed, read by the computer 10 and used, or downloaded and used by the computer 10 via a communication line such as the Internet.

1 ... Virtual space viewing system, 10 ... Computer, 11 ... Data processing device, 12 ... Operation device, 13 ... Display device, 14 ... Sound device, 101 ... Memory, 102 ... Processor, 103 ... Operation input I / F, 104 ... Image output I / F, 105 ... Sound output I / F, 111 ... Operation data acquisition means, 112 ... Storage device, 113 ... Selection means, 114 ... Sound data generation means, 115 ... Image data generation means, 116 ... Instruction means

Claims (7)

Member placement data indicating the arrangement of each of the plurality of members constituting the building arranged in the three-dimensional space in the three-dimensional space, and a sound source indicating the position of each of one or more sound sources arranged in the three-dimensional space. Each of the one or more sound sources is based on the position data, the viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and the member acoustic characteristic data indicating the acoustic characteristics of each of the plurality of members. A sound propagation path from to the viewing point that branches in a plurality of directions in the member, and a sound from the directions of the plurality of propagation paths that branch in the member in the direction of the sound emitted from the sound source. A sound attenuation rate is determined from among a plurality of propagation path directions that branch off in a member in the direction of each of the selected sound propagation path directions by selecting a propagation path whose attenuation rate is less than a predetermined threshold. A selection means for specifying by repeating the process of selecting a propagation path below the threshold value and selecting one or more members located on the specified propagation path from the plurality of members.
The member arrangement data regarding one or more members selected by the selection means, the sound source position data, the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources, and the selection means. Sound data generation means for generating arrival sound data representing the arrival sound, which is the sound heard by the viewer at the viewing point, based on the member acoustic characteristic data for one or more selected members and the viewing point position data. When,
An image data generation means for generating image data representing a figure visible to the viewer based on the viewing point position data, the line-of-sight direction data indicating the line-of-sight direction of the viewer at the viewing point, and the member arrangement data.
An instruction means for instructing the sounding device to pronounce the sound represented by the arrival sound data generated by the sound data generating means, and at the same time instructing the display device to display an image represented by the image data generated by the image data generating means. A data processing device comprising. The member arrangement data is BIM data created in the design of the building, and is
The member acoustic characteristic data is data stored in association with data indicating the physical properties of the material of the member created as the BIM data for each of the plurality of members.
The data processing apparatus according to claim 1, wherein the image data generation means generates image data representing an image visible to the viewer by the same processing as that performed by the BIM . The sound data generation means is used from a storage device that stores material acoustic characteristic data indicating the acoustic characteristics of the material for each of the plurality of materials to the material of the member for each of the one or more members selected by the selection means. The data processing apparatus according to claim 1 or 2, wherein the corresponding material acoustic characteristic data is acquired as the member acoustic characteristic data relating to the member. The selection means collects material acoustic property data corresponding to the material of the member for each of the plurality of members from a storage device that stores material acoustic property data indicating the acoustic property of the material for each of the plurality of materials. The data processing apparatus according to any one of claims 1 to 3, which is acquired as characteristic data and selects one or more members based on the member acoustic characteristic data. At least one of the positions of the one or more sound sources indicated by the sound source position data is inside the building, and the position of the viewing point indicated by the viewing point position data is outside the building. The data processing apparatus according to any one of 4. Computer,
Member placement data indicating the arrangement of each of the plurality of members constituting the building arranged in the three-dimensional space in the three-dimensional space, and a sound source indicating the position of each of one or more sound sources arranged in the three-dimensional space. Each of the one or more sound sources is based on the position data, the viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and the member acoustic characteristic data indicating the acoustic characteristics of each of the plurality of members. A sound propagation path from to the viewing point that branches in a plurality of directions in the member, and a sound from the directions of the plurality of propagation paths that branch in the member in the direction of the sound emitted from the sound source. A sound attenuation rate is determined from among a plurality of propagation path directions that branch off in a member in the direction of each of the selected sound propagation path directions by selecting a propagation path whose attenuation rate is less than a predetermined threshold. A selection means for specifying by repeating the process of selecting a propagation path below the threshold value and selecting one or more members located on the specified propagation path from the plurality of members.
The member arrangement data regarding one or more members selected by the selection means, the sound source position data, the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources, and the selection means. Sound data generation means for generating arrival sound data representing the arrival sound, which is the sound heard by the viewer at the viewing point, based on the member acoustic characteristic data for one or more selected members and the viewing point position data. When,
An image data generation means for generating image data representing a figure visible to the viewer based on the viewing point position data, the line-of-sight direction data indicating the line-of-sight direction of the viewer at the viewing point, and the member arrangement data.
As an instruction means for instructing the sounding device to pronounce the sound represented by the arrival sound data generated by the sound data generating means, and at the same time instructing the display device to display the image represented by the image data generated by the image data generating means. A program to make it work. Computer,
Member placement data indicating the arrangement of each of the plurality of members constituting the building arranged in the three-dimensional space in the three-dimensional space, and a sound source indicating the position of each of one or more sound sources arranged in the three-dimensional space. Each of the one or more sound sources is based on the position data, the viewing point position data indicating the position of the viewing point arranged in the three-dimensional space, and the member acoustic characteristic data indicating the acoustic characteristics of each of the plurality of members. A sound propagation path from to the viewing point that branches in a plurality of directions in the member, and a sound from the directions of the plurality of propagation paths that branch in the member in the direction of the sound emitted from the sound source. A sound attenuation rate is determined from among a plurality of propagation path directions that branch off in a member in the direction of each of the selected sound propagation path directions by selecting a propagation path whose attenuation rate is less than a predetermined threshold. A selection means for specifying by repeating the process of selecting a propagation path below the threshold value and selecting one or more members located on the specified propagation path from the plurality of members.
The member arrangement data regarding one or more members selected by the selection means, the sound source position data, the generated sound data representing the generated sound which is the sound emitted from each of the one or more sound sources, and the selection means. Sound data generation means for generating arrival sound data representing the arrival sound, which is the sound heard by the viewer at the viewing point, based on the member acoustic characteristic data for one or more selected members and the viewing point position data. When,
An image data generation means for generating image data representing a figure visible to the viewer based on the viewing point position data, the line-of-sight direction data indicating the line-of-sight direction of the viewer at the viewing point, and the member arrangement data.
As an instruction means for instructing the sounding device to pronounce the sound represented by the arrival sound data generated by the sound data generating means, and at the same time instructing the display device to display the image represented by the image data generated by the image data generating means. A computer-readable recording medium that continuously records programs for functioning.

Images