JP2882449B2 - Sound image localization control device for video games - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound image localization control device for a television game, and more particularly, to a listener using a pair of transducers (loudspeakers) for reproduction in accordance with a reproduced image. The present invention relates to a sound image localization control device for a television game that makes a sound image appear to be localized at a desired arbitrary position different from the above.

[0002]

2. Description of the Related Art Conventionally, there has been a sound image localization method using a binaural technique in which a sound source is felt at a specific position (direction) by a signal level difference and a phase difference (time difference) in both ears. As a sound image localization method using an analog circuit, for example, Japanese Patent Application Laid-Open No. Sho 53-140001 (Japanese Patent Publication No.
58-3638) There is a sound image localization method described in the official gazette. This means that the level of a specific frequency band is emphasized and attenuated by an analog filter (by controlling the amplitude) to give a sense of the front and back of the sound source, and the analog delay causes a time difference between the left and right sounds (by controlling the phase). The left and right feeling was given.

Further, with the recent development of digital processing technology, there is a sound image localization method realized by a digital circuit. For example, there is a “sound image forming method and apparatus” described in JP-A-2-298200. The sound image localization method using this digital circuit converts a signal from a sound source into an FFT (Fast Fourier
Transform), processes on the frequency axis, and applies a frequency-dependent level difference and a phase difference to both the left and right channel signals to digitally control the localization of the sound image. The level difference and the phase difference depending on the frequency at each sound image localization position in this method are collected as experimental data using an actual listener. However, the sound image localization method using such a circuit has the disadvantage that the circuit scale becomes extremely large when trying to localize the sound image accurately and precisely, and it is only used as a special professional recording system. Did not.

[0004]

By the way, recently,
Amusement game machines and computer terminals using virtual reality have emerged. Also in these game machines and terminals, demands for sound image localization with a sense of reality corresponding to the screen have begun. Moreover,
For example, in a game machine (or a terminal device), the course (position) at which an airplane flies differs according to the operation of the operator, and the moving process of the sound image is performed in real time according to the operation according to the operation. , It needs to be played. This point is significantly different from the above-described sound image localization processing for a record.

Accordingly, the present invention has been made in view of the above-mentioned conventional problems, and has a small circuit size, low cost, and localization in a wide space exceeding 180 degrees corresponding to an image. It is intended to provide a sound image localization control device for a television game which can perform the above.

[0006]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention is to provide a sound image localization control device for a video game having the following constitutions 1) to 7). 1) A predetermined image is reproduced based on the incoming video display data, and the sound image localization processing means
To perform sound image processing on the supplied monaural sound source
From a pair of spaced apart transducers
In a sound image localization control device for a video game for causing a sound image to be localized at an arbitrary position different from the transducer, a reserve for supplying the sound image to the sound image localization processing means is provided.
Head transfer function measured for multiple localization positions
Coefficient holding means for holding a plurality of coefficients based on the sound image
Constituting the localization processing means, by the arrival of the monaural sound source
A tatami mat for the monaural sound source based on the plurality of coefficients;
For performing sound image processing by performing
A convolver, a control unit for generating localization information of the sound image based on an operation of the game controller, and a frame image following the generation of the localization information, controlling the sound image localization processing unit in synchronization with the frame image. A sound image localization control device for a video game, comprising: a synchronization control unit.

2) Video display data and audio data
The digital data and the sound image localization information are supplied, and are used in a television game system for reproducing a predetermined image and sound. The sound supplied from a monaural sound source is supplied to a pair of convolvers, and is supplied to a listener. A sound image localization control device for a video game for causing a pair of transducers to feel as if the sound image is located at a position different from the positions of these transducers in synchronization with an image, Coefficient holding means for holding a coefficient prepared for changing the localization position of the sound image, and coefficient extraction means for receiving the sound image localization information and extracting a coefficient corresponding thereto from the coefficient holding means, A coefficient extracting unit, a synchronizing unit for supplying the sound image localization information during a vertical synchronous blanking period of the image, and a coefficient extracted by the coefficient extracting unit. Sound image localization control apparatus for a video game to the coefficient supply means for supplying to the convolver, characterized in that provided.

3) Video display data and audio data
The digital data and the sound image localization information are supplied, and are used in a television game system for reproducing a predetermined image and sound. The sound supplied from a monaural sound source is supplied to a pair of convolvers, and is supplied to a listener. A sound image localization control device for a video game for causing a pair of transducers to feel as if the sound image is located at a position different from the positions of these transducers in synchronization with an image, Coefficient holding means for holding a coefficient prepared for changing the localization position of a sound image; coefficient extraction means for responding to the sound image localization information and extracting a coefficient corresponding to the information from the coefficient holding means; and Coefficient supply means for supplying the coefficient extracted by the extraction means to the pair of convolvers; and a vertical synchronization blankin of the image for the coefficient supply means. Sound image localization control apparatus for a video game to the synchronizing means for instructing to start the supply to the pair of convolver during, that was provided with the features.

4) Video display data and audio data
The digital data and the sound image localization information are supplied, and are used in a television game system for reproducing a predetermined image and sound. The sound supplied from a monaural sound source is supplied to a pair of convolvers, and is supplied to a listener. A sound image localization control device for a video game for causing a pair of transducers to feel as if the sound image is located at a position different from the positions of these transducers in synchronization with an image, A coefficient holding means for holding a coefficient prepared for changing the localization position of the sound image, a coefficient selection control means for responding to the sound image localization information, and selectively switching and extracting a coefficient corresponding thereto, A video game, comprising: synchronization means for outputting an output of a coefficient selection control means to the coefficient holding means during a vertical synchronization blanking period of the image. The sound image localization control device.

[0010] 5) In the sound image localization control apparatus according to claim 1, 2, 3 or 4 for the description of the video game, only a plurality of sets setting a pair of convolver, monaural sound source these convolver it
A sound image localization control device for a video game, which is supplied to each of them .

6) The sound image localization control device for a video game according to claim 5, further comprising means for performing cross-fade processing on the input or output sides of the plurality of sets of convolvers. apparatus.

(7) In the sound image localization control device for a video game according to the above (2), (3) or (4), switching control for switching the sound input of the pair of convolvers substantially in synchronization with the coefficient supply means or the coefficient selection control means. A sound image localization control device for a video game, characterized by comprising means.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a sound image localization control device for a television game according to the present invention will be described below with reference to the drawings. First, the basic principle of the sound image localization control method employed by the game sound image localization control device will be described. This is a technique in which a sound image is localized at an arbitrary position in a space using a pair of transducers (hereinafter, a speaker will be described as an example) that are arranged apart from each other.

FIG. 10 is a diagram illustrating the principle of sound image localization. sp
Reference numerals 1 and sp2 denote speakers arranged in front and left of a listener (which may be referred to as a listener in the embodiment),
The head transfer characteristic (impulse response) from sp1 to the listener's left ear is h1L, and the head transfer characteristic from the right ear to h1R is s.
The head transfer characteristics from p2 to the left and right ears are defined as h2L and h2R. Also, the head transfer characteristic to the listener's left and right ears when an actual speaker is placed at the target localization position x is pL.
x, pRx. Here, each transfer characteristic is obtained by subjecting a speaker in an acoustic space and microphones to both ears of a dummy head (or a human head) and actually measuring them, and performing appropriate waveform processing or the like.

Next, signals obtained by passing a sound source X to be localized through signal converters cfLx and cfRx (transfer characteristics by a convolver or the like) are respectively converted into sp1 and sp
Consider playing on 2. At this time, assuming that the signals obtained from the left and right ears of the listener are eL and eR, eL = h1L · cfLx · X + h2L · cfRx · X (Equation 1) eR = h1R · cfLx · X + h2R · cfRx · X (〃)

On the other hand, assuming that signals obtained at the listener's left and right ears when the source X is reproduced from the intended localization position are dL and dR, dL = pLx · X (Equation 2) dR = pRx · X (〃)

Here, if the signals obtained in the left and right ears of the listener by the reproduction of sp1 and sp2 coincide with the signals obtained when the source is reproduced from the target position, the listener can operate as if the speaker exists at the target position. It will recognize the sound image. From the conditions eL = dL, eR = dR and (Equation 1) and (Equation 2), X is eliminated and h1L · cfLx + h2L · cfRx = pLx (Equation 3) h1R · cfLx + h2R · cfRx = pRx (〃) (Equation 3) ), CfLx = (h2R · pLx−h2L · pRx) / H (Equation 4a) cfRx = (− h1R · pLx + h1L · pRx) / H (〃) where H = h1L · h2R−h2L · h1R (Formula 4b)

Therefore, if a signal to be localized is processed by a convolver (convolution operation processing circuit) or the like using the transfer characteristics cfLx and cfRx calculated by (Equation 4a) and (Equation 4b), a sound image is obtained at a target position x. Can be localized. Although various methods for realizing the signal conversion device are conceivable, they may be realized using an asymmetric FIR digital filter (convolver). The final transfer characteristic when used in the FIR digital filter is a time response function.

That is, as the transfer characteristics cfLx and cfRx at the required localization position x, (Equation 4a) and (Equation 4b)
The coefficients cfLx and cfRx are created in advance as coefficients to be realized by one FIR filter process, and are prepared as ROM data.
If the necessary coefficient of the sound image fixed position is transferred from the ROM to the FIR digital filter, the signal from the sound source is convolution-processed and reproduced from a pair of speakers, the sound image is localized at a desired arbitrary position.

Next, a method of real sound image localization control based on such a principle will be described in detail with reference to FIG. FIG.
Shows steps of a method of sound image localization control. Measurement of Head Related Transfer Function (HRTF) (Step 101)

This will be described with reference to FIGS. FIG. 12 shows an HRTF measurement system. A pair of microphones ML and MR are installed in both ears of the dummy head (or human head) DM, a measurement sound from the speaker SP is received, and source sounds (reference data) refL and refR and a sound to be measured (measurement data) are received in the recorder DAT. ) L,
Record R synchronously.

As the source sound XH, an impulse sound, white noise, other noise, or the like can be used. In particular, from the viewpoint of statistical processing, white noise is
Since the energy distribution is continuous and the energy distribution is constant over the audio band, the SN
The ratio improves. The position of the speaker SP is defined by a plurality of angles θ (for example,
As shown in FIG. 14, it is set at 12 points every 30 degrees), and each is continuously recorded for a predetermined time.

HRTF impulse response (Impulse Re
calculation of sponse (hereinafter referred to as IR) (step 10)
2) In step 101, the source sounds (reference data) refL, refR and the measured sounds (measurement data) L, R, which are recorded synchronously, are processed on a workstation (not shown).

If the frequency response of the source sound (reference data) is X (S), the frequency response of the measured sound (measurement data) is Y (S), and the frequency response of the HRTF at the measurement position is IR (S), There is an input / output relationship shown in equation 5). Y (S) = IR (S) · X (S) (Equation 5) Therefore, the frequency response of the HRTF is expressed by IR (S) = Y (S) / X (S) (Equation 6) is there. Thus, the frequency response of the reference X (S),
The frequency response Y (S) of the measurement data is calculated in step 10
1 is cut out by time-synchronized windows, and finite Fourier series expansion is performed by FFT transform to calculate discrete frequencies. From (Equation 6), the frequency response IR (S) of the HRTF is calculated by a known calculation method. Is required.

In this case, the accuracy of IR (S) is increased (S
In order to improve the N ratio), it is preferable to calculate IR (S) for each of several hundred windows different in time and average them. Then, the calculated HRTF frequency response IR (S)
Is subjected to an inverse FFT transform to obtain an HRTF time axis response (impulse response) IR (first IR).

IR (Impulse Response) Shaping Process (Step 103) Here, the IR obtained in Step 102 is shaped. First, for example, by FFT transform, the first
The IR band at discrete frequencies over the audio spectrum, and remove unnecessary bands (a large dip occurs in the high frequency band, which is an unnecessary one that does not significantly affect sound image localization) with a BPF (bandpass filter). I do. When the band is limited in this manner, unnecessary peaks and dips on the frequency axis are removed, and unnecessary coefficients are not generated in the cancel filter, so that convergence is improved and the coefficients can be shortened.

Then, the band-limited IR (S) is converted to the inverse F
After performing FT conversion, an IR (impulse response) is multiplied by a cutout window (for example, a window of a cosine function) on the time axis, and a window process is performed (a second IR). By performing the window processing, the effective length of the IR is no longer long, the convergence of the cancel filter is improved, and the sound quality does not deteriorate.

Cancel filters cfLx, cfRx
(Step 104) The cancellation filters cfLx and cfRx, which are convolvers (convolution integrators), are, as shown in (Equation 4a) and (Equation 4b), cfLx = (h2R · pLx−h2L · pRx) / H (Formula 4a) cfRx = (− h1R · pLx + h1L · pRx) / H (〃) where H = h1L · h2R−h2L · h1R (Formula 4b).

Here, the speakers sp1 and sp
2 transfer characteristics h1L, h1R, h2L, h2R
The second IR (formed at steps 101 to 103, which has been shaped and processed for each angle θ, as the head-related transfer characteristics pLx and pRx when an actual speaker is placed at the target localization position x). Impulse response).

The head transfer characteristics h1L and h1R correspond to the position of the L-channel speaker in FIG. 15, and if the head is set at, for example, 30 degrees (θ = 330 degrees) from the front to the left, θ = 330. The degree of IR is used. Head transfer characteristics h2
R and h2L correspond to the position of the R channel speaker in FIG.
1), an IR of θ = 30 degrees is used (that is, a system at the time of actual sound image reproduction (for example, FIG. 1)
3)).

The head-related transfer characteristics pLx and pRx are 90 degrees from the front which is the target sound source localization position.
By substituting the IR for every 30 degrees in a wide space (entire space) beyond the range of 180 degrees as well as in the 180 degree range, the corresponding cfLx,
cfRx, that is, 12 sets of cancellation filters cfLx and cfRx groups are obtained every 30 degrees (in FIG. 15, a position at 240 degrees is taken as an example). The group of cancellation filters cfLx and cfRx is finally obtained as an IR (impulse response) which is a response on the time axis.
Note that the cancellation filter cfLx according to (Equation 4a)
The calculation of cfRx is as follows. First, H-1 which is a kind of inverse filter for H in (Equation 4b) is obtained by the least square method, and this is subjected to inverse FFT transform to obtain a time function h (t). Also, each term h1L, h1R, h2L, p in (Equation 4a)
By expressing Rx, pLx, and h2R with time functions, the following equation is established.

CfLx (t) = (h2R · pLx−h2L · pRx) · h (t) (Equation 7) cfRx (t) = (− h1R · pLx + h1L · pRx) · h (t) (〃)

Therefore, the coefficients of the cancel filters cfLx and cfRx are obtained from these (Equation 7). As is clear from (Equation 7), in order to shorten the coefficients of the cancel filters cfLx and cfRx, the head transfer characteristics h1L, h1R, h2L, pRx, pLx,
It is very important to shorten h2R. Therefore, as described above, in steps 101 to 103, various processes such as a window process and a shaping process are performed, and the respective head transfer characteristics h1L, h1R, h2L, pRx, pLx, h
2R is shortened.

The scaling of the cancel filter at each localization point x (step 105) The spectral distribution of the sound source (source sound) actually processed by the sound image processing by the convolver (cancel filter) is statistically similar to pink noise. In some cases, the sound source is different from a single sound, so there is a danger that the convolution operation (integration) will overflow and cause distortion. is there. Therefore, in order to prevent overflow, the coefficient of the maximum gain (for example, the sum of squares of the sample values of the cancel filters cfLx and cfRx) among the coefficients of the cancel filters cfLx and cfRx is found, and the coefficient and the white noise of 0 db are found. Is scaled so that overflow does not occur when convolving.

In this manner, the scaling processing is performed.
Finally, data groups cfLx and cfRx to be supplied to the convolver as coefficients (in this example, 12 sets of convolver coefficient groups capable of localizing a sound image every 30 degrees) are obtained. Playback by convolution of signal from sound source (step 1
06)

For example, as shown in FIG. 13, a pair of speakers sp1 and sp2 are disposed at right and left sides of a game operator (listener) M at a distance of 30 degrees as a basic configuration of a sound reproducing device of a game machine. And a pair of speakers s
A sound signal processed by a pair of convolvers (convolution operation processing circuit) is configured to be reproduced in p1 and sp2.

A signal from the same sound source X (for example, a monaural sound such as a flying sound from a game synthesizer) is supplied to the pair of convolvers, and a signal is supplied to the pair of convolvers.
The IR coefficients cfLx and cfRx (for example, a coefficient of θ = 240 degrees when the sound image is to be localized at a position 120 degrees to the rear left (θ = 240 degrees)) are selected by the convolver Is set to For example, based on a sound image localization command from a main CPU (central processing unit) of a game machine or the like, the control sub-CPU transfers a coefficient at a desired localization position from the coefficient ROM to a pair of convolvers.

As described above, the signal from the sound source X is convolution-processed on the time axis by the pair of convolvers, and the pair of speakers sp1 and sp
Reproduced from 2. The sound reproduced from the pair of speakers sp1 and sp2 is canceled by the crosstalk to both ears, localized in a sound image so that the sound source is located at a desired position, and heard by the game operator (listener) M, It is reproduced as an extremely realistic sound. As for the convolver coefficient, the optimal sound image position is sequentially selected and switched with the transition of the movement of the airplane according to the operation of the operator. When changing from the flight sound to, for example, a missile sound, the source sound from the sound source X is changed from the flight sound to the missile sound. In this manner, the sound image can be freely localized at an arbitrary position.

Next, a first embodiment of the present invention based on such a basic principle configuration will be described. FIG.
FIG. 1 is a schematic block diagram of a sound image localization control device for a television game according to a first embodiment. Reference numeral 51 denotes a central processing unit (CPU) to which a controller 55 for controlling the CPU 51 and a game cassette 50 are connected via a data bus. ing. The game cassette 50 includes a video
Display data, audio data and sound image position information are recorded in a predetermined relationship with each other. An interface (IF) 52, a graphic system processor (GSP) 54 for expressing a desired graphic image, and a synthesizer 57 are connected to the central processing unit 51 through a data bus. The interface 52 has a CD-ROM 53
Are connected. Similarly, this CD-R
In the OM 53, video display data, audio data, and sound image position information are recorded in a predetermined relationship. A television monitor 60 is connected to the graphic system processor 54 via a video output terminal 56. 58 and 59 are terminals for outputting left and right audio signals of the synthesizer 57, respectively.

Each of the interface 52 and the graphic system processor 54 is connected to a sub central processing unit (SUB CPU) through a data bus.
61 is also connected. This sub central processing unit 6
1 is connected to a PCM sound source 62 through a data bus.
Further, a sound source RAM 63 is connected to the PCM sound source 62. The sound source RAM 63 is temporarily prepared for storage here because the amount of data supplied from the CD-ROM 53 is large, and is controlled by the PCM sound source 62.

Reference numeral 64 denotes audio data supplied from the CD-ROM 53 in a predetermined MIDI (Musical).
Instrument Digital Interf
ace) by MIDI conversion means for the next stage MID
The MIDI sound source 66 is connected to the terminal a of the switch (SW) 69. The output from the PCM sound source 69 is connected to the terminal b of the switch 69. This switch 69 is controlled by the sub central processing unit 61 described above.

Reference numeral 67 denotes a third central processing unit having a coefficient extracting means 67a and a coefficient supplying means 67b. The coefficient extracting means 67a is controlled by sound image localization information supplied from the sub central processing unit 61 via the MIDI sound source 66. It is configured to be. 68 is a storage means (ROM)
Then, for example, 12 sets of convolver coefficient groups cfLx, c for every 30 degrees calculated by the above steps
fRx is stored in advance. In this case, the coefficients for localization may be regarded as symmetric, and only one of the left and right coefficients may be prepared. The output of the coefficient extracting means 67a is connected in parallel to the left convolver 71 and the right convolver 72 via the coefficient supplying means 67b, respectively.
These outputs are analog / digital converters (D / A)
73, 74, and their outputs are connected to output terminals 7
6,77.

Next, the operation of the above configuration will be described. Now, when the game CD-ROM 53 is mounted and the controller 55 is operated, the CD-ROM 5
3, the video display data, audio data and sound image position information recorded in association with each other, and the operation signal from the controller 55 are supplied to the central processing unit 51. You. As a result, an instruction signal for image creation is supplied to the graphic system processor 54, where, for example, an image signal that forms a mosaic image composed of a plurality of partial images is generated and output. The signal is output from the terminal 56 to the television monitor 60. At this time, the graphic system processor 54 generates a field or frame synchronizing signal as shown in FIG. 2A in accordance with the image signal, and this signal is transmitted through the data bus to the sub-center. It is supplied to the processing unit 61.

On the other hand, a controller 55 is connected to the sub central processing unit 61 via the central processing unit 51.
For example, an operation signal shown in FIG. 2B is supplied. This operation signal is asynchronous in time with the field or frame synchronization signal. In the sub central processing unit 61, the sound image localization is determined in the timing period shown in FIG. 2C based on the operation signal and the sound image position information coming from the CD-ROM 53, and the sound image localization information is determined based on this determination. During the blanking period of the field or frame synchronization signal shown in FIG. 2 (D), it is supplied to the third central processing unit 67 via the MIDI conversion means 64 and the MIDI sound source 66 sequentially (that is, through). . In accordance with this timing, the coefficient extracting means 67a extracts predetermined coefficients from the storage means 68 for each block based on the sound image localization information, and retrieves these coefficients from the coefficient supplying means 67b at the timing shown in FIG. Convolvers 71, 72
The coefficients are alternately supplied to a RAM (not shown), and the coefficients are sequentially rewritten.

On the other hand, at this timing, the switch 69
Is a terminal a based on a control signal of the sub central processing unit 61.
Side. In the sub central processing unit 61, sound source information and audio conversion information are detected from the audio data, and the information is converted into a MIDI signal by MIDI conversion means 64, and a MIDI sound source 66 based on the MIDI signal. , And a monaural audio signal corresponding to the selected sound source is supplied to each of the convolvers 71 and 72 via the terminal a. Each convolver 7
In steps 1 and 72, this audio signal is convolution-processed on the time axis by the coefficient supplied from the coefficient supply means 67b, and its output is converted into digital / analog converters 73 and 74.
After being converted into an analog signal at each of the output terminals 76 and 77
Is output to the speaker. Thereby, the listener can be synchronized with the movement of the moving image according to the operation of the controller 55 from the pair of speakers, and the sound image is localized so that the sound source is located at a desired position different from the position of the speaker. Is performed, and a very realistic sound is heard.

When the game cassette 50 is mounted in place of the game CD-ROM 53 and the controller 55 is operated, the central processing unit 51 is operated in the same manner as described above. A signal, video display data, audio data and sound image position information are respectively supplied. The graphic system processor 54 generates an image signal that forms a mosaic image based on the video display data and outputs the image signal to the television monitor 60. In the synthesizer 57, for example, a sound source serving as a sound effect is selected based on the audio data. The selected audio signal is output to output terminal 76,
After being added to the sound image localized sound signal from 77 by an adder (not shown), it is output. Also, in the graphic system processor 54, as described above,
A synchronization signal is generated in accordance with the image signal and supplied to the sub central processing unit 61.

On the other hand, audio data and sound image position information are also supplied to the sub central processing unit 61.
Among them, audio data is supplied to the PCM sound source 62. The PCM sound source 62 selects a sound source based on the data, temporarily stores the signal in a predetermined position of the sound source RAM 63 in the next stage, reads out the signal again, and supplies it to the switch 69 as a monaural sound signal. In this case,
The switch 69 is switched to the terminal b. On the other hand, the sound image localization information is supplied to the third central processing unit 67 at the same timing as described above (the timing shown in FIG. 2D), and a predetermined coefficient is calculated based on this information.
It is taken out from the OM 68 and supplied to the convolvers 71 and 72 at the timing shown in FIG. Then, a convolution operation is performed on this audio signal.

FIG. 3 is a schematic block diagram of a game sound image localization control device according to a second embodiment, which will be described.
In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different points will be described. In the case of this embodiment, sound image localization information is directly supplied from the sub central processing unit 61 to the third central processing unit 67, and based on this, predetermined coefficients are extracted from the storage means 68 by the coefficient extracting means 67a. And
These coefficients are simultaneously supplied to each convolver 7 by a synchronizing signal from the MIDI sound source 66 in the coefficient supply means 67b.
Transfer is started at 1,72. In this case, the sound image localization determination information does not necessarily have to be related to the frame synchronization or the vertical synchronization of the video image. For example, the supply of coefficients is started by a synchronization signal having a timing as shown in FIG. Good.

According to this embodiment, since the coefficients are simultaneously transferred from the coefficient supply means 67b, there is not much time difference between the transfer of the coefficients as compared with the first embodiment. In addition, in this simultaneous transfer, since the transfer to the convolvers 71 and 72 is performed gradually for a short time, the characteristics gradually change until the transfer of all the coefficients is completed, and there is an advantage that noise and the like hardly occur. is there. Also,
The convolvers 71 and 72 may be provided with a gain setting unit to adjust the gain.

FIG. 4 is a schematic block diagram of a game sound image localization control apparatus according to the third embodiment. In this embodiment, the coefficient supply means 67 in the second embodiment (FIG. 3) is used.
b, a coefficient selection control means 91 is provided in place of the coefficient extraction means 67a, and the necessary number of coefficients are stored in the ROM.
The coefficient banks 92 prepared in the convolvers 71 and 72
And the respective coefficients can be switched by the coefficient selection control means 91. According to this embodiment, the coefficient supply means 67b becomes unnecessary, the circuit scale can be reduced, and the apparatus can be kept low in price. Also,
In the above-described embodiment, the coefficients are taken out of the ROM 58 for each block and supplied to the RAM in each convolver so that they are sequentially rewritten. Therefore, it takes time to change the coefficients and a time delay occurs. In the case of the example, it is sufficient to simply switch the coefficients in the coefficient bank, and there is no such delay, and a sound suitable for the motion of the image is obtained.

FIG. 5 is a schematic block diagram of a game sound image localization control device according to a fourth embodiment. In any of the above embodiments, a single sound image is assumed, and a pair of left and right convolvers is provided. In this embodiment, two sound images are added by adding another pair of convolvers. It is assumed. This embodiment has a configuration corresponding to the second embodiment (FIG. 3). That is, the MIDI sound source 6
6 and the connection from the PCM sound source 62 to the switch 69 are divided into two systems, and a second convolver 93,
94 are provided correspondingly. With this configuration, the two sound images can be located at positions different from the positions of the speakers and over 180 degrees or more in accordance with the respective sound images. If there is a fight scene by the fighter, the sound image of each fighter can be localized in synchronization with the video scene.

FIG. 6 is a schematic block diagram of a game sound image localization control device according to the fifth embodiment. This embodiment also assumes two sound images as in the third embodiment described above.
Switch 69 from MIDI sound source 66 and PCM sound source 62
Are connected in two systems, and second convolvers 93 and 94 are provided corresponding to the added sound image, respectively, and correspond to the third embodiment (FIG. 4). . According to this embodiment, the two sound images can be located at 180 degrees or more by making them different from the position of the speaker. Of course, the coefficient bank 92 is provided, and the coefficient supply means is eliminated. Because of the configuration
Accordingly, the circuit scale can be reduced and the price can be reduced. Further, since the coefficients in the coefficient bank 92 are switched at the same time, it is possible to provide a sound image localization suitable for the movement of the video image without causing a time delay in the coefficient switching. In the fourth and fifth embodiments, two sound images are assumed. However, the present invention can be applied to more cases.

FIG. 7 is a schematic block diagram of a sound image localization control device for a game according to a sixth embodiment. In this embodiment, in addition to the pair of convolvers 71 and 72 in the first embodiment (FIG. 1), convolvers 82 and 83 are connected in parallel. Further, the output side is connected to fade means 80, 81 and adders 84, 85, respectively. FIG. 8 is a timing waveform chart of the cross-fade processing. The timing waveforms (A) to (E) are the same timing waveforms as (A) to (E) in FIG. For example, it is assumed that the state moves from the state where the sound image localization processing is performed at the position of 60 degrees to the state where the sound image processing is performed at the position of 90 degrees. In that case, the first convolver 71,
Reference numeral 72 denotes an operation in which a coefficient for 60 degrees is supplied, and the other convolvers 82 and 83 are inactive. In this state, when the sound image constant information from the position of 60 degrees to 90 degrees enters the third central processing unit 67, the corresponding coefficient is read from the ROM 68 by the coefficient extracting means 67a, and the second coefficient is supplied from the coefficient supplying means 67b to the second central processing unit 67. And the cross-fade control signals shown in FIGS. 8F and 8G are supplied to the fade means 80 and 81. Then, in response to the crossfade control signal, the fade means 8
0 and 81 operate as shown in FIGS.
The outputs of the convolvers 71 and 72 are faded out,
The outputs to the second convolvers 82 and 83 are faded in and switched from the first convolvers 71 and 72 to the second convolvers 82 and 83 while cross-fading over a TX time. According to this embodiment, the convolver 71,
Crossfading can be performed in a relatively long time irrespective of the transfer of the coefficients to 72, 82, and 83, and there is an advantage that noise and the like hardly occur. Further, the coefficient supply means 67
The cross-fade control signal supplied from b to the fade means 80, 81 may indicate start.

FIG. 9 is a schematic block diagram of a game sound image localization control device according to the seventh embodiment. In this embodiment, in addition to the pair of convolvers 71, 72 in the third embodiment (FIG. 4), a plurality of pairs of each convolver 82 ₁ ,
83 ₁ is connected to a respective parallel, to their output side, in which fading means 80, 81 and an adder 84 and 85 has a respective connected. Even in such a configuration, crossfading can be performed in the same manner as in the above embodiment. In addition, in these embodiments, the fade means 80,
Although 81 is provided on the output side of each convolver, it may be provided on the input side.

Further, in each of the above-described embodiments, the case where the transfer of the coefficient is completed within the blanking period of the field or frame synchronization signal has been described. As shown in G), even when the blanking period is over several milliseconds, the synchronization relationship is substantially maintained and does not cause any trouble. .

[0056]

As described in detail above, according to the sound image localization apparatus for a video game according to the present invention, so-called (virtual realization) virtual reality can be enjoyed because the image and the sound image can be switched in synchronization. Also, since sound image localization can be finely performed by using audio data of a single channel, audio data of two channels is unnecessary, and in the case of sound movement, audio data for each sound image localization is unnecessary. Data makes that possible.

Further, by controlling between the image processing and the separated sound image localization information, the synchronization is assured and there is no deviation, so that the operation of the game is not hindered.

Further, since the signal from the sound source is processed on the time axis by the convolver to localize the sound image, a circuit for actually processing the sound image requires a convolution operation processing circuit on the time axis. And the circuit scale is very small and inexpensive.

Further, according to the sound image localization apparatus for a video game according to the second to fifth aspects, the switching of the coefficient by the synchronization means is performed during the so-called idle time of the vertical blanking period, so that the switching is quickly and stably performed. be able to.

In particular, according to the sound image localization device for a video game according to the third aspect, since the coefficients are transferred to the convolver, the characteristics gradually change until the transfer of all the coefficients is completed, and noise and the like are changed. Is less likely to occur.

In particular, according to the sound image localization apparatus for a video game according to the fourth aspect, the coefficient supply means becomes unnecessary.
The circuit scale can be reduced and the equipment can be kept low in price.
Since the coefficient is selectively switched by the coefficient selection control means, there is no delay in supplying the coefficient to the convolver, and a sound suitable for the movement of the image can be obtained.

Further, according to the sound image localization apparatus for a video game according to the fifth aspect, a plurality of sound images can be switched in synchronization with an image, so that further virtual reality can be enjoyed.

Further, according to the sound image localization apparatus for a video game according to the sixth aspect, since a plurality of pairs of convolvers are provided and they are switched while cross-fading them over a predetermined time, noise is less likely to occur. The advantage is obtained.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram showing a first embodiment of a sound image localization control device for a television game according to the present invention.

FIG. 2 is a waveform chart for explaining synchronization timing.

FIG. 3 is a schematic block diagram showing a second embodiment.

FIG. 4 is a schematic block diagram showing a third embodiment.

FIG. 5 is a schematic block diagram showing a fourth embodiment.

FIG. 6 is a schematic block diagram showing a fifth embodiment.

FIG. 7 is a schematic block diagram showing a sixth embodiment.

FIG. 8 is a schematic block diagram showing a seventh embodiment.

FIG. 9 is a waveform diagram for explaining a cross-fade process.

FIG. 10 is a configuration diagram for explaining the principle of sound image localization control.

FIG. 11 is a chart showing steps of a method of sound image localization control of the sound image localization control device for a video game of the present invention.

FIG. 12 is a configuration diagram showing an HRTF (head-related transfer function) measurement system.

FIG. 13 is a basic configuration diagram of a sound image localization control device.

FIG. 14 is a diagram for explaining points of HRTF measurement.

FIG. 15 is a diagram illustrating an example of calculating a cancel filter.

[Explanation of symbols]

Reference Signs List 50 Game cassette 51 Central processing unit (CPU) 52 Interface (IF) 53 CD-ROM 54 Graphic system processor (GS)
P) 55 controller 57 synthesizer 60 television monitor 61 sub central processing unit (SUB CPU) 62 PCM sound source 63 sound source RAM 64 MIDI conversion means 66 MIDI sound source 67 third central processing unit 67a coefficient extraction means 67b coefficient supply means 68 storage means (ROM) 69 Switch (SW) 71, 72 Convolver 73, 74 Analog / digital converter (D / A) 80, 81 Fade means 91 Coefficient selection control means 92 Coefficient bank 101 Step for measuring head related transfer function (HRTF) 102 Step of calculating impulse response of HRTF 103 Step of shaping IR (impulse response) 104 Step of calculating cancel filters cfLx and cfRx 105 Step of scaling cancel filter 106 Step of convoluting and reproducing a signal from a sound source sp1, sp2 speakers h1L, h1R Head transfer characteristics from speaker sp1 to listener's left and right ears h2L, h2R Head transfer characteristics from speaker sp2 to listener's left and right ears pLx , PRx Head transfer characteristics to the listener's left and right ears when an actual speaker is placed at the intended localization position x cfLx, cfRx Cancellation filter (convolver) and its coefficient DM Dummy head (or head) M Listener ( Game operator, listener) X sound source x target sound image localization position

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ identification code FI H04S 7/00 G10K 15/00 M (58) Field surveyed (Int.Cl. ⁶ , DB name) A63F 9/22 G10K 15 / 00 H04R 1/40 310 H04S 1/00 H04S 5/00 H04S 7/00

Claims (7)

(57) [Claims] A predetermined image is reproduced based on incoming video display data, and a sound image localization process is performed.
Means to perform sound image processing on the supplied monaural sound source.
To the listener, a pair of transducers
In a sound image localization control device for a video game for causing a sound image to be localized at an arbitrary position different from these transducers, a plurality of localizations to be supplied to the sound image localization processing means are set in advance.
Multiple head-based transfer functions based on measured head-related transfer functions
A coefficient holding unit for holding a coefficient, and the sound image localization processing unit, wherein the input of the monaural sound source is performed.
The monaural sound source
To perform convolution operation processing and perform sound image processing
A pair of convolvers, control means for generating localization information of a sound image based on operation of a game controller, and a frame image subsequent to the generation of the localization information, wherein the sound image localization processing means is synchronized with the frame image. A sound image localization control device for a video game, comprising: synchronous control means for controlling the sound image localization. 2. A video game system for supplying video display data, audio data, and sound image localization information and reproducing predetermined images and sounds, and supplied from a monaural sound source. The supplied sound is supplied to a pair of convolvers, and the listener is synchronized with the image to make the pair of transducers feel that the sound image is located at a position different from the positions of these transducers. A sound image localization control device for a video game, comprising: a coefficient holding unit for holding a coefficient prepared for changing a localization position of the sound image; and a coefficient corresponding to the supplied sound image localization information. A coefficient extracting means for extracting from the coefficient holding means, a synchronizing means for supplying the sound image localization information to the coefficient extracting means during a vertical synchronous blanking period of the image, A sound image localization control device for a video game, comprising: a coefficient supply unit for supplying a coefficient extracted by the coefficient extraction unit to the pair of convolvers. 3. A video game system which receives video display data, audio data and sound image localization information and reproduces predetermined images and sounds, and which is supplied from a monaural sound source. The supplied sound is supplied to a pair of convolvers, and the listener is synchronized with the image to make the pair of transducers feel that the sound image is located at a position different from the positions of these transducers. A sound image localization control device for a video game, comprising: a coefficient holding unit that holds a coefficient prepared for changing a localization position of the sound image; and a coefficient corresponding to the sound image localization information in response to the sound image localization information. Coefficient extracting means for extracting the coefficient from the coefficient holding means; coefficient supplying means for supplying the coefficient extracted by the coefficient extracting means to the pair of convolvers; Stage to sound image localization control apparatus for a video game, characterized in that a, a synchronization means for instructing to start the supply to the pair of convolver the coefficient in the vertical synchronization blanking period of the image. 4. A video game system for supplying video display data, audio data, and sound image localization information and reproducing predetermined images and sounds, and supplied from a monaural sound source. The supplied sound is supplied to a pair of convolvers, and the listener is synchronized with the image to make the pair of transducers feel that the sound image is located at a position different from the positions of these transducers. A sound image localization control device for a video game, comprising: a coefficient holding unit that holds a coefficient prepared for changing a localization position of the sound image; and a coefficient corresponding to the sound image localization information in response to the sound image localization information. Coefficient selection control means for selectively switching and extracting, and an output for outputting the output of the coefficient selection control means to the coefficient holding means during a vertical synchronization blanking period of the image. A sound image localization control device for a video game, comprising: a synchronization unit. In the sound image localization control apparatus 5. The method of claim 1, 2, 3 or 4 for the description of the video game, only a plurality of sets setting a pair of convolver, monaural sound source convolver each of these
A sound image localization control device for a video game, characterized in that it is supplied to a personal computer. 6. The sound image localization control for a video game according to claim 5, further comprising means for performing a crossfade process on the input or output side of the plurality of sets of convolvers. apparatus. 7. The sound image localization control device for a video game according to claim 2, 3 or 4, wherein switching control for switching the sound input of the pair of convolvers substantially in synchronization with the coefficient supply means or the coefficient selection control means. A sound image localization control device for a video game, characterized by comprising means.