JP6127792B2 - Sound reproduction apparatus and sound reproduction program - Google Patents

Description

  The present disclosure relates to a sound reproduction device and a sound reproduction program.

  When sound is reproduced with headphones or the like, sound source localization processing using a head related transfer function (HRTF) corresponding to the left and right ears of the person is performed, so that the sound source has a predetermined direction with respect to the person. There is a technology that makes you feel that

  The sound image localization processing is, for example, by listening to the sound accompanying the movement with a delay corresponding to the virtual distance between the person and the object together with the movement of the object included in the video representing the three-dimensional space. Application to a technique for providing a high-quality video has been proposed (see, for example, Patent Document 1).

JP-A-9-160549

  By the way, the technique of Patent Document 1 gives a sense of distance to an object by a difference between a timing at which a person sees the movement of an object represented by a realistic image and a timing at which a person hears the sound accompanying the movement of the object. Let people perceive. For this reason, for example, when an object outside the three-dimensional space represented by a video presented to a person is a sound source, that is, when a sound source or an image thereof is not included in the person's field of view, It is difficult for a person to perceive a sense of distance to a sound source by the technique 1.

  It is an object of the present disclosure to provide a sound reproduction device and a sound reproduction program that allow a person to intuitively recognize the distance to a sound source regardless of whether the sound field is included in the person's field of view.

  According to one aspect, the sound reproducing device receives an acoustic signal indicating a sound emitted from a sound source, and gives a sound reproduced from the received acoustic signal to a person's hearing, and a sound intensity indicated by the acoustic signal. The distance between the sound source and the person is calculated based on the presentation unit that gives the person's vision a stimulus having a strength that changes according to the change of the height, the information indicating the position of the sound source, and the information indicating the position of the person. The time difference between the first calculation unit and the first timing at which the sound is given to the person's hearing by the reproduction unit and the second timing at which the presentation unit gives the stimulus to the person's vision depends on the distance calculated by the first calculation unit And a control unit that performs control to be changed.

  According to another aspect, the sound reproduction program receives an acoustic signal indicating the sound emitted from the sound source, determines the intensity of the stimulus to be presented to the human vision based on the intensity of the sound indicated by the acoustic signal, Based on the information indicating the position of the sound source and the information indicating the position of the person, the distance between the sound source and the person is calculated, and the first timing for giving the sound reproduced from the acoustic signal to the person's hearing is determined as the first timing. The computer is caused to execute a process of performing control to change the time difference from the second timing for giving the stimulus to the person's vision according to the calculated distance.

  The sound reproduction device and the sound reproduction program of the present disclosure can make a person intuitively recognize the distance to a sound source regardless of whether or not the sound field is included in the person's field of view.

It is a figure which shows one Embodiment of a sound reproducing device. It is a figure which shows the example of the visual stimulus given by the presentation part shown in FIG. It is a figure which shows the example of the positional relationship of the sound source shown in FIG. 1, and a person. It is a figure which shows the relationship between the change of the intensity | strength of the sound emitted from the sound source shown in FIG. 3, the change of the brightness | luminance of the figure etc. which are displayed as a visual stimulus, and the change of the intensity | strength of the sound reproduced | regenerated. It is a figure which shows operation | movement of the sound reproduction apparatus shown in FIG. It is a figure which shows another embodiment of an audio | voice reproduction apparatus. It is a figure which shows the example of the delay time set by the delay part shown in FIG. It is a figure which shows another embodiment of the presentation part shown in FIG. It is a figure which shows the example of the guidance system to which the sound reproduction apparatus shown in FIG. 6 is applied. It is a figure which shows an example of the hardware constitutions of a sound reproduction apparatus. It is a figure which shows operation | movement of the sound reproduction apparatus shown in FIG. It is a figure which shows the example of the process which discriminates the sound source which attracts attention. It is a figure which shows the example of the process which gives a sound and a visual stimulus.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 shows an embodiment of a sound reproducing device.

  The sound reproduction device 10 illustrated in FIG. 1 includes, for example, a presentation unit 11 having glasses-type display screens 111a and 111b, a reproduction unit 12 such as headphones, a first calculation unit 13, and a control unit 14. Yes.

  The presentation unit 11 is attached to the head of the person P1, and gives the image displayed on the display screens 111a and 111b to the vision of the person P1. The presentation unit 11 is not limited to the glasses type having the display screens 111a and 111b corresponding to the left and right eyes, and may have a single display screen that covers both eyes. It is desirable that the display screens 111a and 111b have such transparency that the external environment can be observed through the display screens 111a and 111b. In addition, the presentation unit 11 may include at least one light emitting element arranged in the field of view of the person P1 instead of the display screens 111a and 111b. In addition, the example of the presentation part 11 containing a light emitting element is later mentioned using FIG.

  In addition, the playback unit 12 has built-in speakers facing the left and right ears of the person P1 while being worn on the head of the person P1, and the sound output from the built-in speakers is used as the hearing of the person P1. give.

  The sound source S shown in FIG. 1 is installed in a place separated from the person P1 by a wall WL or the like, or in a place further away from the person P1 than the distance at which the person P1 can hear the sound emitted from the sound source S. Yes.

  The sound emitted from the sound source S is converted into an acoustic signal by, for example, the microphone MC disposed at a position close to the sound source S. The acoustic signal obtained by the conversion by the microphone MC is, for example, a person P1 who is away from the sound source S by transmitting information indicating the acoustic signal using a wireless communication function installed in the server device SV. To the control unit 14 of the sound reproducing device 10 attached to the. That is, the sound reproducing device 10 reproduces the sound emitted by the sound source S located at a location away from the person P1 based on the acoustic signal obtained by the microphone MC, and gives the sound to the hearing of the person P1 by the reproducing unit 12. . Note that the server device SV uses the wireless communication function installed in the mobile terminal UE such as a smart phone or a tablet terminal carried by the person P1, thereby passing the acoustic signal to the mobile terminal UE. The sound signal may be passed to the sound reproduction device 10 via the route.

  Further, the first calculation unit 13 is obtained from, for example, a GPS (Global Positioning System) sensor Gp included in the mobile terminal UE from a mobile terminal UE such as a smart phone or a tablet terminal carried by the person P1. The position information Pdp indicating the position of the given person P1 is received. The position information Pdp indicating the position of the person P1 is not limited to the position information obtained by the GPS sensor Gp mounted on the mobile terminal UE. For example, the position information Pdp indicating the position of the person P1 may be position information obtained by tracking the movement of the person P1 based on information obtained by an acceleration sensor and a gyro sensor built in the mobile terminal UE. The position information Pdp indicating the position of the person P1 may be position information obtained by measuring the position of the person P1 based on the strength of signals received by the mobile terminal UE from a plurality of radio wave sources. The first calculation unit 13 receives, for example, another GPS mounted on the head of the person P1 together with the presentation unit 11 or the playback unit 12 instead of receiving the position information Pdp from the GPS sensor Gp mounted on the mobile terminal UE. You may receive the positional information obtained with the sensor etc.

  Moreover, the 1st calculation part 13 shows the position registered beforehand about the sound source S by receiving the information transmitted, for example from the server apparatus SV using the wireless communication function etc. which were mounted in the server apparatus SV. Receives position information Pds. The position information Pds may be held in advance in a memory provided in the first calculation unit 13.

  The first calculation unit 13 calculates a distance L1 between the person P1 and the sound source S based on the received position information Pdp and Pds, and passes information indicating the calculated distance L1 to the control unit 14.

  The control unit 14 calculates a sound pressure indicating the intensity of the sound emitted from the sound source S based on the acoustic signal received from the server device SV. For example, from the voltage value obtained by sampling the amplitude of the acoustic signal received from the microphone MC, the control unit 14 sets the magnitude of the sound pressure for each period set to several milliseconds to several tens of milliseconds. The effective value of the indicated amplitude is calculated. For example, the effective value V calculated by the control unit 14 is N (N is a positive integer) voltage values V (j) (j is the number of samples N or less) obtained by sampling within the averaging period. (Positive integer) is used to express the equation (1).

また、制御部１４は、式(１)を用いて算出した実効値Ｖを用いて、提示部１１の表示画面１１１ａ，１１１ｂの少なくとも一方に、音響信号で示される音源Ｓからの音の強さに応じて明るさが変化する画像を表示させる画像データを生成する。制御部１４で生成された画像データに基づいて表示画面１１１ａ，１１１ｂの少なくとも一方に表示された画像は、音響信号で示される音の強さに応じた強度を持ち、人物Ｐ１の視覚に与えられる刺激の一例である。また、表示画面１１１ａ，１１１ｂの少なくとも一方に表示された画像の明るさは、人物Ｐ１の視覚に与える刺激の強度の一例である。以下の説明において、「視覚に与える刺激」は、視覚刺激と称される場合がある。

Moreover, the control part 14 uses the effective value V calculated using Formula (1), and the intensity | strength of the sound from the sound source S shown with an acoustic signal on at least one of the display screens 111a and 111b of the presentation part 11 is shown. Image data for displaying an image whose brightness changes in accordance with is generated. An image displayed on at least one of the display screens 111a and 111b based on the image data generated by the control unit 14 has an intensity corresponding to the intensity of the sound indicated by the acoustic signal, and is given to the vision of the person P1. It is an example of a stimulus. The brightness of the image displayed on at least one of the display screens 111a and 111b is an example of the intensity of the stimulus given to the vision of the person P1. In the following description, “stimulation given to vision” may be referred to as visual stimulation.

  For example, the control unit 14 changes the luminance value of image data representing a graphic such as a circle, a rectangle, or an arrow, a character string, or a combination of a character string and a graphic according to the effective value V, thereby providing the presentation unit 11. The intensity of the visual stimulus given by is changed. Further, the control unit 14 changes the intensity of the visual stimulus given to the person P1 by changing the density and size of the color of the figure or the like given to the person P1 as the visual stimulus by the presentation unit 11 according to the effective value V. May be. In the following, as an example of a method for controlling the intensity of the stimulus given to the vision of the person P1 by the presentation unit 11, the case where the control unit 14 controls the brightness of a graphic or the like displayed on the display screens 111a and 111b as a visual stimulus will be described. Is done.

  2A, 2B, and 2C show examples of stimuli given to the vision of the person P1 by the presentation unit 11 shown in FIG. 2 that are the same as those shown in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted. 2A, 2 </ b> B, and 2 </ b> C show examples of display screens 111 a and 111 b viewed from the person P <b> 1 wearing the presentation unit 11.

  FIG. 2A shows a graphic (for example, a visual stimulus corresponding to the sound source S shown in FIG. 1 on one of the two display screens 111a and 111b (in this example, the display screen 111b) included in the presentation unit 11. An example in which the circle 112c) is displayed by the control unit 14 shown in FIG.

  For example, the control unit 14 changes the luminance of the circle 112c illustrated in FIG. 2A in accordance with the change in the effective value V obtained by Expression (1). The luminance of the circle 112c is changed by changing the luminance value of the image data for displaying the circle 112c. Thereby, the control part 14 reflects the change of the intensity of the sound emitted from the sound source S in the change of the luminance of the circle 112c given to the person P1 as a visual stimulus.

  2B shows a character string “Fj” as a visual stimulus corresponding to the sound source S shown in FIG. 1 on one of the two display screens 111a and 111b (in this example, the display screen 111b) included in the presentation unit 11. An example in which a text 112t indicating "is displayed by the control unit 14 is shown.

  For example, the control unit 14 changes the luminance of the text 112t shown in FIG. 2B according to the change in the effective value V obtained by the equation (1). The luminance of the text 112t is changed by changing the luminance value of the image data for displaying the text 112t. Thereby, the control part 14 reflects the change of the intensity of the sound emitted from the sound source S in the change of the luminance of the text 112t given to the person P1 as a visual stimulus.

  FIG. 2C shows a character string and a graphic as a visual stimulus corresponding to the sound source S shown in FIG. 1 on one of the two display screens 111a and 111b (in this example, the display screen 111b) included in the presentation unit 11. An example is shown in which the control unit 14 displays a mark 112m combining the above.

  The mark 112m shown in FIG. 2C includes a text 112t representing the character string “Fj”, a background 112b of the text 112t, and a frame 112f surrounding the text 112t and the background 112b.

  The control unit 14 illustrated in FIG. 1 changes the luminance of any of the text 112t, the background 112b, and the frame 112f illustrated in FIG. 2C in accordance with the change in the effective value V obtained by Expression (1). Thus, the change in the intensity of the sound emitted from the sound source S is reflected in the brightness of the mark 112m. Further, the control unit 14 determines the brightness of two or more elements selected from the elements included in the mark 112m that is the visual stimulus shown in FIG. 2C according to the effective value V obtained by the expression (1). By changing, the change in the intensity of the sound emitted from the sound source S may be reflected in the brightness of the visual stimulus. The brightness of the text 112t, the background 112b, and the frame 112f, which are elements of the mark 112m, is changed by changing the brightness value of the image data that displays the text 112t, the background 112b, and the frame 112f. For example, the control unit 14 may generate image data representing each of the text 112t and the frame 112f using a luminance value that changes in accordance with the effective value V obtained by Expression (1).

Note that the control unit 14 desirably changes the brightness of the visual stimulus corresponding to the sound source S within a range that can be displayed by the presentation unit 11 in accordance with a change in the effective value V obtained by Expression (1). For example, the control unit 14 uses the effective value Vm corresponding to the maximum sound pressure assumed as the sound pressure of the sound emitted from the sound source S and the maximum luminance Bm that can be displayed by the presentation unit 11, and The luminance value B corresponding to the effective value V may be obtained from (2).
B = Bm × V / Vm (2)
Moreover, it is desirable that the control unit 14 display the visual stimulus corresponding to the sound source S on either of the two display screens 111a and 111b at a position that does not obstruct the view of the person P1.

  Further, the control unit 14 obtains the time for sound to propagate through the distance L1 based on the information indicating the distance L1 received from the first calculation unit 13. In addition, the control unit 14 gives a delay corresponding to the obtained time to the acoustic signal received from the server device SV, and passes the delayed acoustic signal to the reproduction unit 12. As a result, the control unit 14 delays the time during which the sound propagates the distance L1 from the timing at which the intensity of the visual stimulus given to the person P1 by the presentation unit 11 changes according to the sound intensity indicated by the acoustic signal. The sound indicated by the acoustic signal is played back by the playback unit 12 at the determined timing. In other words, the control unit 14 gives a larger delay time to the acoustic signal passed to the reproduction unit 12 as the distance L1 is larger, thereby causing the reproduction unit 12 to give a sound to the person P1 and the presentation unit 11 to the person P1. A large time difference from the second timing for applying the visual stimulus is set. In the control of delaying the acoustic signal and passing it to the playback unit 12, the first timing when the strength of the sound played back by the playback unit 12 changes and the brightness of the visual stimulus given to the person P1 by the presentation unit 11 is the strength of the sound. It is an example of the control which sets a time difference to the 2nd timing which changes according to the change of.

  Here, the stimulus given to the vision of the person P1 by the presentation unit 11 shown in FIG. 1 is not an image representing an object that becomes the sound source S but a stimulus having an intensity corresponding to the intensity of the sound emitted from the sound source S. . The intensity of the stimulus given to the visual of the person P1 by the presentation unit 11 is a change in the intensity of the same sound emitted from the sound source S together with the change in the intensity of the sound given to the hearing of the person P1 by the reproduction unit 12. Will change accordingly. For this reason, the visual stimulus given from the presentation unit 11 and the sound reproduced by the reproduction unit 12 are recognized in association with each other by the person P1. For example, a sound source that emits the reproduced sound as the time difference between the second timing at which the stimulus given to the visual of the person P1 changes and the first timing at which the sound given to the hearing of the person P1 changes is increased. The impression that the distance to S is large can be given to the person P1. That is, the sound reproducing device 10 shown in FIG. 1 uses a function of displaying an image representing an object serving as the sound source S by giving a stimulus to the visual and auditory stimuli of the person P1 with a time difference. Instead, the person P1 can recognize the distance L1 to the sound source S. That is, the sound reproducing device 10 illustrated in FIG. 1 supports the recognition of the distance L1 to the sound source S by the person P1 even when the image representing the object serving as the sound source S is not included in the field of view of the person P1. can do.

  Note that the method of setting the time difference between the first timing and the second timing by the control unit 14 is to delay the first timing with respect to the second timing by delaying the acoustic signal and passing it to the reproduction unit 12. It is not limited to the control. For example, the control unit 14 causes the presentation unit 11 to apply a stimulus having an intensity corresponding to the intensity of the sound indicated by the acoustic signal to the person P1 with respect to the first timing at which the reproduction unit 12 reproduces the sound from the acoustic signal. You may delay the 2nd timing given to vision.

  In addition to the movement of the person P1, the sound reproducing device 10 shown in FIG. 1 adjusts the time difference adjusted according to the change in the distance between the person P1 and the sound source S together with the sound from the sound source S indicated by the sound signal. The visual stimulus generated from the acoustic signal is given to the person P1. Therefore, the person P1 who moves while wearing the reproduction unit 12 and the presentation unit 11 shown in FIG. 1 has a time difference between the sound reproduced by the reproduction unit 12 and the stimulus given to the visual of the person P1 by the presentation unit 11. It is possible to recognize a change in the distance to the sound source S based on the change.

  FIG. 3 shows an example of the positional relationship between the sound source S and the person P1 shown in FIG. FIG. 3 shows a sound (for example, sound) emitted from the sound source S including the exhibit EX1 and the persons P2, P3, and P4 by the sound reproducing device 10 attached to the person P1 moving in the exhibition hall EH. An example of listening is shown.

  The sound source S shown in FIG. 3 is located at a position separated from the person P1 by the wall WL. That is, the positional relationship between the person P1 and the sound source S shown in FIG. 3 is an example of a positional relationship in which it is difficult for the person P1 to visually recognize the sound source S directly.

  In the example of FIG. 3, the acoustic signal indicating the sound from the sound source S is sent to the server device SV from the microphone MC disposed close to the exhibit EX1, and the server device SV further accesses the access point AP and the mobile phone. It is passed to the sound reproduction device 10 via the terminal UE. The server device SV generates encoded sound information indicating the sound emitted from the sound source S by encoding the acoustic signal received from the microphone MC, and carries the generated sound information via the access point AP. You may pass to the terminal UE. Further, when the mobile terminal UE receives the encoded sound information, the mobile terminal UE decodes the received sound information to restore the acoustic signal indicating the sound from the sound source S, and the restored sound signal is restored to the acoustic reproduction device 10. 1 is transferred to the control unit 14 shown in FIG.

  Based on the acoustic signal indicating the sound from the sound source S, the control unit 14 causes the presentation unit 11 illustrated in FIG. 1 to display a visual stimulus such as a graphic whose brightness changes corresponding to the sound source S, and the distance L1. The sound from the sound source S is reproduced by the reproducing unit 12 shown in FIG.

  FIG. 4 shows the relationship between the change in the intensity of the sound emitted from the sound source S shown in FIG. 3, the change in the luminance of the graphic displayed as a visual stimulus, and the change in the intensity of the reproduced sound. In FIG. 4, the horizontal axis t indicates the passage of time.

  FIG. 4A shows an example of a change in sound pressure of sound emitted from the sound source S. The vertical axis SP in FIG. 4A indicates sound pressure.

  FIG. 4B shows an example of a change in luminance of an image such as a graphic displayed on at least one of the display screens 111a and 111b of the presentation unit 11 shown in FIG. 1 as a visual stimulus corresponding to the sound source S. The vertical axis Br in FIG. 4B represents the luminance value of the image displayed as the visual stimulus corresponding to the sound source S.

  FIG. 4C shows an example of a change in sound pressure of the sound reproduced by the reproducing unit 12 shown in FIG. The vertical axis SP in FIG. 4C indicates the sound pressure. 4C is reproduced after a delay corresponding to the distance between the person P1 and the sound source S is added to the acoustic signal indicating the sound from the sound source S by the control unit 14 shown in FIG. The change of the sound pressure of the sound reproduced | regenerated from the acoustic signal passed to the part 12 is shown.

  For example, when the person P1 shown in FIG. 3 is separated from the sound source S by the distance L1 at the time T0 shown in FIGS. 4A, 4B, and 4C, the sound from the sound source S at the time T0. The change in sound pressure is reproduced with a delay of time DL1 corresponding to the distance L1, as shown in FIG. On the other hand, as can be seen from the comparison between FIGS. 4A and 4B, the luminance of the image displayed as the visual stimulus corresponding to the sound source S by the presentation unit 11 is the change in the sound pressure of the sound from the sound source S. It changes almost synchronously.

  Thereafter, the person P1 shown in FIG. 3 moves in the exhibition hall EH, and the position of the person P1 is indicated by a broken line in FIG. 3 at time T1 shown in FIGS. 4 (A), (B), and (C). When it changes to position P1 ', the timing of the sound reproduced | regenerated by the reproducing part 12 shown in FIG. 1 changes as follows. When the distance L1 ′ between the position P1 ′ and the sound source S is shorter than the distance L1, the change in the sound pressure of the sound from the sound source S at time T1 is shorter than the time DL1, as shown in FIG. Playback is delayed by time DL1 '.

The control unit 14 illustrated in FIG. 1 is set between the first timing and the second timing by using the distance L1 from the sound source S to the person P1 and the speed of sound in the air, using Equation (3), for example. You may obtain | require the time difference to do. In Equation (3), the symbol DL indicates the time difference between the first timing and the second timing, and the symbol γs indicates the speed of sound in the air. Moreover, in Formula (3), code | symbol (alpha) shows the coefficient set to 1 or more real numbers, for example.
DL = α × L1 / γs (3)
For example, the control unit 14 passes the acoustic signal obtained by delaying the time difference DL calculated using Expression (3) including the coefficient α to the reproduction unit 12. Thereby, the time difference between the 1st timing which gives a stimulus to hearing, and the 2nd timing which gives a stimulus to vision can be emphasized more than the difference of distance L1 and distance L1 '. For example, the time difference DL set between the first timing and the second timing by the control unit 14 according to the change from the distance L1 between the person P1 and the sound source S to the distance L1 ′ is the distance L1, L1 ′. The amount of change can be changed larger than the time for sound to propagate. As a result, the person P1 is based on the change of the time difference DL between the visual stimulus and the auditory stimulus compared to the case where the time difference DL is obtained by setting the value of the coefficient α to the numerical value “1” in Equation (3). Thus, it becomes easy to recognize a change in the distance to the sound source S. Note that the coefficient α included in the expression (3) is set to a value at which the value of the time difference DL is about several seconds when the distance L1 from the sound source S to the person P1 is an assumed maximum value, for example. It is desirable. For example, when the person P1 moves inside the exhibition hall EH shown in FIG. 3, the coefficient α indicates the upper limit of the time difference to a value obtained by dividing the sound speed γs by the length of the rectangular diagonal line indicating the exhibition hall EH. A value obtained by multiplying time (for example, 2 seconds) is set.

  FIG. 5 shows the operation of the sound reproducing device 10 shown in FIG. The process of step S301 to step S305 shown in FIG. 5 shows the operation of the sound reproduction device 10 shown in FIG. 1 and an example of the sound reproduction program. For example, the process shown in FIG. 5 is realized by a processor mounted in the sound reproduction device 10 executing a sound reproduction program. Note that the processing shown in FIG. 5 may be executed by hardware mounted on the sound reproduction device 10. That is, FIG. 5 shows an embodiment of a sound reproduction program.

  In step S301, the control unit 14 shown in FIG. 1 receives an acoustic signal indicating a sound emitted from the sound source S shown in FIG. The control unit 14 may receive the acoustic signal generated by the microphone MC shown in FIG. 1 from the server device SV, or may receive it via the server device SV and the mobile terminal UE as shown in FIG. Good. When the microphone MC has a wireless communication function, the sound signal may be passed to the sound reproduction device 10 by transmitting the sound signal to the sound reproduction device 10 by the wireless communication function of the microphone MC.

  In step S302, the control unit 14 determines the intensity of the visual stimulus to be given to the person P1 based on the received acoustic signal according to the intensity of the sound indicated by the acoustic signal. For example, as described with reference to FIGS. 2A, 2 </ b> B, and 2 </ b> C, the control unit 14 brightness of at least a part of an image representing a graphic, a character, or a mark that combines a graphic and a character. Is set according to the sound intensity indicated by the acoustic signal.

  In step S303, the first calculation unit 13 shown in FIG. 1 receives the position information Pdp indicating the position of the person P1 and the position information Pds indicating the position of the sound source S, and based on the received position information Pds and Pdp, The distance between the person P1 and the sound source S is calculated.

  In step S304, based on the calculated distance, the control unit 14 reproduces the sound indicated by the acoustic signal by the reproduction unit 12 with a time difference having a magnitude proportional to the distance, and the presentation unit 11 reproduces the acoustic signal. A visual stimulus corresponding to the sound indicated by is given.

  In step S305, the control unit 14 ends the sound reproduction process that is a process of reproducing the sound indicated by the sound signal based on the sound signal indicating the sound emitted from the sound source S by, for example, the operation of the person P1. Whether or not is instructed is determined.

  When the end of the sound reproduction process is not instructed (No determination in step S305 (NO)), the control unit 14 returns to the process of step S301, and for the sound signal newly generated from the sound emitted from the sound source S, Steps S301 to S305 are repeated.

  On the other hand, when the end of the sound reproduction process is instructed (Yes determination in step S305 (YES)), the control unit 14 ends the process of reproducing the sound.

  The sound reproducing device 10 shown in FIG. 1 desirably executes the processing of steps S301 to S305 shown in FIG. 5 repeatedly every several milliseconds to several tens of milliseconds, for example. As a result, the sound reproducing device 10 reflects the change in the distance to the sound source S accompanying the movement of the person P1 in the change in the time difference between the first timing that gives the stimulus to the auditory sense and the second timing that gives the stimulus to the visual sense. Thus, it is possible to support the grasp of the distance to the sound source S by the person P1.

  Note that the processing in steps S301 and S302 and the processing in step S303 shown in FIG. 5 may be executed in the reverse order of the flowchart shown in FIG. Moreover, the process of step S301 and step S302 and the process of step S303 may be performed in parallel.

  FIG. 6 shows another embodiment of the audio reproduction device 10. 6 that are equivalent to the components shown in FIG. 1 are denoted by the same reference numerals, and the description thereof may be omitted.

  The sound reproduction device 10 illustrated in FIG. 6 includes a memory 131 and a second calculation unit 15 in addition to the presentation unit 11, the reproduction unit 12, the first calculation unit 13, and the control unit 14 illustrated in FIG. .

  For example, the memory 131 receives position information Pds indicating a position registered in advance with respect to the sound source S in the server apparatus SV from the server apparatus SV outside the sound reproducing apparatus 10, and holds the received position information Pds. Further, the memory 131 passes the held position information Pds to both the first calculation unit 13 and the second calculation unit 15.

  The second calculation unit 15 receives position information Pdp indicating the position of the person P1 from a portable terminal UE or the like carried by the person P1 and, for example, from a gyro sensor Gyr attached to the head of the person P1 The angle information Pda indicating the posture of the head of P1 is received. In addition, the second calculation unit 15 calculates the direction of the sound source S based on the orientation of the head of the person P1 based on the received position information Pds, position information Pdp, and angle information Pda. For example, the second calculation unit 15 uses a line segment connecting the position of the sound source S indicated by the position information Pds and the position of the person P1 indicated by the position information Pdp, and the orientation of the head of the person P1 indicated by the angle information Pda. The angle formed by and is calculated as the direction of the sound source S.

  The angle information Pda passed to the second calculation unit 15 is not limited to information detected by the gyro sensor Gyr, and may be information indicating the posture of the head of the person P1. For example, the second calculation unit 15 may receive an angular velocity indicating a change in the orientation of the head of the person P1. When the angular velocity indicating the change in the orientation of the head of the person P1 is received as the angle information Pda, the second calculation unit 15 performs a process of calculating the orientation of the head of the person P1 from the received angular velocity, and the calculated head Based on the direction of the part, the direction of the sound source S with respect to the direction of the head of the person P1 is calculated.

  The control unit 14 illustrated in FIG. 6 includes a generation unit 141, a delay unit 142, a conversion unit 143, a determination unit 144, and an adjustment unit 145. The acoustic signal generated from the sound emitted from the sound source S by the microphone MC is passed to both the generation unit 141 and the delay unit 142 via the wireless communication function mounted on the server device SV. The distance between the person P1 and the sound source S calculated by the first calculation unit 13 is passed to the adjustment unit 145. Further, the direction of the sound source S calculated by the second calculation unit 15 is passed to the generation unit 141, the determination unit 144, and the conversion unit 143.

  The generation unit 141 uses the expression (1) for the voltage value obtained by sampling the acoustic signal received from the microphone MC, so that the generation unit 141 has a predetermined period set to several milliseconds to several tens of milliseconds. Then, an effective value indicating the intensity of the sound indicated by the acoustic signal is obtained. In addition, based on the calculated effective value, the generation unit 141 calculates, for example, a luminance value indicating the strength of the stimulus given to the vision of the person P1 by Expression (2). Further, the generation unit 141 displays the graphics shown in FIGS. 2A, 2B, and 2C on the display screens 111a and 111b as visual stimuli given to the visual of the person P1 by the presentation unit 11 as visual stimuli. The calculated luminance value is set as the luminance value of the image data for displaying.

  In addition, the generation unit 141 determines, for example, whether the sound source S is on the right side or the left side of the person P1 based on the information indicating the direction of the sound source S passed from the second calculation unit 15, and determines the determination result. In response, a graphic or the like that becomes a visual stimulus may be displayed on either of the display screens 111a and 111b. For example, when the generation unit 141 determines that the sound source S is on the right side of the person P1, the generation unit 141 generates image data that displays a graphic or the like that serves as a visual stimulus on the display screen 111a. On the other hand, when determining that the sound source S is on the left side of the person P1, the generation unit 141 generates image data for displaying a graphic or the like serving as a visual stimulus on the display screen 111b.

  Further, the generation unit 141 may display a graphic or the like indicating a visual stimulus at a position on the display screen 111a or 111b corresponding to the angle indicating the direction of the sound source S based on the information indicating the direction of the sound source S. For example, the generation unit 141 associates a range of a predetermined angle measured counterclockwise from the front of the head of the person P1 with a length in the left-right direction of the display screen 111a, and a range of the predetermined angle designed clockwise. May be associated with the length in the left-right direction of the display screen 111b. Then, the generation unit 141 generates image data that displays a graphic or the like that becomes a visual stimulus at any position on the display screens 111 a and 111 b corresponding to the angle calculated by the second calculation unit 15.

  The image data generated by the generation unit 141 is transferred to the presentation unit 11 and used for displaying graphics, marks, and the like on the display screens 111a and 111b included in the presentation unit 11. The figures, marks, etc. displayed on the display screens 111a, 111b are given to the vision of the person P1 as stimuli from the direction corresponding to the positions where the figures, marks, etc. are displayed. Therefore, the person P1 can recognize the approximate direction of the sound source S based on the positions of figures and marks displayed in the field of view.

  For example, the determination unit 144 pays attention to the sound source S when the direction of the sound source S calculated by the second calculation unit 15 indicates a direction indicating the front of the head of the person P1 for a predetermined period or longer. It is determined that Here, the direction of the sound source S obtained by the second calculation unit 15 is information that reflects a change in the orientation of the head of the person P1, and is an example of information that indicates the movement of the person P1. That is, the determination unit 144 determines whether the person P1 is paying attention to the sound source S based on the information indicating the movement of the person P1.

  The adjustment unit 145 performs adjustment to shorten the distance between the sound source S calculated by the first calculation unit 13 and the person P1 when the determination unit 144 determines that the person P1 is paying attention to the sound source S. Then, the adjusted distance is passed to the delay unit 142.

For example, the adjustment unit 145 uses the equation (4) based on the distance between the sound source S and the person P1 and the time when the determination result indicating that the person P1 is paying attention to the sound source S is obtained from the determination unit 144. ) To calculate the adjusted distance. In Expression (4), the symbol L1a indicates the adjusted distance between the sound source S and the person P1, and the symbol L1 indicates the distance calculated by the first calculation unit 13. Further, in Expression (4), the symbol tf indicates the length of the period in which the person P1 is paying attention to the sound source S, that is, the attention time. In the equation (4), the symbol R1 is a coefficient indicating the magnitude of the tendency of the distance L1a adjusted according to the attention time tf to be shortened. In the formula (4), the function max (A, B) is a function for selecting the larger one of the numbers A and B.
L1a = max (0, L1-tf × R1) (4)
Further, the adjustment unit 145 may use Expression (5) instead of Expression (4) for calculation of the adjusted distance L1a. In addition, in Formula (5), the element equivalent to Formula (4) is shown with the same code | symbol as Formula (4). Further, in the equation (5), the symbol R2 is a coefficient indicating the time until the adjusted distance L1a becomes 0.
L1a = max (0, L1 (1-tf / R2)) (5)
On the other hand, when the determination unit 144 determines that the person P1 is not paying attention to the sound source S, the adjustment unit 145 determines the distance between the sound source S calculated by the first calculation unit 13 and the person P1 as it is as a delay unit. 142.

  The delay unit 142 uses the expression (3), for example, based on the distance between the pre-adjustment or post-adjustment sound source S passed from the adjustment unit 145 and the person P1, and based on the acoustic signal, The time difference between the first timing for reproducing the sound and the second timing for applying the visual stimulus is calculated. In addition, the delay unit 142 gives a delay corresponding to the calculated time difference to the acoustic signal delivered from the server device SV, and passes the delayed acoustic signal to the conversion unit 143.

  For example, the conversion unit 143 performs conversion processing using the head-related transfer function corresponding to the direction of the sound source S obtained by the second calculation unit 15 for the delayed acoustic signal received from the delay unit 142. To do. In other words, the conversion unit 143 convolves the head-related transfer function corresponding to the direction of the sound source S with respect to the head direction of the person P1 into the delayed acoustic signal, so that the sound source S viewed from the person P1. Sound image localization processing in the direction.

  In addition, the conversion unit 143 passes the sound signal after the sound image localization process to the reproduction unit 12, and causes the reproduction unit 12 to reproduce sound from the sound signal after the sound image localization process, thereby causing the sound source S to be viewed from the person P1. The person P1 listens to the sound that has been localized.

  The control unit 14 shown in FIG. 6 uses the generation unit 141 to generate image data that displays a graphic or mark that becomes a visual stimulus at a position corresponding to the direction of the sound source S with respect to the head direction of the person P1. Then, the conversion unit 143 performs sound image localization processing in the direction of the sound source S viewed from the person P1. Then, the control unit 14 causes the person P1 to listen to the sound image-localized sound by the reproduction unit 12, and causes the presentation unit 11 to display a graphic or mark that becomes a visual stimulus at a position according to the direction of the sound source S. That is, the control unit 14 having the generation unit 141 and the conversion unit 143 indicates the direction of the sound source S according to the position where the sound image localized sound and the graphic or the like serving as the visual stimulus are displayed, as well as visual stimulation and sound reproduction. The distance to the sound source S is indicated by the time difference between. Thereby, the sound reproducing device 10 including the control unit 14 illustrated in FIG. 6 intuitively determines the position of the sound source S by the person P1 regardless of whether or not the actual position of the sound source S is included in the field of view of the person P1. Can help you understand.

  In the control unit 14 illustrated in FIG. 6, the adjustment unit 145 determines the distance between the sound source S and the person P1 when the determination unit 144 determines that the person P1 is paying attention to the sound source S. The delay set by the delay unit 142 is shortened by making the distance shorter than the distance calculated by the 1 calculating unit 13.

  FIG. 7 shows an example of the delay time set by the delay unit 142 shown in FIG. FIG. 7A shows an example of a change in delay time set by the delay unit 142 when the adjustment unit 145 uses Equation (4) to calculate the adjusted distance. FIG. 7B shows an example of a change in the delay time set by the delay unit 142 when the adjustment unit 145 uses Equation (5) to calculate the adjusted distance. In FIGS. 7A and 7B, the horizontal axis t indicates time, and the vertical axis DL indicates the magnitude of the delay time.

  The delay time DL shown in FIG. 7A, that is, the delay time DL when the adjustment unit 145 uses the equation (4) for calculating the adjusted distance, pays attention to the sound source S by the person P1. It decreases monotonically from the time Ta when it is detected. Then, after time Ta + L1 / R1 when a time L1 / R1 indicated by a value obtained by dividing the distance L1 calculated by the first calculation unit 13 by the coefficient R1 shown in Equation (4) from the time Ta, the delay time is reached. The value of DL is 0.

  Similarly, the delay time DL shown in FIG. 7B, that is, the delay time DL when the adjustment unit 145 uses the equation (5) to calculate the adjusted distance, It decreases monotonously from the time Ta when it is detected that attention is paid. On the other hand, the delay time DL shown in FIG. 7B is after the time Ta + R2 when the time indicated by the coefficient R2 shown in the equation (5) has passed, regardless of the distance L1 calculated by the first calculation unit 13. The value is 0.

  The delay time set by the delay unit 143 is indicated by an acoustic signal by the reproduction unit 12 and the second timing when a stimulus having a strength corresponding to the sound indicated by the acoustic signal by the presentation unit 11 is given to the visual sense of the person P1. This corresponds to the time difference from the first timing at which the sound is reproduced. That is, the control unit 14 illustrated in FIG. 6 reduces the time difference between the second timing and the first timing when the person P1 is paying attention to the sound source S as compared with the case where the person P1 is not paying attention, The person P1 can easily concentrate on the sound from the sound source S indicated by the acoustic signal.

  The method for calculating the adjusted distance by the adjustment unit 145 is not limited to the method using the formula (4) or the formula (5), and the determination unit 144 has determined that the person P1 is paying attention to the sound source S. In this case, any method may be used as long as the distance calculated by the first calculation unit 13 is adjusted to be short. For example, the adjustment unit 145 may reduce the adjusted distance step by step each time the attention time increases by a predetermined value, or it is detected that the person P1 is paying attention to the sound source S. In this case, the adjusted distance may be set to 0 regardless of the length of the attention time.

  FIG. 8 shows another embodiment of the presentation unit 11 shown in FIG. FIG. 8A shows the presentation unit 11 worn on the head of the person P1. FIG. 8B shows a correspondence relationship between a direction based on the orientation of the head of the person P <b> 1 and the light emitting elements included in the presentation unit 11.

  In the presentation unit 11 shown in FIG. 8A, a plurality of light emitting elements 113-1, 113-2,..., 113-k (k is an integer of 2 or more) such as light emitting diodes For example, they are arranged at almost equal intervals. Further, the plate-like member 114 on which the light emitting elements 113-1,..., 113-k are arranged is held in front of the face of the person P1 by a vine-like member 115 extending from the reproducing unit 12, for example.

  The presentation unit 11 illustrated in FIG. 8B includes seven light emitting elements 113-1, 113-2, 113-3, 113-4, 113-5, 113-6, and 113-7. That is, FIG. 8B illustrates a case where k = 7 in the presentation unit 11 illustrated in FIG. Each of the seven light emitting elements 113-1 to 113-7 has seven ranges φ1, φ2, and φ3 obtained by dividing a range of a predetermined angle with respect to the front of the head of the person P1, for example, into seven equal parts. , Φ4, φ5, φ6, and φ7. In the following description, the k light emitting elements 113-1 to 113-k may be simply referred to as the light emitting element 113 in some cases.

  In the example of FIG. 8B, the range φ1 is a range from minus 70 degrees to minus 50 degrees in an angle coordinate in which the front of the head of the person P1 is an angle 0 and the clockwise angle is a positive value. It is. Similarly, in the example of FIG. 8B, the range φ2 is a range from −50 degrees to −30 degrees, and the range φ3 is a range from −30 degrees to −10 degrees. In the example of FIG. 8B, the range φ4 is a range from minus 10 degrees to plus 10 degrees, and the range φ5 is a range from plus 10 degrees to plus 30 degrees. In the example of FIG. 8B, the range φ6 is a range from plus 30 degrees to plus 50 degrees, and the range φ7 is a range from plus 50 degrees to plus 70 degrees.

  Note that the correspondence relationship between each of the light emitting elements 113 included in the presentation unit 11 and the range of angles based on the front of the head of the person P1 is not limited to the example in FIG. 8B. For example, the range of angles associated with each light emitting element 113 may be larger than the range of angles indicated by each of the ranges φ1 to φ7, or conversely, the range of angles indicated by each of the ranges φ1 to φ7. A smaller range may be used.

  When the presentation unit 11 shown in FIG. 8B is used, the generation unit 141 of the control unit 14 shown in FIG. 6 uses the direction of the sound source S calculated by the second calculation unit 15 as shown in FIG. A range including the calculated direction of the sound source S is determined from the ranges φ1 to φ7 shown in B). In addition, the generation unit 141 generates a drive signal for causing the light emitting element 113 to emit the light amount indicated by the luminance value calculated using the expression (2), and for the light emitting element 113 associated with the determined range. The generated drive signal is supplied. For example, the generation unit 141 causes the light emitting element 113-5 illustrated in FIG. 8 to selectively emit light at a luminance according to the intensity of the sound indicated by the acoustic signal, so that the direction of the sound source S with respect to the person P1. Can be shown to be a direction of 10 degrees to 30 degrees with respect to the head direction of the person P1.

  The sound reproducing device 10 described above is useful, for example, in a guidance system that guides a user to a pre-designated exhibition position in an exhibition hall or the like.

  FIG. 9 shows an example of a guidance system to which the sound reproducing device 10 shown in FIG. 6 is applied. The guidance system GS shown in FIG. 9 has the function of the sound reproduction device 10 for guiding the positions of a plurality of exhibitions designated in advance by the person P1 to the person P1 who is a user at the exhibition hall EH. Use.

  The guidance system GS includes, for example, the position information indicating the position of each exhibit displayed at the exhibition hall EH, and the information indicating the contents of the exhibition, the sound reproducing device 10, the portable terminal UE carried by the person P1, the server device SV, and the like. And an exhibition database DB for storing information. The server device SV is connected to the exhibition database DB and can refer to information stored in the exhibition database DB. In addition, the server device SV and the mobile terminal UE are connected via, for example, the network NW and the access point AP provided in the exhibition hall EH, and can exchange information with each other. Further, the mobile terminal UE has a function of passing information received from the server device SV to the sound reproduction device 10.

  In the exhibition hall EH, the exhibitions Ex1-1, Ex1-2, Ex1-3, and Ex1-4 are arranged along the wall WL1. In the exhibition hall EH, the exhibits Ex2-1, Ex2-2, Ex2-3, and Ex2-4 are arranged along the wall WL2.

  The exhibition database DB includes information indicating the positions of the exhibitions Ex1-1 to Ex1-4 and the exhibitions Ex2-1 to Ex2-4, as well as a sound for explaining the exhibition contents as part of the information indicating the exhibition contents. The voice information that represents is stored.

  The guidance system GS shown in FIG. 9 passes the acoustic signal generated based on the audio information stored in the exhibition database DB to the acoustic reproduction device 10 instead of the acoustic signal representing the sound emitted from each exhibition. Then, the sound reproducing device 10 gives the sound reproduced from the received sound signal to the hearing of the person P1, and gives a stimulus having an intensity corresponding to the intensity of the sound indicated by the sound signal to the vision of the person P1.

  For example, when the exhibition Ex1-3 and the exhibition Ex2-4 are designated in advance by the person P1 as guidance targets, the sound reproducing device 10 corresponds to the exhibition Ex1-3 and the exhibition Ex2-4 in the exhibition database DB. The stored voice information and position information are received from the server device SV.

  For example, the sound reproducing device 10 controls sound image localization processing for sound reproduced from the sound information received from the server device SV based on the position information of the exhibition Ex1-3, and supports sound reproduced from the sound information. Controls the position to display the graphic showing the visual stimulus. As a result, the sound reproducing device 10 determines the sound image corresponding to the sound reproduced from the sound information received from the server device SV in the direction in which the sound image is localized and the position of the light emitting element 113 that emits light as a visual stimulus for the person P1. The direction of the exhibition Ex1-3 can be shown to the person P1. Further, by providing a time difference between the timing of reproducing the sound from the audio information received from the server device SV and the timing of applying the visual stimulus corresponding to the sound, the person P1 can be shown the distance L1 to the exhibition Ex1-3. it can.

  Similarly, the sound reproduction device 10 controls the sound image localization processing for sound reproduced from the sound information received from the server device SV based on the position information of the exhibition Ex2-4, and converts the sound reproduced from the sound information. Controls the position to display a figure showing the corresponding visual stimulus. As a result, the sound reproducing device 10 can determine the sound image corresponding to the sound reproduced from the sound information received from the server device SV by the direction in which the sound image is localized and the position of the light emitting element 113 that is caused to emit light as a visual stimulus for the person P1. The direction of the exhibition Ex2-4 can be shown to the person P1. Further, by providing a time difference between the timing of reproducing the sound from the audio information received from the server device SV and the timing of applying the visual stimulus corresponding to the sound, the person P1 can be shown the distance L2 to the exhibition Ex2-4. it can.

  That is, based on the sound and the stimuli given to the vision that are reproduced by the sound reproducing device 10 corresponding to each of the exhibition Ex1-3 and the exhibition Ex2-4, the person P1 has the exhibition Ex1-3 and the exhibition Ex2-4. Each position can be grasped intuitively. In other words, the person P1 has approximately the exhibition Ex1-3 and Ex2-4 even though both the exhibition Ex1-3 and the exhibition Ex2-4 are separated from the person P1 by another exhibition Ex1-4 and the wall WL2. And the approximate distance to each exhibition Ex1-3, Ex2-4. For example, for the person P1, the difference between the timing at which the brightness of the visual stimulus indicating the direction of the exhibition Ex2-4 changes and the timing at which the corresponding sound intensity changes is smaller than the deviation corresponding to the exhibition Ex1-3. From this, it can be known that the exhibition Ex2-4 is closer.

  Note that the presentation unit 11 illustrated in FIG. 8B can be realized using, for example, a plurality of light emitting elements, and has a function of displaying high-definition images of the exhibitions Ex1-3 and Ex2-4 serving as sound sources. Compared with the head-mounted display apparatus etc. which have, hardware is small. Further, even when an image representing a figure, a mark, or the like shown in FIG. 2 is given to the person P1 as a visual stimulus, the resolution is lower than that of a head-mounted display device having a function of displaying a high-definition image of an object serving as a sound source. The display screens 111a and 111b can be used. That is, the guidance system to which the sound reproducing device 10 shown in FIG. 9 is applied has a display device for presenting a realistic image of a designated exhibition (for example, exhibition Ex1-3, Ex2-4) to the person P1. Compared with the case where it uses, it is realizable at low cost.

  The sound reproduction device 10 of the present disclosure described above can be realized by using a computer device including a mobile terminal such as a smart phone or a tablet terminal, for example.

  FIG. 10 shows an example of a hardware configuration of the sound reproducing device 10. 10 that are equivalent to the components shown in FIGS. 1, 6, and 9 are denoted by the same reference numerals, and the description thereof may be omitted.

  The mobile terminal UE shown in FIG. 10 is a smart phone or a tablet-type terminal possessed by the person P1. The mobile terminal UE includes a processor 21, a memory 22, a GPS sensor Gp, a general-purpose interface 24, an audio processing unit 25, a network interface 26, a display control unit 27, a touch panel 28, and an input control unit 29. Contains. The processor 21, the memory 22, the GPS sensor Gp, the general-purpose interface 24, the voice processing unit 25, the network interface 26, the display control unit 27, the touch panel 28, and the input control unit 29 illustrated in FIG. Are connected to each other via a bus.

  The mobile terminal UE is connected to the network NW via the network interface 26, and is further connected to the server device SV via the network NW. The mobile terminal UE is connected to the presentation unit 11 including, for example, the light emitting elements 113-1 to 113-7 illustrated in FIG. 8B via the general-purpose interface 24. In addition, the audio processing unit 25 included in the mobile terminal UE is connected to, for example, a headphone-type playback unit 12 attached to the head of a person P1 who is a user of the mobile terminal UE. In addition, the presentation unit 11 illustrated in FIG. 10 is fixed to the ear pad included in the reproduction unit 12 with a vine-like member. The gyro sensor Gyr is fixed to an ear pad included in the playback unit 12. The angle information Pda detected by the gyro sensor Gyr is passed to the processor 21 via the general-purpose interface 24.

  The processor 21, the memory 22, the GPS sensor Gp, the general-purpose interface 24, the sound processing unit 25, and the network interface 26 included in the mobile terminal UE are included in the sound reproduction device 10.

  In addition, the display control unit 27 included in the mobile terminal UE performs control for displaying an image on the display screen included in the touch panel 28 in accordance with an instruction from the processor 21. The input control unit 29 has a function of passing an instruction input by the person P1 operating the touch panel 28 to the processor 21. Therefore, the person P1 operates the touch panel 28 to perform, for example, an instruction for designating a sound source to be presented to the sound reproducing device 10 or a process of giving sound and visual stimulus corresponding to the designated sound source. An instruction to start or end can be input. For example, the person P1 displays the floor plan in the exhibition hall EH shown in FIG. 9 on the display screen of the touch panel 28, and touches an icon or the like indicating a desired exhibition included in the floor plan, so that the sound reproduction device 10 Specify as a sound source to play sound.

  The memory 22 shown in FIG. 10 stores an application program for the processor 21 to execute an acoustic reproduction process, which is a process for reproducing a sound emitted from a designated sound source, together with the operating system of the mobile terminal UE. Yes. Note that the application program for executing the sound reproduction process may be downloaded to the memory 22 by downloading from the server device SV or the like via the network interface 26 and the network NW, for example.

  The processor 21 performs the functions of the first calculation unit 13, the second calculation unit 15, and the control unit 14 illustrated in FIG. 6, for example, by executing an application program for sound reproduction processing. Further, the function of the memory 131 shown in FIG. 6 is realized by using a part of the storage area of the memory 22 shown in FIG.

  The exhibition database DB shown in FIG. 10 corresponds to the exhibition Ex1-1 to Ex1-4 and the exhibition Ex2-1 to Ex2-4 arranged in the exhibition hall EH shown in FIG. The position information indicating the voice information indicating the contents of the exhibition is stored. 10 can acquire audio information associated with a desired exhibition from the exhibition database DB via the network NW and the server apparatus SV, for example.

  FIG. 11 shows the operation of the sound reproducing device 10 shown in FIG. The process of step S301 to step S305 and step S311 to step S316 shown in FIG. 11 shows the operation of the sound reproduction device 10 shown in FIG. 10 and another embodiment of the sound reproduction program. The process of step S301 to step S305 and step S311 to step S316 illustrated in FIG. 11 is an example of the process included in the application program for the sound reproduction process. Further, the processing of step S301 to step S305 and step S311 to step S316 is executed by the processor 21 shown in FIG.

  In step S311, the processor 21 receives, for example, information indicating the sound source designated by the user's operation on the touch panel 28 from the input control unit 29 illustrated in FIG. For example, the processor 21 causes the display control unit 27 to display a floor plan or the like in the exhibition hall EH shown in FIG. 9 on the display screen of the touch panel 28 and touches an icon or the like indicating a desired exhibition to the user. Encourage operation. Further, when the processor 21 receives information indicating an icon touched by the user from the input control unit 29, the processor 21 displays the display associated with the icon indicated by the received information with the sound reproducing device 10. The sound source to be played back.

  In step S311, the processor 21 sets initial values “0” for the still time, the attention time, and the attention flag, which are variables used in the process of step S314.

  In step S312, the processor 21 receives position information indicating the position of an exhibition designated as a sound source to be reproduced by the sound reproducing apparatus 10 as position information indicating the position of the sound source. For example, the processor 21 inquires of the server apparatus SV of the location information stored in the exhibition database DB corresponding to the designated exhibition via the network interface 26 shown in FIG. Further, the processor 21 stores the position information obtained by the inquiry to the server device SV in the memory 22 corresponding to the exhibition designated as the sound source. When a plurality of exhibitions are specified as the sound source to be presented in the process of step S311, the processor 21 inquires the server device SV for position information for each exhibition, and the position information obtained by the inquiry. Are stored in the memory 22 corresponding to each exhibition. For example, when the exhibition Ex1-3 and the exhibition Ex2-4 shown in FIG. 9 are designated by the person P1, the processor 21 uses each of the exhibitions Ex1-3 and Ex2-4 as the position information of the designated sound source. The position information held in the exhibition database DB is acquired.

  In the example of FIG. 11, the processor 21 executes the processes of step S303 and steps S313 to S316 prior to the process of step S301. Further, the processor 21 performs the processing of the loop including step S303, step S313 to step S316, step S301, step S302, and step S304 shown in FIG. 11 for a predetermined time set to several milliseconds to tens of milliseconds. It is desirable to run at intervals.

  In step S303, the processor 21 calculates the distance between each of the exhibits that are sound sources and the person P1 based on the position information of each sound source received in the process of step S312. For example, the processor 21 first receives information indicating the position of the mobile terminal UE from the GPS sensor Gp illustrated in FIG. 10 as position information indicating the position of the person P1. The processor 21 also displays the person and the exhibition based on the position information of the person P1 received from the GPS sensor Gp and the position information of the exhibits Ex1-3 and Ex2-4 received as the position information of each sound source in the process of step S312. The distance between each of Ex1-3 and Ex2-4 is calculated.

  In step S313, the processor 21 acquires angle information Pda indicating the orientation of the head of the person P1 from the gyro sensor Gyr shown in FIG. 10, and determines the orientation of the head of the person P1 based on the acquired angle information Pda. As a reference, the direction of each sound source is calculated. For example, the processor 21 measures each from the front direction of the head of the person P1 based on the angle information acquired from the gyro sensor Gyr, the position information corresponding to each sound source, and the position information indicating the position of the person P1. An angle indicating the direction of the sound source is calculated.

  In step S <b> 314, the processor 21 determines a sound source that is noticed by the person P <b> 1 among the designated sound sources based on the information indicating the movement of the person P <b> 1 shown in FIG. 10. The processor 21 uses, for example, the angle information Pda acquired in the process of step S313 as information indicating the movement of the person P1 in the process of determining the sound source focused by the person P1. In addition, the processor 21 maintains the head direction of the person P1 indicated by the angle information Pda for a predetermined period or more, and when any of the designated sound sources is in front of the person P1, A sound source in the front direction is determined as a sound source that has received attention. For example, the processor 21 may determine the sound source that has been noticed by executing the processing of steps S321 to S330 shown in FIG.

  FIG. 12 shows an example of the process of determining the focused sound source shown in step S314 in FIG. The process of step S321 to step S330 illustrated in FIG. 12 is an example of a process of determining a focused sound source included in an application program for sound reproduction processing. Further, the processing in steps S321 to S330 is executed by the processor 21 shown in FIG.

  In step S321, the processor 21 acquires angle information Pda obtained in the past τ seconds by the gyro sensor Gyr shown in FIG. For example, the processor 21 holds the angle information Pda received from the gyro sensor Gyr via the general-purpose interface 24 in the memory 22 for a time τ set to about 1 second, and acquires the angle information Pda held in the memory 22. To do.

  In step S322, the processor 21 compares the angle information Pda acquired in the process of step S321 with each other, so that the head direction of the person P1 indicated by the angle information Pda is within the range indicated by the predetermined threshold value. Determine whether or not. For example, the processor 21 performs processing in step S322 as follows. For example, the head direction of the person P1 indicated by the angle information Pda for the past τ seconds is an angle corresponding to each of the ranges φ1 to φ7 illustrated in FIG. You may determine whether it is in the range (for example, 20 degree | times).

  If the change in the orientation of the head of the person P1 is within a range set to about 20 degrees, for example (Yes in step S322 (YES)), the processor 21 determines the orientation of the head of the person P1. It is determined that it is substantially constant, and the process proceeds to step S323.

  In step S323, the processor 21 repeats the processing of the loop from step S303 to step S305 shown in FIG. 11 during a stationary time that is a variable indicating the length of time that the head direction of the person P1 is constant. A predetermined value τ1 corresponding to the interval is added.

  In step S324, the processor 21 determines whether or not the value “1” indicating that there is a possibility that the person P1 is paying attention to any of the sound sources is set in the attention flag by the processing so far. To do.

  If the value “1” has not yet been set for the attention flag (No at Step S324), the processor 21 proceeds to the process at Step S325.

  In step S325, the processor 21 determines whether or not the value of the stationary time is equal to or greater than a predetermined threshold set to, for example, about several seconds.

  If the still time is still less than the threshold value (No determination in step S325 (NO)), the processor 21 proceeds to the process of step S327.

  In step S327, the processor 21 outputs information indicating that there is no sound source focused by the person P1. Thereafter, the processor 21 ends the process of determining the sound source of interest, and proceeds to the process of step 315 shown in FIG.

  Here, the processor 21 executes the processing of step S321 to step S329 shown in FIG. 12, for example, every few milliseconds to several tens of milliseconds as the processing of step S314 included in the loop of steps S303 to S305. To do. Accordingly, the stationary time is a predetermined value τ1 in the process of step S323 every time it is determined in step S322 that the head direction of the person P1 is substantially constant in the process in which the process of step S314 is repeated. It is added one by one.

  Then, in the process of step S323, when the stationary time to which the predetermined value τ1 is added becomes equal to or greater than the threshold set for the stationary time (Yes determination in step S325) (YES in step S325), the processor 21 proceeds to step S326. Proceed to processing.

  In step S326, the processor 21 sets a value “1” in the attention flag to indicate that the person P1 may be paying attention to any of the sound sources. After the process of step S326 is completed, the processor 21 proceeds to the process of step S327, and outputs information indicating that there is no sound source focused by the person P1. Thereafter, the processor 21 ends the process of determining the sound source of interest, and proceeds to the process of step 315 shown in FIG.

  After the value “1” is set in the attention flag by the process of step S326, the processor 21 proceeds to the process of step S328 according to the affirmative determination route (YES) of step S324 in the process of performing the process of step S314 again. move on.

  In step S328, the processor 21 determines whether or not a direction indicating the front of the head of the person P1 has been calculated for any of the designated sound sources by the process of step S313 illustrated in FIG. For example, the processor 21 determines whether the direction of each sound source calculated in the process of step S313 illustrated in FIG. 11 is included in the range φ4 illustrated in FIG. Processing may be performed. Here, the range φ4 shown in FIG. 8B corresponds to the front of the head of the person P1.

  When none of the directions calculated for each of the designated sound sources is a direction indicating the front of the head of the person P1 (No determination in step S328 (NO)), the processor 21 proceeds to the process of step S327. . Then, in step S327, the processor 21 outputs the information indicating that there is no sound source that has been noticed by the person P1, and then ends the process of determining the sound source that has been noticed. In step 315 shown in FIG. Proceed to processing.

  On the other hand, for example, when the direction calculated in the process of step S313 for the exhibition Ex1-3 illustrated in FIG. 9 indicates the front of the head of the person P1 (Yes determination in step S328 (YES)), the processor 21 The process proceeds to step S329.

  In step S329, the processor 21 determines the sound source (for example, exhibition Ex1-3) determined to be in the direction hitting the front of the head of the person P1 in the process of step S328 as the sound source attracting attention by the person P1. . Further, the processor 21 adds the predetermined value τ1 that is the same as the predetermined value τ1 added to the stationary time in step S323 to the attention time that is a variable indicating the time during which the person P1 is paying attention to any of the sound sources. Thereafter, the processor 21 ends the process of determining the sound source of interest, and proceeds to the process of step S315 illustrated in FIG.

  By the way, when the head direction of the person P1 changes more than a range set to, for example, about 20 degrees (No determination in step S322 (NO)), the processor 21 proceeds to the process of step S330.

  In step S330, the processor 21 sets initial values “0” for the stationary time and the attention time, and clears the attention flag. After that, in step S327, the processor 21 outputs information indicating that there is no sound source that has been noticed by the person P1, and then ends the process of determining the sound source that has been noticed. In step 315 shown in FIG. Proceed to processing.

  By executing the processing of steps S321 to S330 described above, the processor 21 has the head orientation of the person P1 constant over the threshold value for the stationary time, and the sound source in front of the head. If there is, it is determined that the sound source is receiving attention. That is, the processor 21 fulfills the function of the determination unit 144 illustrated in FIG. 6 by executing the processes of steps S321 to S330 described above.

  In step S315 illustrated in FIG. 11, the processor 21 determines whether or not the sound source focused by the person P1 is determined in the process of step S314.

  For example, in step S329 shown in FIG. 12, when the exhibition Ex1-3 is determined as a sound source attracting attention by the person P1, the processor 21 proceeds to step S316 according to the affirmative determination route (YES) of step S315.

  In step S316, the processor 21 adjusts the distance to the sound source attracting attention by the person P1 using the formula (4) or the formula (5). For example, when the exhibition Ex1-3 shown in FIG. 9 is determined as the sound source of interest, the processor 21 determines the distance between the exhibition Ex1-3 and the person P1 in the process of step S313 using the formula ( By adjustment using 4) or equation (5), the distance is shorter than the original distance L1. Thereafter, the processor 21 proceeds to the process of step S301. That is, the processor 21 performs the function of the adjustment unit 145 illustrated in FIG. 6 by executing the process of step S316.

  On the other hand, if the sound source focused by the person P1 is not determined by the process of step S314 (No determination of step S315 (NO)), the processor 21 performs the process of step S301 without performing the process of step S316. Proceed to

  In step S301, the processor 21 stores the audio information stored in the exhibition database DB corresponding to each of the exhibits designated as sound sources from the server device SV as an acoustic signal indicating the sound emitted from each designated sound source. Receive a part of. For example, the processor 21 divides the audio information stored in the exhibition database DB by a length that can be reproduced in several milliseconds to tens of milliseconds, and sequentially executes the divided audio information every time the process of step S301 is executed. To receive. When the audio information stored in the exhibition database DB is encoded audio information, for example, the processor 21 determines whether an audio signal indicating an audio before encoding or an audio signal from the received audio information. Information indicating a change in amplitude is acquired. The processor 21 may use the function of the sound processing unit 25 illustrated in FIG. 10 when acquiring the sound signal indicating the sound before encoding or the information indicating the change in the amplitude of the sound signal. In addition, the processor 21 uses the acquired acoustic signal or information indicating a change in the amplitude of the acoustic signal in the processing of step S302 and subsequent steps.

  In step S302, the processor 21 determines the intensity of the stimulus given to the vision of the person P1 by the presentation unit 11 illustrated in FIG. 10 based on each of the acoustic signals received in the process of step S301. For example, the processor 21 uses the information indicating the change in the amplitude of each acoustic signal and the expressions (1) and (2) to calculate a luminance value corresponding to the intensity of the sound indicated by the acoustic signal. The obtained luminance value is set as the intensity of the visual stimulus. That is, the processor 21 performs the function of the generation unit 141 illustrated in FIG. 6 by executing the process of step S302.

  In step S304, the processor 21 is indicated by the sound and the sound reproduced from the sound signal acquired corresponding to each sound source in the process of step S301 with the time difference according to the distance between the person P1 and each sound source. A stimulus having an intensity corresponding to the intensity of the sound is given to the person P1. The processor 21 applies the time difference corresponding to the distance adjusted in the process of step S316 to the sound source indicated as being noticed by the person P1 in the process of step S314 among the designated sound sources. On the other hand, the processor 21 applies a time difference according to the distance calculated in the process of step S303 for sound sources other than the sound source indicated to be noticed by the person P1. For example, the processor 21 controls the presentation of the visual stimulus by the presentation unit 11 and the reproduction of the sound by the reproduction unit 12 illustrated in FIG. 10 by executing the processes of steps S331 to S335 illustrated in FIG.

  FIG. 13 shows an example of the process for giving the sound and visual stimulus shown in step S304 in FIG. The process of step S331 to step S335 shown in FIG. 13 is an example of a process for providing sound and visual stimulus included in the application program for the sound reproduction process. Further, the processing in steps S331 to S335 is executed by the processor 21 shown in FIG.

  In step S331, the processor 21 uses, for example, the equation (3) to calculate the distance calculated in the process of step S303 shown in FIG. 11 or the distance adjusted in the process of step S316 shown in FIG. The time difference according to the distance is calculated. For example, when it is determined that the person P1 is paying attention to the exhibition Ex1-3 illustrated in FIG. 9, the processor 21 calculates the time difference represented by Expression (3) based on the distance adjusted in the process of step S316. Ask for. For this reason, the time difference between the 1st timing which reproduces the sound signal acquired corresponding to exhibition Ex1-3, and the 2nd timing which gives the visual stimulus corresponding to the sound signal concerned is distance L1 shown in FIG. The time difference is shorter than the time difference corresponding to. On the other hand, the processor 21 uses the distance obtained in the process of step S303 for the exhibition Ex2-4, and corresponds to the first timing for reproducing the acoustic signal acquired corresponding to the exhibition Ex2-4 and the acoustic signal. A time difference from the second timing for applying the visual stimulus is obtained.

  In step S332, based on the time difference calculated in step S331, the processor 21 reproduces the sound corresponding to each piece of audio information acquired in step S301, and the second timing for giving the visual stimulus corresponding to the sound. Set the timing. For example, the processor 21 sets the first timing and the second timing for the audio information acquired corresponding to the exhibition Ex1-3 shown in FIG. 9 and the audio information acquired corresponding to the exhibition Ex2-4, respectively. Set. For example, the processor 21 delays the timing delayed by the time difference calculated in the process of step S323 with respect to the second timing that gives the visual stimulus whose intensity is determined according to the intensity of the sound by the process of step S302. It is set as the first timing for reproducing sound.

  That is, the processor 21 performs the functions of the delay unit 142 illustrated in FIG. 6 by executing the processes of steps S331 and S332.

  In step S333, the processor 21 performs a sound image localization process based on the direction of each sound source obtained in step S313 of FIG. 11, thereby obtaining a sound image in the direction of the sound source with reference to the head direction of the person P1. An acoustic signal representing the localized sound is generated. That is, the processor 21 performs sound image localization processing in the direction of the exhibition Ex1-3 calculated in step S313 on the acoustic signal generated from the audio information acquired corresponding to the exhibition Ex1-3. Similarly, the processor 21 performs sound image localization processing in the direction of the exhibition Ex2-4 calculated in step S313 on the acoustic signal generated from the audio information acquired corresponding to the exhibition Ex2-4. The processor 21 may use a function installed in the audio processing unit 25 shown in FIG. 10 when executing the sound image localization process.

  That is, the processor 21 performs the function of the conversion unit 143 illustrated in FIG. 6 by executing the process of step S333.

  In step S334, the processor 21 selects a sound source from among the light emitting elements 113-1 to 113-7 shown in FIG. 8B based on the directions of the sound sources obtained in the process of step S313 in FIG. The light emitting element 113 that emits light as the corresponding visual stimulus is selected. As described with reference to FIG. 8, the processor 21 selects the light emitting element 113 associated with the range including the direction of the sound source obtained in step S <b> 313. For example, the processor 21 emits light corresponding to the range φ4 shown in FIG. 8B when the angle indicating the front of the head of the person P1 is obtained as the direction of the exhibition Ex1-3 shown in FIG. The element 113-4 is selected corresponding to the exhibition Ex1-3. On the other hand, when the processor 21 obtains an angle included in the range φ2 shown in FIG. 8B with respect to the front of the head of the person P1, for example, as the direction of the exhibition Ex2-4 shown in FIG. In addition, the light emitting element 113-2 corresponding to the range φ2 is selected corresponding to the exhibition Ex2-4.

  In step S335, the processor 21 causes the reproduction unit 12 illustrated in FIG. 10 to reproduce sound at the first timing set for each sound source in the process of step S332, and presents it at the second timing set for each sound source. The unit 11 is caused to output a visual stimulus of the person P1. For example, when the time indicated by the internal clock indicates the first timing set for the exhibition Ex1-3, the processor 21 reproduces the acoustic signal generated from the audio information acquired corresponding to the exhibition Ex1-3. Playback by the unit 12 is started. Further, the processor 21 causes the light emitting element 113 selected corresponding to the exhibition Ex1-3 to emit light having the luminance determined corresponding to the exhibition Ex1-3 at the second timing set for the exhibition Ex1-3. By supplying a signal, a stimulus is given to the vision of the person P1. Similarly, for example, when the time indicates the first timing set for the exhibition Ex2-4, the processor 21 uses the acoustic signal reproducing unit 12 generated from the audio information acquired corresponding to the exhibition Ex2-4. Start playback. Further, the processor 21 causes the light emitting element 113 selected corresponding to the exhibition Ex2-4 to emit light having the luminance determined corresponding to the exhibition Ex2-4 at the second timing set for the exhibition Ex2-4. By supplying a signal, a stimulus is given to the vision of the person P1.

  The processor 21 proceeds to the process of step S305 illustrated in FIG. 11 after the processes of step S331 to step S335 are completed.

  In step S305, the processor 21 determines whether or not to end the reproduction of the sound and the presentation of the visual stimulus corresponding to the sound.

  When the instruction indicating that the reproduction of the sound and the presentation of the visual stimulus corresponding to the sound are not yet received has been received (No determination in step S305 (NO)), the processor 21 returns to the process of step S303, The processing for the newly received sound signal corresponding to the sound source is started. For example, the processor 21 desirably repeats the processing in steps S303 to S305 every time (for example, several milliseconds to several tens of milliseconds) for reproducing one segment of the audio information acquired in the processing in step S301. .

  On the other hand, when receiving an instruction from the input control unit 29 shown in FIG. 10 to end the reproduction of the sound and the presentation of the visual stimulus corresponding to the sound (Yes in step S305 (YES)), the processor 21 Ends the processing of the application program for the sound reproduction process.

  As described with reference to FIGS. 9 to 13, for example, the sound reproducing device 10 illustrated in FIG. 10 reproduces the explanation sound corresponding to the exhibitions Ex1-3 and Ex2-4 and performs visual stimulation corresponding to the explanation sound. By setting a time difference to the presentation, the person P1 intuitively perceives the difference in distance. 10 applies a sound image localization process to the sound to be reproduced by the reproduction unit 12, and turns on the light emitting element selected according to the direction of the sound source, thereby causing the person P1 to turn on. On the other hand, the directions of the exhibits Ex1-3 and Ex2-4 designated as sound sources can be shown. That is, the sound reproducing device 10 illustrated in FIG. 10 can make the person P1 intuitively grasp the positions of the plurality of exhibitions Ex1-3 and Ex2-4 in the exhibition hall EH and the like.

  Here, the sound reproducing device 10 illustrated in FIG. 10 indicates the distance to the sound source from the person P1 by the time difference between the timing at which the sound is reproduced and the timing at which the intensity of the visual stimulus changes according to the intensity of the sound. . Therefore, the sound reproducing device 10 shown in FIG. 10 has the exhibits Ex1-3 and Ex2 as sound sources regardless of whether or not the views Ex1-3 and Ex2-4 as sound sources are included in the field of view of the person P1. Intuitive grasping by the person P1 at a distance up to -4 can be supported. This characteristic of the sound reproducing device 10 shown in FIG. 10 is useful in application to a system or the like for guiding a person P1 to a desired display position in a large exhibition hall or the like.

  Furthermore, the sound reproducing device 10 shown in FIG. 10 determines the sound source that the person P1 is paying attention to, and calculates the time difference between the visual stimulus and the auditory stimulus generated from the acoustic signal corresponding to the determined sound source. Make adjustments to shorten. That is, the sound reproducing device 10 shown in FIG. 10 reproduces the sound reproduced from the sound signal corresponding to the sound source focused on by the person P1 as the sound from the sound source closer to the actual sound source. To do. Then, the time difference between the visual stimulus and the auditory stimulus generated from the acoustic signal corresponding to the sound source focused on by the person P1 is shortened, so that the person P1 receives the sound from the focused sound source. It becomes possible to concentrate on listening.

  From the above detailed description, features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments described above without departing from the spirit and scope of the right. Further, any improvement and change should be easily conceived by those having ordinary knowledge in the technical field. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate improvements and equivalents included in the scope disclosed in the embodiments can be used.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
A playback unit that receives an acoustic signal indicating a sound emitted from a sound source and gives a sound reproduced from the received acoustic signal to a person's hearing;
A presentation unit that gives the person's vision a stimulus having an intensity that changes in accordance with a change in sound intensity indicated by the acoustic signal;
A first calculation unit that calculates a distance between the sound source and the person based on information indicating the position of the sound source and information indicating the position of the person;
A time difference between a first timing at which sound is given to the hearing of the person by the reproducing unit and a second timing at which the stimulus is given to the visual of the person by the presentation unit is set to a distance calculated by the first calculating unit. A control unit that performs control to change according to the
A sound reproducing device comprising:
(Appendix 2)
In the sound reproduction device according to attachment 1,
The controller is
A determination unit that determines whether the person is paying attention to the sound source based on information indicating the movement of the person;
When the determination unit determines that the person is paying attention to the sound source, control is performed to make the time difference smaller than a value set according to a distance between the sound source and the person. A sound reproducing device.
(Appendix 3)
In the sound reproduction device according to attachment 2,
The controller is
The sound reproducing apparatus is characterized in that control is performed to reduce the time difference as the time during which the person focuses on the sound source is longer.
(Appendix 4)
In the sound reproduction device according to any one of appendix 1 to appendix 3,
Based on information indicating the position of the sound source and information indicating the position of the person, a second calculation unit for determining the direction of the sound source with respect to the position of the person,
The presenting unit gives the stimulus to the visual sense of the person from the direction of the sound source obtained by the second calculation unit,
The sound reproduction device, wherein the reproduction unit gives sound reproduced from the sound signal localized in the direction of the sound source obtained by the second calculation unit to the hearing of the person.
(Appendix 5)
In the sound reproduction device according to any one of appendix 1 to appendix 4,
The controller is
The sound reproduction apparatus according to claim 1, wherein the time difference is set in a direction in which the first timing is delayed from the second timing.
(Appendix 6)
In the sound reproduction device according to attachment 1,
The control unit sets the time difference between the first timing and the second timing to be greater than the time for sound to propagate through the distance calculated by the first calculation unit. apparatus.
(Appendix 7)
In the sound reproduction device according to any one of supplementary notes 1 to 6,
The sound reproducing device, wherein the presenting unit changes the luminance of the stimulus given to the person's vision according to a change in the intensity of the sound.
(Appendix 8)
Receive an acoustic signal indicating the sound emitted from the sound source,
Based on the intensity of the sound indicated by the acoustic signal, determine the intensity of the stimulus given to the human vision,
Based on the information indicating the position of the sound source and the information indicating the position of the person, the distance between the sound source and the person is calculated,
A time difference between a first timing at which a sound reproduced from the acoustic signal is given to the auditory sense of the person and a second timing at which a stimulus having the determined intensity is given to the visual sense of the person is defined as the calculated distance. Control to change accordingly,
A sound reproduction program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 10 ... Sound reproduction apparatus; 11 ... Presentation part; 12 ... Reproduction part; 13 ... 1st calculation part; 14 ... Control part; 15 ... 2nd calculation part; 111a, 111b ... Display screen; 112m ... mark; 112b ... background; 112f ... frame; 113 ... light emitting element; 114 ... plate-like member; 115 ... vine-like member; 141 ... generating part; 142 ... delaying part; 143 ... converting part; 145: Adjustment unit; 21 ... Processor; 22 ... Memory; 24 ... General-purpose interface; 25 ... Audio processing unit; 26 ... Network interface; 27 ... Display control unit; 28 ... Touch panel; 29 ... Input control unit; UE ... portable terminal; Gp ... GPS sensor; S ... sound source; MC ... microphone; SV ... server apparatus; AP ... access point; Ex1, Ex1-1, Ex1-2 Ex1-3, Ex1-4, Ex2-1, Ex2-2, Ex2-3, Ex2-4 ... exhibitions; DB ... exhibition database; NW ... network; GS ... guidance system

Claims (5)

A playback unit that receives an acoustic signal indicating a sound emitted from a sound source and gives a sound reproduced from the received acoustic signal to a person's hearing;
A presentation unit that gives the person's vision a stimulus having an intensity that changes in accordance with a change in sound intensity indicated by the acoustic signal;
A first calculation unit that calculates a distance between the sound source and the person based on information indicating the position of the sound source and information indicating the position of the person;
A time difference between a first timing at which sound is given to the hearing of the person by the reproducing unit and a second timing at which the stimulus is given to the visual of the person by the presentation unit is set to a distance calculated by the first calculating unit. A control unit that performs control to change according to the
A sound reproducing apparatus comprising: The sound reproducing device according to claim 1,
The controller is
A determination unit that determines whether the person is paying attention to the sound source based on information indicating the movement of the person;
When the determination unit determines that the person is paying attention to the sound source, control is performed to make the time difference smaller than a value set according to a distance between the sound source and the person. A sound reproducing device. The sound reproducing device according to claim 1 or 2,
Based on information indicating the position of the sound source and information indicating the position of the person, a second calculation unit for determining the direction of the sound source with respect to the position of the person,
The presenting unit gives the stimulus to the visual sense of the person from the direction of the sound source obtained by the second calculation unit,
The sound reproduction device, wherein the reproduction unit gives sound reproduced from the sound signal localized in the direction of the sound source obtained by the second calculation unit to the hearing of the person. The sound reproducing device according to claim 1, wherein
The controller is
The sound reproduction apparatus according to claim 1, wherein the time difference is set in a direction in which the first timing is delayed from the second timing. Receive an acoustic signal indicating the sound emitted from the sound source,
Based on the sound intensity indicated by the acoustic signal, determine the intensity of the stimulus to be presented to the human vision,
Based on the information indicating the position of the sound source and the information indicating the position of the person, the distance between the sound source and the person is calculated,
A time difference between a first timing at which a sound reproduced from the acoustic signal is given to the auditory sense of the person and a second timing at which a stimulus having the determined intensity is given to the visual sense of the person is defined as the calculated distance. Control to change accordingly,
A sound reproduction program that causes a computer to execute processing.

Images