JP5595027B2 - Information display processing device - Google Patents

Description

  The present invention relates to an information display processing device that provides information according to, for example, an installation location or a person approaching.

  Information display processing apparatuses are conventionally installed in the outdoors, storefronts, public spaces, transportation facilities, and the like, and are used for the purpose of displaying and advertising advertisements as information. In order to enhance, the structure which performs the display linked with the motion of the pedestrian is disclosed. According to Patent Document 1, an image pickup unit that captures a pedestrian and outputs an image is provided. The image is analyzed to detect the position of the pedestrian. When the detected position of the pedestrian moves by a predetermined amount, a current display is displayed. By synthesizing and displaying a pre-set image and another preset image, the display content is changed to attract the interest of the pedestrian.

JP 2002-123237 A

  However, the configuration of the conventional information display processing device, for example, takes a picture of a pedestrian with a camera of an image pickup means, and there is a problem that the pedestrian hates the camera and avoids it because of the recent trend of emphasizing privacy. Further, the conventional information display processing device only displays the two images stored in advance when the position of the pedestrian moves by a predetermined amount, and is difficult for a moving pedestrian to view. There was a problem.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an information display processing device that generates a display frame that is easily visible to a moving pedestrian.

The present invention information display processing apparatus according to the signal measurement unit for outputting the sound data to sound and vibration signals sampled input from sound and vibration sensors on walkways,-out based on the sound data from the signal measurement section The gait data of the target is calculated for a pedestrian existing in a measurable range, and the viewpoint position of the target one display cycle ahead and the walking timing at which the target foot touches the ground based on the calculated gait data. A gait calculation unit to calculate, an attribute-corresponding content search unit that determines target attributes based on gait data, target viewpoint position, and target walking timing, and obtains provided information corresponding to the target attributes; contents data, the viewpoint position of the target, based on the provided information, the walking time for each display object obtained from the information provided A display frame creation unit for creating a target viewpoint display frame by performing the display processing in accordance with the position and and a display-walking synchronization unit for outputting the walking time target the display frame from the display frame creation unit Is.

According to the information display processing device of the present invention, the signal measurement unit that outputs the sound data obtained by sampling the sound / vibration signal input from the sound / vibration sensor on the pedestrian passage, and the sound data from the signal measurement unit. Dzu-out, a pedestrian present in measurable range to calculate the gait data of the target as the target, one display period the target to the viewpoint and the target of the foot to the ground on the basis of the calculated gait data walking A gait calculation unit that calculates timing, an attribute-corresponding content search unit that determines target attributes based on gait data, target viewpoint position, target walking timing, and obtains provision information corresponding to the target attributes; , gait data, the viewpoint position of the target, based on the provided information, walking for each display object obtained from the information provided in Thailand A display frame creating unit for creating a display frame by performing the display processing in accordance with the viewpoint position of the target in the ring, and a display-walking synchronization unit for outputting a display frame from the display frame creation unit walking time of the target, the By comprising so that the gait data indicating the state of physical movement of the pedestrian can be calculated based on the sound / vibration signal, a display frame can be created based on the gait data. As a result, based on the sound / vibration signals, a display frame that is easily visible to a moving pedestrian and has a high eye catching effect can be generated.

It is a figure which shows the structure of the information display system using the information display processing apparatus by Embodiment 1 of this invention. It is a figure which shows the structure of the information display processing apparatus by Embodiment 1 of this invention. It is a figure which shows the structure of the gait calculation part of the information display processing apparatus by Embodiment 1 of this invention. It is a figure which shows the process of the gait calculation part of the information display processing apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing operation of the information display processing apparatus by Embodiment 1 of this invention. It is a figure which shows the display frame which the information display processing apparatus by Embodiment 1 of this invention produced. It is a figure which shows the structure of the information display processing apparatus by Embodiment 2 of this invention. It is a figure which shows the display frame which the information display processing apparatus by Embodiment 2 of this invention produced. It is a figure which shows the structure of the information display processing apparatus by Embodiment 3 of this invention. It is a figure which shows the display frame which the information display processing apparatus by Embodiment 3 of this invention produced. It is a figure which shows the structure of the gait calculation part of the information display processing apparatus by Embodiment 4 of this invention. It is a flowchart which shows the process of the information display processing apparatus by Embodiment 4 of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a diagram showing an overall configuration of an information display system using an information display processing device. As shown in FIG. 1, the information display system 1 includes an acoustic / vibration sensor 10, an information display processing device 20, a display / acoustic device 30, and the Internet 40. The information display processing device 20 includes the acoustic / vibration sensor 10, The display / acoustic device 30, the Internet 40, and a management center (not shown) are connected via a communication line.

  The acoustic / vibration sensor 10 is installed on the pedestrian passage, measures the sound of the installed space and the vibration of the floor at the installed location, and outputs an acoustic / vibration signal for each measurement position. The acoustic / vibration sensor 10 measures, for example, sound and vibration from a stereo microphone and a diaphragm, and is installed along a pedestrian passage direction on a pedestrian passage that is a predetermined distance away from the display / acoustic device 30. Has been.

  The acoustic / vibration sensor 10 is installed at a distance of, for example, 11 m as a distance that the sound wave travels in 1/30 seconds. Here, 1/30 seconds is the maximum refresh rate indicating the update time of the display frame, and the acoustic / vibration sensor 10 is installed at a distance more than the distance that the sound wave travels in 1/30 seconds, so that it depends on the footsteps of the pedestrian. It is arranged so that the sound is not duplicated. When only one acoustic / vibration sensor 10 is installed, for example, the acoustic / vibration sensor 10 is installed on a pedestrian passage that is a predetermined distance away from the front of the display / acoustic device 30.

  The information display processing device 20 performs a process based on the input sound / vibration signal and functions to output a display frame and a synchronization sound. The information display processing device 20 extracts a target pedestrian (hereinafter abbreviated as a target) from pedestrians walking on a pedestrian passage where the acoustic / vibration sensor 10 is installed, and gait data of the target. The viewpoint position and walking timing are calculated. In addition, the information display processing device 20 acquires provided information based on the target gait data, creates a display frame based on the target gait data and the provided information, and generates a synchronization sound at the same timing as the display frame. And output. The detailed configuration of the information display processing device 20 will be described later.

  The display / acoustic device 30 receives the display frame and the synchronization sound from the information display processing device 20, displays the input display frame on the screen, and outputs the synchronization sound from the speaker.

  The Internet 40 is connected to the information display processing device 20 via a communication line. For example, an information providing server for distributing provided information is provided on the Internet 40.

  A detailed configuration of the information display processing device 20 will be described. FIG. 2 is a diagram showing the configuration of the information display processing device 20. As shown in FIG. 2, the information display processing device 20 includes a signal measurement unit 21, a gait calculation unit 23, an attribute-corresponding content search unit 24, a display frame creation unit 25, a walk synchronization sound generation unit 26, and a display / walk synchronization unit 27. It consists of

  The signal measuring unit 21 functions to perform a sampling process on the sound / vibration signal for each measurement position input from the above-described sound / vibration sensor 10 of FIG. 1 and output sound data. Further, the signal measuring unit 21 functions to calculate and output the phase difference of the sound / vibration signal for each measurement position input from the sound / vibration sensor 10 of FIG. 1 described above.

  The gait calculation unit 23 calculates a frequency spectrum based on the acoustic data from the signal measurement unit 21 and extracts a target based on the transition of the frequency spectrum and the phase difference of the signal with the configuration described later. The gait data indicating the state of physical movement of the target, the target viewpoint position, and the target walking timing are calculated and output.

  Here, the gait data includes the cadence that is the number of steps per minute of the pedestrian, the direction in which the pedestrian is walking, the position of the pedestrian, the pedestrian's stride, the height of the pedestrian, etc. It is data which shows the state of physical exercise. If the target cannot be extracted, the gait calculation unit 23 notifies the attribute-corresponding content search unit 24 and the walking synchronization sound generation unit 26 to that effect.

  The attribute-corresponding content search unit 24 sets and stores target attributes corresponding to target acoustic data and gait data in advance and stores keywords corresponding to each attribute. It has a function of determining the attribute of the target based on the target acoustic data and gait data, using the keyword corresponding to the attribute as a search condition, and searching the Internet 40 to obtain provided information.

  For example, if the target acoustic data indicates a predetermined high frequency range and the gait data has a large cadence value, the attribute-corresponding content search unit 24 determines that the target is wearing a high heel and is a young woman walking fast. Then, keywords corresponding to young women are extracted. The attribute-corresponding content search unit 24 performs a search on the Internet 40 using the extracted keyword as a search condition, and obtains provided information based on the search result.

  If the attribute-corresponding content search unit 24 receives a notification from the gait calculation unit 23 that the target frequency band sound / vibration signal cannot be extracted, the attribute-corresponding content search unit 24 uses preset provision information or preset keywords. The provided information acquired by the Internet 40 search is acquired.

  The display frame creation unit 25 includes a motion blur calculation unit 25a. The motion blur calculation unit 25a recognizes the viewpoint position of the target based on the gait data from the gait calculation unit 23, and determines the target viewpoint position. It functions to calculate the direction and amount of shake in response to movement. The display frame creation unit 25 has a function of creating a display frame corresponding to gait data based on the direction and amount of blur calculated by the motion blur calculation unit 25a and the information provided from the attribute corresponding content search unit 24. To do.

  The display frame creation unit 25 creates a display frame based on the provided information, and moves the display objects constituting the display frame based on the blur direction and blur amount of the target viewpoint calculated by the motion blur calculation unit 25a. To output a display frame in which the position of the display object is changed. With such a configuration, the display frame creation unit 25 creates a display frame that has been processed so as to be easy to see for a pedestrian as a target.

  The walking synchronization sound generator 26 functions to generate and output a synchronization sound in accordance with the timing of outputting the display frame from the display frame creation unit 25. In addition, when the notification that the target cannot be extracted is input from the gait calculator 23, the walking synchronization sound generator 26 monitors whether the display frame is output from the display frame generator 25 and outputs the display frame. Synchronized sound is generated and output according to the timing.

  Based on the gait data from the gait calculation unit 23, the display / walking synchronization unit 27 functions to output the display frame and the synchronization sound from the walking synchronization sound generation unit 26 in accordance with the target walking timing.

  Here, a detailed configuration of the gait calculator 23 will be described with reference to FIG. The gait calculator 23 includes a frequency spectrum calculator 132, a target extractor 133, a gait data calculator 134, a pedestrian viewpoint predictor 135, and a walking timing extractor 136 as shown in FIG.

  The frequency spectrum calculation unit 132 is configured to perform processing by a bandpass filter and FFT (Fast Fourier Transform) processing. The frequency spectrum calculation unit 132 sets and stores a frequency by measuring a walking sound from a typical shoe in advance, and reduces the noise by limiting the frequency range to which the bandpass filter is set, thereby obtaining acoustic data. Output. Even when the acoustic / vibration sensor 10 is installed in public space by the processing of the bandpass filter, the frequency spectrum calculation unit 132 can detect the walking sound obtained by stepping on the diaphragm from the acoustic data including the noise of the hustle and the large amount of walking sound. Sound data is measured.

  Further, the frequency spectrum calculation unit 132 calculates the frequency spectrum by performing FFT processing with a sampling frequency of 48 kHz, a window of 1/30 second, and 1600 samples, for example. At the time of this process, the frequency spectrum calculation unit 132 calculates a frequency spectrum by performing a calculation process continuously for a plurality of acoustic data in time series.

  The target extraction unit 133 extracts the frequency spectrum of the target frequency band based on the phase difference of the sound / vibration signal for each different measurement position and the frequency spectrum from the frequency spectrum calculation unit 132, and based on the extracted frequency spectrum Functions to output target walking timing data. For example, the target extraction unit 133 estimates and calculates the sound generation position based on the phase difference and frequency spectrum of the sound / vibration signal. The target extraction unit 133 sets the calculated sound generation position as the relative position of each pedestrian to the display / acoustic device, and targets the pedestrian who is closest to the display / acoustic device 30 based on the relative position of each pedestrian. And the transition of the frequency spectrum of the target is output as walking timing data. If the target cannot be extracted, the target extraction unit 133 outputs a notification that the target cannot be extracted to the attribute corresponding content search unit 24 and the display frame creation unit 25.

  In addition, when only one acoustic / vibration sensor 10 is installed, the target extraction unit 133 is based on the frequency spectrum from the frequency spectrum calculation unit 132, and the acoustic data strength and time-series acoustic data for each frequency band. It is also possible to configure so as to output walking timing data of, for example, the transition of acoustic data in the strongest frequency band based on the comparison result.

  The gait data calculation unit 134 has a function of calculating gait data. The gait data calculation unit 134 calculates how many times there are signals per minute based on the walking timing data from the target extraction unit 133, and calculates a cadence indicating the number of steps per minute of the pedestrian. Also, the gait data calculation unit 134 calculates and calculates the target direction, the target position, the pedestrian stride, and the pedestrian height based on the calculated cadence and the phase difference between the sound and vibration signals at each measurement position. Is configured to do. The gait data calculation unit 134 outputs the cadence, target direction, target position, pedestrian stride, and pedestrian height calculated as described above as gait data.

  The pedestrian viewpoint prediction unit 135 functions to estimate and calculate the target viewpoint position one display cycle ahead based on the gait data from the gait data calculation unit 134. The pedestrian viewpoint prediction unit 135 calculates the viewpoint position or viewpoint direction of the target 1/30 seconds ahead as one display cycle ahead, for example.

  The walking timing extraction unit 136 has a function of extracting the walking timing at which the target touches the ground based on the walking timing data from the target extraction unit 133 and the gait data from the gait data calculation unit 134. The walking timing extraction unit 136 extracts and outputs the walking timing at which the target touches the ground, for example, based on the cadence of the gait data and the walking timing data.

Here, an example of the processing operation of the information display processing device 20 of the first embodiment will be described.
FIG. 4A is a volume / vibration amount-time graph showing the acoustic data output by the signal measuring unit 21. FIG. 4B is a volume / vibration amount-frequency graph showing the frequency spectrum obtained by the processing of the frequency spectrum calculation unit 132. 4C shows the frequency range of the low frequency range of FIG. 4B, FIG. 4D shows the frequency range of the mid frequency range of FIG. 4B, and FIG. (E) has shown the period of the frequency of the treble range of FIG.4 (b), and is a graph of a volume, vibration amount-time, respectively. FIG. 5 is a flowchart showing the processing operation of the information display processing device 20. FIG. 6 shows a display in which the object of the display frame is moved in accordance with the vertical movement of the pedestrian.

First, a frequency spectrum and gait data extraction method will be described.
The signal measuring unit 21 receives the sound / vibration signal from the sound / vibration sensor 10 as shown in FIG. 5 (step ST101), samples the sound / vibration signal, and has a waveform as shown in FIG. 4A, for example. Are output (step ST102).

  The frequency spectrum calculation unit 132 performs a process of limiting the frequency band of the acoustic data on the acoustic data input from the signal measurement unit 21 with a bandpass filter in a preset frequency range, thereby reducing noise. Subsequently, the frequency spectrum calculation unit 132 calculates the frequency spectrum shown in FIG. 4B by performing FFT processing with a sampling frequency of 48 kHz, a window of 1/30 second, and 1600 samples, and the frequency spectrum is compared with the target extraction unit 133. The data is output to the content data calculation unit 134 (step ST103). The frequency spectrum calculation unit 132 continuously performs calculation processing on a plurality of sound / vibration data in time series, and performs predetermined processing as shown in FIGS. 4 (c), 4 (d), and 4 (e). The period is detected for each frequency band.

  The target extraction unit 133 estimates and calculates the sound generation position based on the phase difference of the sound / vibration signal for each measurement position from the signal measurement unit 21 and the frequency spectrum from the frequency spectrum calculation unit 132. The target extraction unit 133 sets the calculated sound generation position as the relative position of each pedestrian to the display / acoustic device, and targets the pedestrian who is closest to the display / acoustic device 30 based on the relative position of each pedestrian. And the transition of the frequency spectrum of the target is output as gait timing data to the gait data calculation unit 134 and the gait timing extraction unit 136 (step ST104).

  At this time, the target extraction unit 133 determines whether or not the target has been extracted (step ST105). If the target cannot be extracted ("NO" in step ST105), a notification indicating that the target cannot be extracted is sent to the attribute-corresponding content search unit. 24 and the provided information are output to the display frame creation unit 25, and the process proceeds to step ST115. Further, when the target extraction unit 133 can extract the target (step ST105 “YES”), the process proceeds to step ST106.

  When the gait data calculation unit 134 inputs the walking timing data from the target extraction unit 133, the gait data calculation unit 134 calculates how many times the signal is present per minute based on the walking timing data input from the target extraction unit 133. A cadence indicating the number of steps per minute is calculated. The gait data calculation unit 134 also determines the direction of the target, the position of the target, the stride of the target, the height of the target based on the phase difference between the calculated cadence and the sound / vibration signal for each measurement position input from the signal measurement unit 21. It is comprised so that it may estimate and calculate. The gait data calculation unit 134 outputs the calculated cadence, target direction, target position, target stride, and target height as gait data (step ST106).

  When gait data is input from the gait data calculation unit 134, the pedestrian viewpoint prediction unit 135 estimates and calculates the viewpoint position of the pedestrian 1/30 seconds ahead as one display cycle ahead based on the gait data. (Step ST107).

  The walking timing extraction unit 136 extracts and outputs the walking timing at which the target touches the ground based on the walking timing data input from the target extraction unit 133 and the gait data input from the gait data calculation unit 134 (step ST108). .

Next, a processing operation for acquiring provided information corresponding to the attribute of a pedestrian will be described.
When the attribute-corresponding content search unit 24 receives the target sound / vibration signal and data from the gait calculation unit 23, the attribute corresponding content search unit 24 determines the target gender, age, etc. Is determined (step ST109).

  Based on the pedestrian's gender and age, the attribute-corresponding content search unit 24 extracts keywords corresponding to gender and age in advance, and searches the Internet 40 using the extracted keywords as search conditions. The attribute corresponding content search unit 24 creates provision information based on the search result, and outputs the created provision information to the display frame creation unit 25 (step ST110).

Next, display frame conversion processing will be described.
In FIG. 6, the pedestrian 232 a is approaching toward the display / acoustic device 30. A display object 231 a is displayed on the display / acoustic device 30. At this time, as shown in FIGS. 6A and 6B, the pedestrian 232 is walking while changing the height of the viewpoint.

  When the display frame creation unit 25 continuously inputs gait data from the gait calculation unit 23, the motion blur calculation unit 25a is based on the pedestrian's viewpoint position of the gait data, and the blur direction of the pedestrian's viewpoint position, Calculate the amount of shake. Based on the blur direction and blur amount of the viewpoint position of the pedestrian and the walking timing from the walking timing extracting unit 136, the display frame creating unit 25 is at the timing when the target foot touches down as shown in FIG. A display frame is created so that the display object 231b is displayed at the position where the viewpoint position of the pedestrian is lowest, and is output to the walking synchronization sound generating unit 26 (step ST111).

  In step ST111, when the movement of the target viewpoint position shifts from the viewpoint 232a to the viewpoint 232b by the processing operation of the motion blur calculation unit 25a, the display position drops from the display object 231a to the display object 231b. When the movement of the target viewpoint position shifts from the viewpoint 232b to the viewpoint 232a, the display position rises from the display object 231b to the display object 231a. The blurring is large when the viewpoint movement is fast, and the blurring is small when the viewpoint movement is slow. The motion blur calculation unit 25a corrects the display frame in this way, thereby displaying an easily viewable display frame on the target.

  The walking synchronization sound generation unit 26 generates a synchronization sound when the display frame is input from the display frame creation unit 25 in step ST111, and outputs the display frame and the synchronization sound to the display / walking synchronization unit 27 at the same timing (step ST112). .

  When the display / walking synchronization unit 27 inputs the walking timing from the target extraction unit 133 and inputs the display frame and the synchronization sound from the walking synchronization sound generation unit 26, the display / acoustic device displays the display frame and the synchronization sound in accordance with the walking timing. 30 (step ST113).

  The display / acoustic device 30 outputs the display frame and the synchronization sound input from the display / walking synchronization unit 27 from the screen and the speaker (step ST114), and ends the process (end).

  In the case of step ST105 “NO”, the attribute-corresponding content search unit 24 receives a notification that the target cannot be extracted from the target extraction unit 133, and acquires it by Internet 40 search using preset provision information or preset keywords. The provided information is acquired and output to the display frame creation unit 25 (step ST115).

  In the case of step ST105 “NO”, the display frame creation unit 25 inputs a notification that the target cannot be extracted from the target extraction unit 133, and inputs the provision information from the attribute corresponding content search unit 24 in step ST115. A display frame is created based on the above and output to the walking synchronization sound generator 26 (step ST116).

  The walking synchronization sound generation unit 26 generates a synchronization sound when the display frame is input from the display frame creation unit 25 in step ST116, and outputs the display frame and the synchronization sound to the display / acoustic device 30 at the same timing (step ST117).

  The display / acoustic device 30 outputs the display frame and the synchronization sound input from the display / walking synchronization unit 27 from the screen and the speaker (step ST118), and ends the process (END).

  As described above, according to the first embodiment, the signal measuring unit 21 outputs the sound data obtained by sampling the sound / vibration signal input from the sound / vibration sensor 10 on the pedestrian passage, and the gait calculating unit 23 is The gait data is calculated based on the acoustic data from the signal measuring unit 21, and the target viewpoint position and the target walking timing of one display cycle ahead are calculated based on the calculated gait data, and the attribute corresponding content search unit 24 determines the attribute of the target based on the gait data from the gait calculator 23, the target viewpoint position of the display cycle ahead, and the target walking timing, and provides provided information corresponding to the target attribute via the communication line. The display frame creation unit 25 obtains the gait data from the gait calculation unit 23, the target viewpoint position of the display cycle ahead, the attribute corresponding content. Based on the information provided from the search unit 24, a display frame corresponding to the target viewpoint position is created, and the display / walking synchronization unit 27 outputs the display frame from the display frame creation unit 25 at the target walking timing. Since it is configured, it is possible to calculate gait data indicating the state of physical movement of the target based on the walking sound of the target, obtain provided information based on the gait data, and display in conjunction with the movement of the target You can create a frame. As a result, it is possible to display a display frame that is easy to see for pedestrians who are walking, enhance the eye catching effect, and can be used in the digital signage field as a new advertising display with improved advertising effectiveness. it can.

  In addition, according to the first embodiment, the signal measuring unit 21 outputs sound data obtained by sampling the sound / vibration signals input from the plurality of sound / vibration sensors 10 on the pedestrian passage, and the plurality of sound / vibration sensors 10 are output. Since the diaphragm is arranged at the installation position, the sound / vibration signal emitted by the pedestrian can be extracted even when there are many pedestrians or the surrounding noise is intense.

  Further, according to the first embodiment, the gait data calculation unit 134 determines the generation source of the sound / vibration signal based on the phase difference between the sound / vibration signals from the sound / vibration sensors 10 installed at a plurality of positions. Since the estimation is configured, the direction and position of the target can be calculated.

  Further, according to the first embodiment, the pedestrian viewpoint prediction unit 135 calculates the target viewpoint position based on the cadence, direction, position, step length, and height from the gait data calculation unit 134, and the display frame generation unit 25. Calculates the blur direction and blur amount due to the movement of the target viewpoint position based on the target viewpoint position from the pedestrian viewpoint prediction unit 135, and displays the display frame based on the calculated blur direction and blur amount. Since the motion blur calculation unit 25a to be created is provided, the display frame seen while the target moves can be easily seen.

  Further, according to the first embodiment, the attribute-corresponding content search unit 24 determines the target attribute based on the sound / vibration signal and the gait data from the gait calculation unit 23, and searches for information corresponding to the attribute. Information suitable for the target can be provided.

  Note that, as a method for analyzing the sound / vibration signal in the first embodiment, Weblet transform may be used as a means for localizing the frequency and time change.

Embodiment 2. FIG.
In the first embodiment, the configuration in which the display position of the display object of the display frame is moved up and down in accordance with the movement of the target viewpoint position has been described. However, the second embodiment induces the focus adjustment of the target and senses depth. The display that makes you feel is explained.

  FIG. 7 shows the configuration of the information display processing device 20 of the second embodiment. The configuration of the information display processing device 20 of the second embodiment is configured by an unsharpness calculation unit 25b instead of the motion blur calculation unit 25a of the display frame creation unit 25 of FIG. 2 described above. Since the configuration other than the frame creation unit 25 is the same as the configuration of FIG. 2 described above, description thereof is omitted.

  The display frame creation unit 25 includes an unsharpness calculation unit 25b as shown in FIG. 7, and the unsharpness calculation unit 25b includes target gait data from the gait calculation unit 23, target viewpoint position, Based on the target walking timing, it functions to calculate an unsharpness indicating the degree of blurring of the display objects that constitute the display frame. The display frame creation unit 25 creates a display frame by focusing the display objects in order according to the walking timing and processing the display objects other than the focused display object to blur based on the unsharpness. To do.

  Next, the processing operation of the information display processing device 20 according to the second embodiment will be described. FIG. 8 shows an example of a display frame displayed on the display / acoustic device 30 of the second embodiment. The display frame includes independent display objects 231c, 231d, 231e, 231f, and 231g as five layers in the depth direction. Since the processing operation of the information display processing device 20 is substantially the same as that in FIG. 5 described above, the description of processing operations other than the display frame creation unit 25 is omitted.

  When the target gait data, the target viewpoint position, and the target walking timing are input from the gait calculation unit 23, the display frame creation unit 25 receives the display objects 231c, 231d, 231e, 231f, Focus on 231g in order. At this time, the unsharpness calculator 25b calculates the unsharpness based on the viewpoint position of the pedestrian. For example, when the display object 231c is focused, the display frame creation unit 25 processes the display objects 231d, 231e, 231f, and 231g other than the focused display object 231c to blur based on the unsharpness. A display frame is created (step ST111 in FIG. 5 described above).

  As described above, according to the second embodiment, the unsharpness calculation unit 25b blurs the foreground and the background based on the walking timing from the walking timing extraction unit 136 and the target viewpoint position from the pedestrian viewpoint prediction unit 135. Since the condition is calculated and the display objects 231c, 231d, 231e, 231f, and 231g constituting the display frame are blurred, the focus adjustment of the target can be induced to generate a sense of depth. As a result, it is possible to display a display frame that is easy to see for pedestrians who are walking, enhance the eye catching effect, and can be used in the digital signage field as a new advertising display with improved advertising effectiveness. it can.

Embodiment 3 FIG.
In the second embodiment, a configuration has been described in which a display frame in which the display objects 231c, 231d, 231e, 231f, and 231g are sequentially focused is created. However, in the third embodiment, exercise according to the target viewpoint position is performed. A configuration for creating a display frame of a parallax image will be described.

  FIG. 9 shows the configuration of the information display processing device 20 of the third embodiment. Note that the configuration of the information display processing device 20 of the third embodiment is configured by a motion parallax display object calculation unit 25c instead of the motion blur calculation unit 25a of the display frame creation unit 25 of FIG. 2 described above. Since the configuration other than the display frame creation unit 25 is the same as the configuration of FIG. 2 described above, description thereof is omitted.

  As shown in FIG. 9, the display frame creation unit 25 includes a motion parallax display object calculation unit 25c. The motion parallax display object calculation unit 25c receives the target gait data and the target from the gait calculation unit 23. The motion parallax effect of the display objects constituting the display frame is calculated based on the viewpoint position and the target walking timing. Here, the motion parallax effect indicates, for example, the position of the display object so that the position of each display object in the target viewing space changes regularly in a certain direction.

  Next, the processing operation of the information display processing device 20 according to the second embodiment will be described. FIG. 10 shows an example of a display frame displayed on the display / acoustic device 30 of the third embodiment. The display frame is composed of independent display objects 231c, 231d, 231e, 231f, and 231g as five layers in the depth direction, similar to the display frame of FIG. 10 described above. Since the processing operation of the information display processing device 20 is substantially the same as that in FIG. 5 described above, the description of the processing operations other than the display frame creation unit 25 will be omitted.

  When the display frame creation unit 25 inputs the target gait data, the target viewpoint position, and the target walking timing from the gait calculation unit 23, the motion parallax display object calculation unit 25 c includes the target position included in the gait data. And the motion parallax effect is calculated based on the target walking timing. The display frame creation unit 25 moves the positions of the display objects 231c, 231d, 231e, 231f, and 231g constituting the display frame based on the motion parallax effect, and the motion parallax display objects 231c ′ and 231d corresponding to the target positions. ', 231e', 231f ', and 231g' are generated to create a display frame (step ST111 in FIG. 5 described above).

  In FIG. 10A, the pedestrian 235a is moving in the right direction of the display / acoustic device 30. The pedestrian 235a proceeds in the front direction of the display / acoustic device 30, and reaches the front of the display / acoustic device 30 in FIG. In FIG. 10A, a full moon appears on the right of the steel tower in the display of the display / acoustic device 18. On the other hand, in FIG. 10B as viewed from the front of the display / acoustic device 30, the full moon is displayed on the left of the steel tower. In this way, when the pedestrian as a target moves the body by the motion parallax display object calculation unit 25c, the display frame is displayed so that the positions of the objects in the visual space change regularly in a certain direction. It is processed so that it gives a sense of depth according to the walking timing.

  As described above, according to the third embodiment, the motion parallax display object calculation unit 25c is based on the target position from the gait data calculation unit 134 and the walking timing from the walking timing extraction unit 136. Since the effect is calculated and the display frame is generated by generating the motion parallax display objects 231c ′, 231d ′, 231e ′, 231f ′, and 231g ′ based on the motion parallax effect, the display is generated according to the angle viewed from the target. A display frame can be created to give a sense of depth, and a display frame that is easy to see for pedestrians can be obtained.

Embodiment 4 FIG.
In the first to third embodiments, the configuration for creating the display frame based on the provision information corresponding to the target attribute has been described. However, the fourth embodiment obtains and displays the provision information related to the abnormality. Will be described. Since the configuration other than the gait calculation unit 23 is the same as the configuration of FIGS. 1 and 2 described above, the same reference numerals are given and description thereof is omitted.

  FIG. 11 shows the configuration of the gait calculator 23 of the fourth embodiment. As shown in FIG. 11, the gait calculation unit 23 includes an abnormality detection unit 131, a frequency spectrum calculation unit 132, a target extraction unit 133, a gait data calculation unit 134, a pedestrian viewpoint prediction unit 135, and a walking timing extraction unit 136. It is configured.

  The abnormality detection unit 131 monitors the presence / absence of an abnormality based on the sound / vibration signal, and when an abnormality is detected, notifies the management center (not shown) and collects provision information regarding the abnormality from the Internet 40. For example, the abnormality detection unit 131 analyzes the value of the sound / vibration signal input from the signal measurement unit 21 illustrated in FIG. 2 described above, and if there is an overflow of the dynamic range, determines an abnormal value to estimate the occurrence of the earthquake, In addition to notifying the management center via the Internet 40, access to the Japan Meteorological Agency website to collect information on earthquakes (earthquake information).

  The frequency spectrum calculation unit 132 is configured to perform processing by a bandpass filter and FFT (Fast Fourier Transform) processing. The frequency spectrum calculation unit 132 reduces noise by processing using a bandpass filter on the acoustic data input from the signal measurement unit 21 shown in FIG. 2 described above. For example, the sampling frequency is 48 kHz, the window is 1/30 second, and 1600 samples. An FFT process is performed to calculate a frequency spectrum. Note that the frequency spectrum calculation unit 132 calculates a frequency spectrum by performing calculation processing for a plurality of acoustic data continuously in time series.

  The target extraction unit 133 extracts the frequency spectrum of the target frequency band based on the phase difference of the sound / vibration signal for each different measurement position and the frequency spectrum from the frequency spectrum calculation unit 132, and based on the extracted frequency spectrum Functions to output target walking timing data. For example, the target extraction unit 133 estimates and calculates the sound generation position based on the phase difference and frequency spectrum of the sound / vibration signal. The target extraction unit 133 sets the calculated sound generation position as the relative position of each pedestrian to the display / acoustic device, and targets the pedestrian who is closest to the display / acoustic device 30 based on the relative position of each pedestrian. And the transition of the frequency spectrum of the target is output as walking timing data. If the target cannot be extracted, the target extraction unit 133 outputs a notification that the target cannot be extracted to the attribute corresponding content search unit 24 and the display frame creation unit 25.

  In addition, when only one acoustic / vibration sensor 10 is installed, the target extraction unit 133 is based on the frequency spectrum from the frequency spectrum calculation unit 132, and the acoustic data strength and time-series acoustic data for each frequency band. It is also possible to configure so as to output walking timing data of, for example, the transition of acoustic data in the strongest frequency band based on the comparison result.

  The gait data calculation unit 134 has a function of calculating gait data. The gait data calculation unit 134 calculates how many times there are signals per minute based on the walking timing data from the target extraction unit 133, and calculates a cadence indicating the number of steps per minute of the target. The gait data calculation unit 134 estimates and calculates the target direction, the target position, the target stride, and the target height based on the calculated cadence and the phase difference between the sound and vibration signals for each measurement position. It is configured. The gait data calculation unit 134 outputs the cadence, target direction, target position, target stride, and target height calculated as described above as gait data.

  The pedestrian viewpoint prediction unit 135 functions to estimate and calculate the target viewpoint position one display cycle ahead based on the gait data from the gait data calculation unit 134. The pedestrian viewpoint prediction unit 135 calculates the viewpoint position or viewpoint direction of the target 1/30 seconds ahead as one display cycle ahead, for example.

  The walking timing extraction unit 136 has a function of extracting the walking timing at which the target touches the ground based on the walking timing data from the target extraction unit 133 and the gait data from the gait data calculation unit 134. The walking timing extraction unit 136 extracts and outputs the walking timing at which the target touches the ground, for example, based on the cadence of the gait data and the walking timing data.

  Next, an example of the processing operation of the information display processing device 20 according to the first embodiment will be described. FIG. 12 is a flowchart showing the processing operation of the information display processing device 20. First, a frequency spectrum and gait data extraction method will be described.

  The signal measuring unit 21 receives the sound / vibration signal from the sound / vibration sensor 10 as shown in FIG. 5 (step ST201), samples the sound / vibration signal, and has a waveform as shown in FIG. 4A, for example. Are output (step ST202).

  When the sound / vibration signal is input from the signal measurement unit 21, the abnormality detection unit 131 analyzes the input sound / vibration signal, monitors whether the sound / vibration signal is an abnormal value, and determines whether an abnormality is detected. (Step ST203). If it is determined in step ST203 that no abnormality has been detected (step ST203 “NO”), abnormality detection section 131 notifies frequency spectrum calculation section 132 that there is no abnormality, and proceeds to step ST205.

If there is an overflow outside the dynamic range in step ST203, the abnormality detection unit 131 determines that the sound / vibration signal is abnormal (step ST203 “YES”). The abnormality detection unit 131 estimates that the type of abnormality is, for example, an earthquake based on the abnormal value of the sound / vibration signal, and transmits an earthquake occurrence notification to a management center (not shown) via the Internet 40.
In addition, the anomaly detection unit 131 accesses the Japan Meteorological Agency home page, collects provision information about the earthquake, and notifies the display frame creation unit 25 of the provision information about the earthquake together with the earthquake occurrence notification (step ST204).

  The frequency spectrum calculation unit 132 performs a process of limiting the frequency band of the acoustic data on the acoustic data input from the signal measurement unit 21 with a bandpass filter in a preset frequency range, thereby reducing noise. Subsequently, the frequency spectrum calculation unit 132 calculates the frequency spectrum shown in FIG. 4B by performing FFT processing with a sampling frequency of 48 kHz, a window of 1/30 second, and 1600 samples, and the frequency spectrum is compared with the target extraction unit 133. The data is output to the content data calculation unit 134 (step ST205). The frequency spectrum calculation unit 132 continuously performs calculation processing on a plurality of sound / vibration data in time series, and performs predetermined processing as shown in FIGS. 4 (c), 4 (d), and 4 (e). The period is detected for each frequency band.

  The target extraction unit 133 estimates and calculates the sound generation position based on the phase difference of the sound / vibration signal for each measurement position from the signal measurement unit 21 and the frequency spectrum from the frequency spectrum calculation unit 132. The target extraction unit 133 sets the calculated sound generation position as the relative position of each pedestrian to the display / acoustic device, and targets the pedestrian who is closest to the display / acoustic device 30 based on the relative position of each pedestrian. And the transition of the frequency spectrum of the target is output as gait timing data to the gait data calculation unit 134 and the gait timing extraction unit 136 (step ST206).

  At this time, the target extraction unit 133 determines whether or not the target has been extracted (step ST207). If the target cannot be extracted ("NO" in step ST207), a notification indicating that the target cannot be extracted is sent to the attribute-corresponding content search unit. 24 and the provided information are output to the display frame creation unit 25, and the process proceeds to step ST218. Further, when the target extraction unit 133 can extract the target (step ST207 “YES”), the process proceeds to step ST208.

  When the gait data calculation unit 134 inputs the walking timing data from the target extraction unit 133 in step ST206, the gait data calculation unit 134 calculates how many times there are signals per minute based on the walking timing data input from the target extraction unit 133, A cadence indicating the number of steps per minute of the target is calculated. The gait data calculation unit 134 also determines the direction of the target, the position of the target, the stride of the target, the height of the target based on the phase difference between the calculated cadence and the sound / vibration signal for each measurement position input from the signal measurement unit 21. It is comprised so that it may estimate and calculate. The gait data calculation unit 134 outputs the calculated cadence, target direction, target position, target stride, and target height as gait data (step ST208).

  When gait data is input from the gait data calculation unit 134, the pedestrian viewpoint prediction unit 135 estimates and calculates the target viewpoint position 1/30 seconds ahead as one display cycle based on the gait data. (Step ST209).

  The walking timing extraction unit 136 extracts and outputs the walking timing at which the target touches the ground based on the walking timing data input from the target extraction unit 133 and the gait data input from the gait data calculation unit 134 (step ST210). .

  When the gait data, the viewpoint position, and the walking timing are input from step ST208 to step ST210, the attribute-corresponding content search unit 24 determines the presence / absence of provision information related to the earthquake from the abnormality detection unit 131 (step ST211). If the attribute-corresponding content search unit 24 determines in step ST211 that there is provision information related to the earthquake (step ST211 “NO”), the process proceeds to step ST214.

  If the attribute-corresponding content search unit 24 determines in step ST211 that there is no provision information related to the earthquake (step ST211 “YES”), the target acoustic / vibration signal and gait data are input from the gait calculation unit 23. The gender, age, and the like of the target are determined from attributes previously associated with the sound / vibration signal and gait data (step ST212).

  The attribute-corresponding content search unit 24 extracts keywords corresponding to sex and age in advance based on the sex and age of the target, and searches the Internet 40 using the extracted keyword as a search condition. The attribute corresponding content search unit 24 creates provision information based on the search result, and outputs the created provision information to the display frame creation unit 25 (step ST213).

  The display frame creation unit 25 creates a display frame in the same manner as in step ST111 of FIG. 5 described above based on the provision information regarding the earthquake from the abnormality detection unit 131 or the provision information from the attribute corresponding content search unit 24 (step ST214). ). Henceforth, since the process of step ST215 to step ST217 is the same as the process of step ST112 to step ST114 of FIG. 5 mentioned above, description is abbreviate | omitted.

  In step ST207 “NO”, the attribute-corresponding content search unit 24 receives the notification that the target cannot be extracted from the target extraction unit 133, and determines whether there is provision information related to the earthquake from the abnormality detection unit 131 (step ST218). . If the attribute-corresponding content search unit 24 determines in step ST218 that there is provided information related to the earthquake (step ST218 “NO”), the process proceeds to step ST220.

  If the attribute-corresponding content search unit 24 determines in step ST218 that there is no provision information related to the earthquake (step ST218 “YES”), the provision information acquired by the Internet 40 search using preset provision information or a preset keyword Is acquired (step ST219).

  The display frame creation unit 25 creates a display frame based on the provision information about the earthquake from the abnormality detection unit 131 or the provision information from the attribute corresponding content search unit 24 as in step ST116 of FIG. 5 described above (step ST220). ). Thereafter, the processing of step ST221 and step ST222 is the same as the processing of step ST117 and step ST118 of FIG.

  As described above, according to the fourth embodiment, the abnormality detection unit 131 can detect an abnormality in accordance with the sound / vibration signal and notify the management center via the Internet 40, and the abnormality information regarding the abnormality. Therefore, it is possible to provide abnormality information to pedestrians when an abnormality occurs.

  In the first to third embodiments, the configuration in which the display frame creation unit 25 includes any one of the motion blur calculation unit 25a, the unsharpness calculation unit 25b, and the motion parallax display object calculation unit 25c has been described. However, the display frame creation unit 25 includes a motion blur calculation unit 25a, an unsharpness calculation unit 25b, and a motion parallax display object calculation unit 25c. You may comprise so that it may select and perform an information display process.

  Further, instead of the acoustic / vibration sensor 10, an image sensor may be used. In that case, the gait calculator 23 is configured to calculate gait data based on the image signal from the image sensor.

  DESCRIPTION OF SYMBOLS 1 Information display system, 10 Sound / vibration sensor, 20 Information display processing apparatus, 30 Display / acoustic device, 40 Internet, 21 Signal measurement part, 23 Gait calculation part, 24 Attribute corresponding content search part, 25 Display frame creation part, 25a Motion blur calculation unit, 25b Unsharpness calculation unit, 25c Motion parallax display object calculation unit, 26 Walking synchronization sound generation unit, 27 Display / walking synchronization unit, 131 Abnormality detection unit, 132 Frequency spectrum calculation unit, 133 Target extraction Part, 134 gait data calculation part, 135 pedestrian viewpoint prediction part, 136 walking timing extraction part, 231a, 231b, 231c, 231d, 231e, 231f, 231g image, 232, 232a, 232b pedestrian.

Claims (9)

A signal measuring unit that outputs sound data obtained by sampling sound / vibration signals input from sound / vibration sensors on a pedestrian passage;
-Out based on the sound data from the signal measurement unit, a pedestrian present in measurable range to calculate the gait data of the target as the target, based on the calculated gait data one display cycle destination target A gait calculation unit that calculates a walking timing at which the viewpoint position and the target foot touch the ground ;
The gait data, the viewpoint position of the target, and the attribute corresponding content search unit that determines the attribute of the target based on the walking time of the target, obtains the providing information corresponding to the attribute of the target,
Based on the gait data, the target viewpoint position, and the provided information, a display frame is created by performing display processing corresponding to the target viewpoint position at the walking timing for each display object obtained from the provided information. A display frame creation unit to
An information display processing device comprising: a display / walking synchronization unit that outputs a display frame from the display frame creation unit at a walking timing of the target . The signal measuring unit outputs sound data obtained by sampling sound / vibration signals input from a plurality of sound / vibration sensors on the pedestrian passage,
2. The information display processing apparatus according to claim 1, wherein a diaphragm is disposed at a position where the plurality of acoustic / vibration sensors are installed. The signal measurement unit outputs the phase difference between the sound data obtained by sampling the sound / vibration signals input from the plurality of sound / vibration sensors on the pedestrian passage and the sound / vibration signals for each measurement position,
The gait calculation unit calculates the direction and position of the target based on the acoustic data, the installation positions of the plurality of acoustic / vibration sensors, and the phase difference of the acoustic / vibration signals from the signal measurement unit. The information display processing device according to claim 1. The display frame creation unit calculates a blur direction and a blur amount of the target viewpoint position based on the target viewpoint position continuously input from the gait calculation unit, and creates a display frame. The information display processing device according to claim 1, further comprising a unit. The display frame creation unit creates an display frame by calculating an unsharpness indicating the degree of blur of a display object constituting the display frame based on the target viewpoint position and the walking timing from the gait calculation unit. The information display processing apparatus according to claim 1, further comprising a sharpness calculation unit. The display frame creation unit calculates a motion parallax effect indicating a display position that changes in accordance with the target position based on the target viewpoint position and walking timing of the target from the gait calculation unit, and creates a display frame The information display processing apparatus according to claim 1, further comprising a parallax display object calculation unit. When an abnormal value is detected based on the sound / vibration signal from the signal measurement unit, an abnormality detection unit that determines that an earthquake has occurred and obtains provision information about the earthquake via a communication line is provided,
The information display processing device according to claim 1, wherein the display frame creation unit creates a display frame based on information provided from the abnormality detection unit.   8. The apparatus according to claim 1, further comprising: a synchronization sound generated in accordance with a timing at which a display frame is input from the display frame creation unit, and a walking synchronization sound generation unit that outputs the display frame. The information display processing device according to claim 1. A signal measuring unit that outputs image data obtained by sampling an image signal input from an image sensor on a pedestrian passage;
-Out based on the image data from the signal measurement unit, a pedestrian present in measurable range to calculate the gait data of the target as the target, based on the calculated gait data one display cycle destination target A gait calculation unit that calculates a walking timing at which the viewpoint position and the target foot touch the ground ;
The gait data, the viewpoint position of the target, and the attribute corresponding content search unit that determines the attribute of the target based on the walking time of the target, obtains the providing information corresponding to the attribute of the target,
Based on the gait data, the target viewpoint position, and the provided information, a display frame is created by performing display processing corresponding to the target viewpoint position at the walking timing for each display object obtained from the provided information. A display frame creation unit to
An information display processing device comprising: a display / walking synchronization unit that outputs a display frame from the display frame creation unit at a walking timing of the target .

Images