JP6645480B2 - Sound generation device, sound generation control program - Google Patents

Description

  The present invention relates to a sound generation device and a sound generation control program for calling a driver's attention while a vehicle is running.

  2. Description of the Related Art Conventionally, in a racing game, a drive simulator, or the like, an engine simulation sound generation device that generates an engine simulation sound has been proposed in order to enhance the sense of presence or to generate an engine sound according to an actual driving state in an electric vehicle. Have been.

  Patent Literature 1 describes generating an engine simulation sound based on a map formed by allocating sound pressure waveform data of an engine sound within a predetermined time period to a rotation speed and an accelerator operation amount.

  Further, Patent Document 2 discloses that an engine operation state is divided into a plurality of ranges by using a driving operation amount and an engine rotation speed as parameters, and each of engine sounds recorded in a substantially central state of each range is converted by the crankshaft for one fuel cycle. It is stored as digital data of the engine sound in units of the sound pressure waveform of the length corresponding to the rotation time, and the engine sound data in the operating state range according to the input of the operation amount and the engine speed can be repeatedly sounded. Has been described.

  In the conventional sound generation devices for vehicles and the like, including Patent Documents 1 and 2, the sounds related to the running of the vehicle and the like correspond to four corners of a two-dimensional parameter (two-dimensional coordinates) of an engine speed and an accelerator operation amount. A sample of an engine sound to be prepared is prepared, and a sound at a coordinate point corresponding to the detected engine speed and accelerator operation amount is complemented and output.

  The complementing means changes the sound reproduction speed according to the engine speed and the accelerator operation amount. Further, a weight is set according to the engine speed and the accelerator operation amount, and the volume is changed.

  On the other hand, Patent Literature 3 describes that automatic music is performed in accordance with the surrounding environment. That is, in Patent Document 3, music is generated by simultaneously playing a plurality of parts. The sound of each part sounds according to the reaction of a video, a radio wave sensor, or the like.

  Further, Patent Literature 4 discloses that a danger determining unit determines a danger of driving based on a signal from a speed sensor, and generates a scale of a sound in music to be generated, a chord to sound in the music, a timing to sound a musical instrument, and a musical instrument. It is described that at least one of the performance speeds is changed.

  As a reference, a vehicle music generating device that generates music using music data and a detection result of a vehicle state is disclosed (Patent Document 5).

  Further, as a reference, a signal monitoring music performance device that converts a change in the monitoring target into a change in music based on an external signal input from the monitoring target displayed on the monitor, thereby reflecting the monitoring work to the auditory sense. Is disclosed (Patent Document 6).

  Also, as an example, there is disclosed an acoustic composition device that automatically reproduces information using an external environment or human biological information (see Patent Document 7).

JP 2000-1142 A JP-A-2000-10576 JP-A-2006-183988 JP-A-2006-102863 Japanese Patent No. 4984861 Japanese Patent No. 4458581 Japanese Patent No. 3381074

  For example, a vehicle with a high level of silence has a structure that shuts off engine noise in the vehicle compartment as much as possible. Further, in an electric vehicle, no engine sound is present while the driving source of the vehicle is shifted from the engine to the motor or the like. For example, a sound output target is a person walking around a vehicle or a person running on a bicycle, which is effective for the purpose of calling attention to the person, but a vehicle and an electric vehicle with high quietness are effective. For a driver driving a car, it may be harsh while driving.

  On the other hand, there is a demand for a driver to recognize the state of the surrounding environment as part of safe driving of the vehicle.

  However, although there is a sound generation technology using the operation state of the vehicle as a parameter, the relationship between the surrounding environment during driving and the sound is not considered.

  The present invention has been made in consideration of the above facts, and has a relationship between a surrounding environment and a sound during driving, and a sound generation device capable of giving a function as an alert function when the driver drives the output sound. It is an object to obtain a sound generation control program.

The sound generation device of the present invention includes an imaging unit attached to a vehicle and capturing at least a front image during traveling, and at least one of ambient environment information of the vehicle and physical condition information of an occupant during traveling of the vehicle. Acquiring means for analyzing an image captured by the image capturing means and acquiring surrounding environment information of the vehicle, and sound generating means for generating a sound corresponding to the environmental parameter acquired by the acquiring means as environmental parameters including: An output unit that outputs a sound generated by the sound generation unit to an occupant; and a first degree of change of an entire front image captured by the imaging unit as the surrounding environment information of the vehicle. A first calculating means for calculating a complexity parameter and a second complexity parameter which is a degree of change specialized for a vehicle side portion of the front image, The unit, the first based on the first complexity parameter and the second balance complexity parameter 2, which is computed by the computing means is characterized by changing the sound to be generated.
In addition, the sound generation device according to the present invention includes an imaging unit attached to the vehicle and capturing at least a forward image while the vehicle is traveling. An acquisition unit that analyzes an image captured by the imaging unit to acquire surrounding environment information of the vehicle, and generates a sound corresponding to the environment parameter acquired by the acquisition unit as an environment parameter including at least one of the following. A sound generation unit, an output unit that outputs a sound generated by the sound generation unit to an occupant, and, as surrounding environment information of the vehicle, a front image captured by the imaging unit, and A second calculating means for calculating a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance, wherein the sound generating means includes a second calculating means. A third complexity parameter calculated by the stage, on the basis of the relationship between the speed of the vehicle, is characterized by changing the sound to be generated.

  According to the present invention, while the vehicle is running, the acquisition unit acquires the environment parameters including at least one of the surrounding environment information of the vehicle and the physical condition information of the occupant. The sound generating means generates a sound corresponding to the environmental parameter acquired by the acquiring means, and the output means outputs a sound generated by the sound producing means to an occupant.

  Here, the sound generation unit generates a sound based on the environment parameters acquired by the acquisition unit. In other words, it is not a parameter based on the operation state for running the vehicle.

  That is, in the present invention, for example, since a sound for alerting is generated based on the surrounding environment information of the vehicle, the surrounding environment of the vehicle can be grasped by the output of the generated sound, and consideration is given to driving. can do.

  In the present invention, the acquisition unit acquires a complexity parameter representing complexity of surrounding environment information of the vehicle as the environment parameter.

  The environment parameter can express complexity representing the complexity of the surrounding environment information of the vehicle (complexity parameter).

  As a source of information for acquiring the surrounding environment, a front image while the vehicle is running is captured. In traveling of the vehicle, various surrounding environments can be acquired from the most important forward image.

As the ambient environment information before Symbol vehicle, first and complexity parameter is a total change degree of the captured forward image by the imaging means is the degree of change specific to the vehicle side of said front image A first complexity means for calculating a second complexity parameter; and the sound generation means, wherein the first complexity parameter and the second complexity parameter calculated by the first calculation means are provided. The sound to be generated is changed based on the balance between the two.

  A first complexity parameter and a second complexity parameter are calculated as surrounding environment information of the vehicle.

  The first complexity parameter is a degree of change of the entire front image captured by the imaging unit. Further, the second complexity parameter is a degree of change specialized for the vehicle side in the front image.

  The changing means changes the sound generation by the sound generating means based on the balance between the first complexity parameter and the second complexity parameter 2. Since the sound changes depending on the balance of different elements, the notification form can be made different by two-dimensional spread.

As the ambient environment information before Symbol vehicle, from the captured forward image by the imaging means, second calculating a third complexity parameter indicating the degree of change in obstacle within a distance determined in advance with respect to the vehicle 2 Calculating means, wherein the sound generating means changes the sound to be generated based on a relationship between the third complexity parameter calculated by the second calculating means and the speed of the vehicle. Features.

  A third complexity parameter is calculated as the surrounding environment information of the vehicle. The third complexity parameter is a degree of change of an obstacle existing within a predetermined distance from the vehicle from the front image captured by the imaging unit.

  The changing unit changes the sound to be generated based on the relationship between the third complexity parameter and the speed of the vehicle. The change in sound includes the volume. For example, it is possible to recognize a situation where there are many obstacles within a predetermined distance and the speed of the vehicle is slow, and this situation may be a narrow road with many people and the like. This can be an opportunity to emphasize the alert.

  In the present invention, the sound generation unit generates a sound by reading a plurality of sounds stored in advance, or generates a sound from a plurality of sounds that are automatically composed based on an image captured by the imaging unit, The sound change based on the environment parameter adjusts a synthesis degree of a plurality of sounds.

  Since the alert can be recognized through hearing, a change in the line of sight during driving can be reduced as compared with the alert through vision.

  The sound generation device of the present invention is attached to a vehicle, and captures at least a forward image while the vehicle is running, and reads out a plurality of sounds stored in advance to generate a synthesized sound. Sound generating means for generating a synthesized sound from a plurality of sounds automatically composed based on the captured image, a first complexity parameter that is a degree of change of the entire front image captured by the imaging means, A first complexity calculating means for calculating a second complexity parameter, which is a degree of change specialized for the side of the vehicle, and a predetermined distance to the vehicle from a front image captured by the imaging means. Second calculating means for calculating a third complexity parameter indicating a degree of change of an obstacle present within the first and second complexity parameters, and a first complexity parameter and a second complexity parameter calculated by the first calculating means Two roses First changing means for changing the degree of synthesis of a plurality of sounds synthesized by the sound generating means, a third complexity parameter calculated by the second calculating means, and a speed of the vehicle. A second changing unit that changes a volume of the synthesized sound generated by the sound generating unit based on the relationship between the second sound generating unit and an output unit that outputs a sound generated by the sound generating unit to an occupant; have.

  According to the present invention, in the first calculating means, the first complexity parameter, which is a degree of change of the entire front image picked up by the image pickup means, and a change specific to the side of the vehicle in the front image. A second complexity parameter, which is a degree, is calculated, and the sound generated by the sound generating means is changed based on a balance between the two (first changing means).

  The second calculating means calculates a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle from the front image captured by the imaging means, and calculates the third complexity parameter. The sound generated by the sound generating means is changed based on the relationship between the third complexity parameter and the speed of the vehicle (second changing means).

  As an example, while the first change is a timbre, a synthesis balance, a rhythm, or the like, the second changing unit sets the sound output by the first changing unit as a volume.

  By outputting the generated sound, a change in the surrounding environment is expressed as a change in the sound, and it is possible to draw an interest that another place has a change.

  For example, when driving in a rural area where there is nothing around, the sound is relatively quiet, while in a city with many buildings, multiple sounds are mixed or the volume is increased, so that the surrounding situation can be changed. It can be expressed by sound.

  Further, since the sound changes according to the surrounding environment, it is possible to avoid a monotonous driving that tends to be distracted.

  Further, the change in the sound makes it possible to intuitively grasp the surrounding situation and to concentrate the gaze on driving.

  The sound generation control program according to the present invention reads out a plurality of sounds stored in advance to a computer to generate a synthesized sound, or generates a synthesized sound from a plurality of sounds automatically composed based on an image captured by an imaging unit. A first complexity parameter which is a degree of change of the entire front image generated and imaged by the imaging means, and a second complexity parameter which is a degree of change specific to the vehicle side of the front image. And a third complexity parameter indicating the degree of change of an obstacle existing within a predetermined distance from the vehicle from the front image captured by the image capturing means, and The degree of synthesis of a plurality of synthesized sounds is changed based on the balance between the first complexity parameter and the second complexity parameter 2, and the relationship between the calculated third complexity parameter and the vehicle speed is changed. Based Te, change the volume of the generated synthesized speech, characterized in that to perform the.

  As described above, according to the present invention, the relationship between the surrounding environment during driving and the sound can be provided, and the output sound can have a function as an alert function when the driver drives.

It is a schematic diagram of a sound generation control device according to the present embodiment. FIG. 3 is a functional block diagram showing a sound generation process and a reproduction process in a sound generation control device executed in a RAM based on a control program in a block form. FIG. 3A is a front view of an image for obtaining a complexity parameter 1, FIG. 3B is a front view of an image for obtaining a complexity parameter 2, and FIG. FIG. 7 is an array diagram of numbers for specifying image frames divided by (). FIG. 3 is a characteristic diagram of a complexity parameter 1 to a complexity parameter 2 for determining a weight of a control coordinate point. FIG. 9 is a characteristic diagram showing a list of weight settings for each sound information file. It is a speed-complexity parameter 3 characteristic diagram for deriving the degree of danger during driving. FIG. 7 is a characteristic diagram illustrating a relationship between a risk and an amplification factor derived in FIG. 6. It is a flow chart which shows a sound generation control routine concerning this embodiment which starts when IG is turned on. 9 is a flowchart showing a transition state of information (signal) processing in sound generation control executed based on the flowchart of FIG. 14 is a flowchart showing a transition state of information (signal) processing when an automatic music program is applied according to a modification of the present embodiment.

  FIG. 1 shows a schematic diagram of a sound generation control device 10 as an alert control device of the present invention.

  As shown in FIG. 1, the sound generation control device 10 includes a microcomputer 12 that performs various controls and various calculations. The microcomputer 12 includes a CPU 12A, a RAM 12B, a ROM 12C, an input / output unit (I / O) 12D, and a bus 12E such as a data bus or a control bus connecting these.

  The hard disk 14 is connected to the I / O 12D as a nonvolatile memory. Note that another nonvolatile memory such as a USB memory or an SD memory may be connected to the I / O 12D via an I / F (not shown).

  The I / O 12D is connected to the imaging unit 16 as an input device. In the present embodiment, the imaging unit 16 is configured by a camera that images the front of the vehicle. Note that the camera applied as the imaging unit 16 may be dedicated to the sound generation control device 10 of the present embodiment, or may be an existing in-vehicle camera for covering a blind spot area or the like during driving. It suffices if at least image information that captures the front of the vehicle while traveling can be acquired.

  The imaging unit 16 includes an image expressing a temperature difference due to infrared rays and an image expressing a distance to a target using sonar (ultrasonic waves), as well as an image expressed by a so-called general camera using colors and gradations. Means that.

  A speaker 20 is connected to the I / O 12D via a driver 18 as an output system device. In the sound generation control device 10 according to the present embodiment, a sound according to the surrounding environment during running of the vehicle is selected and output from the speaker 20 (details will be described later). The speaker 20 may be dedicated to the sound generation control device 10 of the present embodiment, or a speaker attached to an audio device mounted on a vehicle may be used.

  Here, a control program for executing sound generation processing and sound reproduction processing for sound generation control is stored in the ROM 12C. The CPU 12A reads the control program stored in the ROM 12C, and executes the control program using, for example, the RAM 12B as a work area. Here, the software is synonymous with the control program.

  FIG. 2 is a functional block diagram showing, in a block diagram, a sound generation process and a reproduction process in the sound generation control device 10 executed in the RAM 12B based on the control program. The block shown in FIG. 2 does not limit the hardware configuration of the sound generation control device 10.

  The imaging unit 16 is connected to the image acquisition unit 22. For example, a signal (for example, an ignition (IG) signal or the like) indicating the running state of the vehicle is input to the image acquisition unit 22, and when the IG signal is recognized as being turned on, image information is output from the imaging unit 16. To get.

  The image acquisition unit 22 is connected to the sound information file reading unit 24 and the image analysis unit 26.

  The image acquisition unit 22 outputs an instruction signal for instructing the reading of the sound information file to the sound information file reading unit 24 in response to the input of the IG signal. In response to the input of the instruction signal, the sound information file reading unit 24 reads the sound information file from the sound information file storage unit 28 and outputs the read sound information file to the reproducing unit 30.

  The sound information file storage unit 28 stores four types of sound information files 1 to 4 (see FIGS. 4 and 9) in advance. In the present embodiment, as the types of the sound information file 1 to the sound information file 4, sounds expressing nature (river, wind, insect, etc.), sounds expressing seasonal features (festival, fireworks, etc.), and musical instruments Sound and the like.

  Although the sound information file 1 to the sound information file 4 do not necessarily have to be selected from the same sound type, in the present embodiment, for example, the sound information file 1 is specialized for sounds expressing nature, Different timbres from the same type of sound, such as wind sound in the sound information file 2, insect sound in the sound information file 3, and thunder in the sound information file 4, are stored.

  As a default, the reproducing unit 30 reproduces all kinds of timbres at a predetermined cycle and a predetermined volume, respectively, and outputs the reproduced sounds from the speaker 20 via the driver 18.

  On the other hand, as shown in FIG. 2, an image analysis unit 26 connected to the image acquisition unit 22 analyzes the captured image information.

  In the image analysis unit 26, as a basic process, the captured image is converted to gray scale to perform edge detection.

  The gray scale conversion may be, for example, black-and-white data subjected to binarization processing or density data (for example, 8-bit gradation data or the like). Note that the black-and-white data is less accurate in the degree of change because the amount of information is smaller than the density data, but the analysis processing speed can be reduced. In the present embodiment, black-and-white data is used, but selection may be made according to imaging conditions (region, time zone, etc.).

  The image analyzer 26 is connected to the complexity parameter generator 32. The complexity parameter generation unit 32 generates information for extracting a complexity parameter 1, a complexity parameter 2, and a complexity parameter 3 as environment parameters.

(Complexity parameter 1)
The complexity parameter 1 generated by the complexity parameter generation unit 32 is information for monitoring the degree of change of the entire image.

  When generating the complexity parameter 1, black and white data generated as a basic process is acquired (see FIG. 3A), and a difference (absolute value) of temporally preceding and succeeding image information is calculated (difference calculation). An average value is calculated when the difference calculation is repeated several tens of times (for example, about 60 times) according to the traveling (moving) of. With the complexity parameter 1, it is possible to recognize a change in the entire image captured while the vehicle is running.

  The complexity parameter 1 is sent to the weight determination unit 34.

(Complexity parameter 2)
The complexity parameter 2 generated by the complexity parameter generation unit 32 is information for mainly monitoring the degree of change of the left and right ends of the traveling direction of the vehicle.

  When generating the complexity parameter 2, black and white data (absolute value) generated as basic processing is obtained, and the obtained image of the black and white data is divided into a plurality of image frames. For example, as shown in FIG. 3C, the image is divided into 9 × 5 and each image frame is numbered from upper left to lower right.

  Among the divided image frames, the image frames corresponding to the left and right of the vehicle, which are the image frames No. in FIG. 21 and no. 26, and the average of the absolute values of the differences from No. 26, 25 and no. The average value of the absolute value of the difference from 30 is extracted. And No. 21 and no. 26, and the average of the absolute values of the differences from No. 26, 25 and no. The average of the absolute value of the difference from 30 is calculated (average value calculation), and the average obtained when the average value calculation is repeated several tens of times (for example, about 40 times) according to the traveling (movement) of the vehicle. Calculate the value.

  With the complexity parameter 2, it is possible to recognize a change in the side, in particular, while the vehicle is running. The complexity parameter 2 is sent to the weight determination unit 34.

(Weighting by complexity parameters 1 and 2)
As shown in FIG. 2, the weight determination unit 34 weights four types of sound information files (sound information files 1 to 4) based on the complexity parameter 1 and the complexity parameter 2, respectively. .

  FIG. 4 is a characteristic diagram for specifying a current control coordinate point (x, y) of an image captured by the imaging unit 16 when the complexity parameter 1 is set on the horizontal axis and the complexity parameter 2 is set on the vertical axis. It is. That is, the control coordinate point is determined by the balance between the complexity parameter 1 and the complexity parameter 2.

  The sound information file 1 to the sound information file 4 are respectively located at four corners of the characteristic diagram of FIG. Weights 1 to 4 set in the sound information files 1 to 4 correspond to the distances from the respective sound information files (sound information files 1 to 4) to the control coordinate points (x, y). ing.

  The sound information file 1 is arranged at a position where the complexity parameter 1 is minimum and the complexity parameter is minimum (xmin, ymin). As shown in FIG. 5, the sound information file 1 has such a characteristic that the smaller the complexity parameter 1, the stronger the weight, and the smaller the complexity parameter 2, the stronger the weight.

  In the sound information file 2, the complexity parameter 1 is arranged at the minimum position and the complexity parameter is arranged at the maximum position (xmin, ymax). As shown in FIG. 5, the sound information file 2 has such a characteristic that the smaller the complexity parameter 1 is, the stronger the weight is, and the larger the complexity parameter 2 is, the stronger the weight is.

  In the sound information file 3, the complexity parameter 1 is arranged at the maximum position and the complexity parameter is arranged at the maximum position (xmax, ymax). As shown in FIG. 5, the sound information file 3 has such a characteristic that the larger the complexity parameter 1, the stronger the weight, and the larger the complexity parameter 2, the stronger the weight.

  In the sound information file 4, the complexity parameter 1 is located at the maximum position and the complexity parameter is located at the minimum position (xmax, ymin). As shown in FIG. 5, the sound information file 4 has such a characteristic that the larger the complexity parameter 1, the stronger the weight, and the smaller the complexity parameter 2, the stronger the weight.

  As shown in FIG. 2, the weight determination unit 34 is connected to the weight adjustment unit 36 and sends out the determined weight information to the weight adjustment unit 36. The weight adjusting unit 36 performs weighting on the sound information files 1 to 4 reproduced by the reproducing unit 30 based on the weight information for which the weights are determined. As a result, the sound information files 1 to 4 are output with the degree of enhancement (period, volume, etc.) at the time of output changed based on the complexity parameter 1 and the complexity parameter 2.

  For example, when each value of the complexity parameter 1 and the complexity parameter 2 is small, the sound information file 1 has the strongest weight and is emphasized.

(Complexity parameter 3)
The complexity parameter 3 generated by the complexity parameter generation unit 32 is information for monitoring the presence or absence of an obstacle existing near the vehicle.

  When generating the complexity parameter 3, a reference line A (see FIG. 6) of the characteristic curve of the speed-complexity parameter 3 is set.

  The reference line A shown in FIG. 6 is a characteristic curve assuming that the complexity parameter 3 changes linearly with respect to the speed of the vehicle. The difference between the reference line A and the actual characteristic curve is the characteristic curve of the vehicle. This is information for monitoring the presence or absence of a nearby obstacle.

  For example, when the vehicle is traveling on a road with few obstacles, the actual speed-complexity parameter 3 characteristic curve becomes the characteristic curve B, and there is no element exceeding the reference line A.

  On the other hand, when the vehicle is traveling on a road with many obstacles, the actual speed-complexity parameter 3 characteristic curve becomes a characteristic curve C, and an element exceeding the reference line A exists (see the hatched area D in FIG. 6). ). The situation in which the hatched area D occurs is when the vehicle is on the premises of a parking lot, the road is narrow, telephone poles and vending machines are protrudingly installed on the road side, and there are many people. No.

  The complexity parameter 3 generated by the complexity parameter generation unit 32 is sent to the risk determination unit 38. The risk determination unit 38 determines the current risk of the vehicle by monitoring whether or not the hatched area D in FIG. 6 has occurred. The situation where the degree of danger during driving is high is not limited to the above.

  As shown in FIG. 2, the risk determination unit 38 is connected to the gain adjustment unit 40. The gain adjustment unit 40 is connected to the reproduction unit 30. That is, when the risk determination unit 38 sends a numerical value indicating the risk to the gain adjustment unit 40, the gain adjustment unit 40 performs a risk-gain (here, volume) characteristic diagram as shown in FIG. , The gain adjustment amount is specified. The reproducing unit 30 adjusts the gain (amplification rate) of the synthesized sound of the sound information files 1 to 4 based on the specified gain adjustment amount.

  For example, if the vehicle is on a narrow road and a telephone pole or vending machine is protrudingly installed on the road side, or if you are traveling in a place with a lot of traffic, increase the volume above the default volume, It is designed to alert. In the present embodiment, the gain is set to the volume. For example, the reproduction pitch amount (increase the reproduction pitch when the degree of risk is high), the scale (increase the scale when the degree of risk is high) and the like are adjusted. It may be.

  Hereinafter, the operation of the present embodiment will be described.

  FIG. 8 is a flowchart illustrating a sound generation control routine according to the present embodiment that is started when the IG is turned on. FIG. 9 is a flowchart showing a transition state of information (signal) processing in sound generation control executed based on the flowchart of FIG.

  As shown in FIG. 8, when the IG is turned on, in step 100, the imaging unit 16 is activated to start imaging.

  In the next step 102, the sound information file 1 to the sound information file 4 stored in the sound information file storage unit 28 are read, and then the process proceeds to step 104 to start the reproduction process. That is, the sound sources of the sound information files 1 to 4 read out in step 102 are synthesized and repeatedly output from the speaker.

  In the next step 106, the image captured by the imaging unit 16 is analyzed. In the present embodiment, the captured image is converted to grayscale (black and white data), and edge detection is performed (see FIG. 3).

  In the next step 108, a complexity parameter 1 is calculated from the information subjected to the image analysis.

  That is, the complexity parameter 1 calculates the absolute value of the difference between temporally preceding and succeeding image information based on the black and white data, and calculates the average value when the difference calculation is repeated according to the running of the vehicle.

  In the next step 110, a complexity parameter 2 is calculated from the information subjected to the image analysis.

  That is, the complexity parameter 2 is an average of the absolute value of the difference between the image frames corresponding to the left and right of the vehicle from among the image frames obtained by dividing the image of the monochrome data into a plurality of image frames (for example, 9 × 5). A value is calculated, and an average value when the average value calculation is repeated according to the running of the vehicle is calculated.

  In the next step 112, a weight is determined based on the complexity parameter 1 obtained in step 108 and the complexity parameter 2 obtained in step 110. The weight is determined according to the distance between the position of the control coordinate point (x, y) and the coordinate point of the sound information file (see FIGS. 4 and 5).

  In the next step 114, a weighting process is performed on each of the sound information files 1 to 4 based on the determined weights, and the process proceeds to step 116.

  In step 116, a complexity parameter 3 is calculated from the information analyzed.

That is, the complexity parameter 3 is information for monitoring the presence or absence of an obstacle present near the vehicle, and is a speed-complexity parameter 3 characteristic curve when traveling on a road with many obstacles (see FIG. 6 ). , The information is used to determine whether or not there is an area (hatched area D) exceeding the reference line A.

  In the case of the characteristic curve C in FIG. 6, it means that the degree of danger during driving is high. The specific situation is as follows.

  (Situation 1) The vehicle is on the premises of the parking lot.

  (Situation 2) The road is narrow and telephone poles and vending machines are installed protruding toward the road.

  (Situation 3) Roads are narrow and there are many people.

  The situation where the degree of danger during driving is high is not limited to the above situations 1 to 3.

  In the next step 118, a gain (amplification factor) is determined based on the size (area, maximum value, etc.) of the hatched area D (see FIG. 6) determined by the complexity parameter 3 calculated in step 116. The process proceeds to 120 to execute the gain adjustment processing. By this gain adjustment processing, the higher the degree of danger, the greater the volume of sound flowing from the loudspeaker, or the faster the playback pitch of the sound information file.

  In the next step 122, it is determined whether or not the IG has been turned off. If a negative determination is made, the process returns to step 106 and the above steps are repeated. If an affirmative determination is made in step 122, the process proceeds to step 124 to end the sound generation control, stops imaging and sound reproduction, and ends this routine.

(Modification)
In this embodiment, four types of sound information files 1 to 4 are stored in the sound information file storage unit 28 (see FIG. 2) in advance, and the sound information file storage unit 28 The file 1 to the sound information file 4 are read and reproduced.

  On the other hand, in the modified example, the automatic music file 1 to the automatic music file 4 are applied as four types of sound information files.

  The automatic composition file 1 to the automatic composition file 4 are composed each time by executing the automatic composition program based on the image information captured by the imaging unit 16.

  The automatic composition program is stored in the ROM 12C of FIG. 1 and executed by the CPU 12A.

  In the present embodiment, “software for converting MIDI (Musical Instrument Digital Interface) signals” and “software for generating music information from MIDI signals” are stored in the ROM 12C.

  MIDI is an international standard for communicating music information by digital signals. The MIDI signal is information of the MIDI standard, and is information of a standard MIDI file format (SMF: Standard MIDI File Format). The "software for MIDI signal conversion" uses "note number" for the basic frequency of the sound, "velocity level" for the sound pressure level, "number representing the instrument name" for the tone, and "note on / off" for the time change. Are converted to data representing "."

  More specifically, in the automatic music file 1 to the automatic music file 4, sound sources of musical instruments serving as reference sounds, for example, sound sources such as a piano, a guitar, a violin, and a trumpet are set.

  Image information picked up by the image pickup unit 16 is color-separated into RGB to obtain respective brightness (density) information.

  Based on the brightness information of each color, three elements of melody (melody), harmony (harmony), rhythm (rhythm), height (frequency), size (sound pressure level), timbre (frequency component) Adjust and compose the three elements of the sound.

  The applicants have filed an application for an automatic music composition that generates sound from imaging information (camera image) of the imaging unit 16 (for example, Japanese Patent Application Laid-Open No. 2017-102863).

  FIG. 10 is a flowchart showing a transition state of information (signal) processing in sound generation control according to a modification of the present embodiment.

  In the automatic composition program, when the imaging information captured by the imaging unit 16 is input, the automatic music is executed, and the automatic music files 1 to 4 are created. By replacing the automatic music file 1 to the automatic music file 4 with the sound information file 1 to the sound information file 4 shown in FIG. 9, this modification is shown in FIG. Processed by the same flow as.

  That is, the weight of the automatic music file 1 to the automatic music file 4 at the time of reproduction is adjusted by the complexity parameter 1 and the complexity parameter 2, and the automatic music file 1 to the automatic music file 4 is adjusted by the complexity parameter 3. The gain at the time of reproduction is adjusted, and the same effect as in the present embodiment can be obtained.

  In the present embodiment (including the modifications), the complexity parameter 1 to the complexity parameter 3 are obtained based on the imaging information of the imaging unit 16, but in addition to the imaging unit 16 or the imaging unit Instead of 16, the complexity parameter 1 to the complexity parameter 3 may be obtained by monitoring the driver's physical condition (heart rate, blood flow value, eyeball movement, etc.).

  The complexity parameter 1 to the complexity parameter 3 may be corrected by monitoring not only the driver but also the physical condition of the occupant (presence of an infant, sleeping, etc.). For example, in a passenger compartment where an infant is present or a sleeping occupant is present, adjustments may be made such that the degree of change in sound is gradual or the pitch of sound is slowed.

  Further, in the present embodiment (including the modification), the output sound from the speaker is changed based on the complexity parameter 1 to the complexity parameter 3, but the output target is not only the sound, but also the mounted on the vehicle. May be associated with the control of the ambient lighting performed.

Reference Signs List 10 sound generation control device 12 microcomputer (sound generation means, first calculation means, second calculation means, first change means, second change means)
12A CPU
12B RAM
12C ROM
12D input / output unit (I / O)
12E bus 14 hard disk 16 imaging unit (acquisition means)
18 Driver 20 Speaker (output means)
Reference Signs List 22 image acquisition unit 24 sound information file reading unit 26 image analysis unit 28 sound information file storage unit 30 playback unit 32 complexity parameter generation unit 34 weight determination unit 36 weight adjustment unit 38 risk degree determination unit 40 gain adjustment unit

Claims (6)

Mounted on a vehicle includes an imaging means for capturing an anterior image of at least running, during running of the vehicle, the ambient environment information of the vehicle, and, as the environmental parameters including at least one of the occupant's body condition information, the image pickup Analysis means for analyzing the image captured by the means, to obtain the surrounding environment information of the vehicle ,
Sound generating means for generating a sound corresponding to the environmental parameter acquired by the acquiring means,
Output means for outputting a sound generated by the sound generation means to an occupant;
As the surrounding environment information of the vehicle, a first complexity parameter that is a degree of change of the entire front image captured by the imaging unit, and a first degree of change that is a degree of change specific to the side of the vehicle in the front image. And first computing means for computing the complexity parameter of 2 and
The sound generation device changes a sound to be generated based on a balance between the first complexity parameter and the second complexity parameter 2 calculated by the first calculation device. . An image pickup unit attached to the vehicle for taking at least an image of the front of the vehicle while the vehicle is traveling; Analysis means for analyzing the image captured by the means, to obtain the surrounding environment information of the vehicle,
Sound generation means for generating a sound according to the environmental parameter acquired by the acquisition means,
Output means for outputting a sound generated by the sound generation means to an occupant;
Calculating a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle from a front image captured by the imaging unit as the surrounding environment information of the vehicle; Computing means,
The sound generation device changes a sound to be generated based on a relationship between the third complexity parameter calculated by the second calculation unit and a vehicle speed . The sound generation device according to claim 1, wherein the acquisition unit acquires a complexity parameter representing complexity of surrounding environment information of the vehicle as the environment parameter . The sound generating means reads out a plurality of sounds stored in advance and generates a sound, or generates a sound from a plurality of sounds automatically composed based on an image captured by the imaging means,
The sound generator according to any one of claims 1 to 3, wherein the change of the sound based on the environment parameter adjusts a degree of synthesis of a plurality of sounds . Imaging means attached to the vehicle, for capturing at least a forward image while traveling,
A sound generation unit that reads out a plurality of sounds stored in advance to generate a synthesized sound, or generates a synthesized sound from a plurality of sounds that are automatically composed based on an image captured by the imaging unit;
A first complexity parameter that is a degree of change of the entire front image captured by the imaging unit, and a second complexity parameter that is a degree of change specific to the vehicle side in the front image. First calculating means for calculating;
A second calculating unit configured to calculate a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle from the front image captured by the imaging unit;
Changing a synthesis degree of a plurality of sounds synthesized by the sound generation unit based on a balance between the first complexity parameter and the second complexity parameter 2 calculated by the first calculation unit; Change means;
Second changing means for changing the volume of the synthesized sound generated by the sound generating means based on a relationship between the third complexity parameter calculated by the second calculating means and the speed of the vehicle;
Output means for outputting a sound generated by the sound generation means to an occupant;
A sound generation device having:   On the computer,
  Generating a synthesized sound by reading a plurality of sounds stored in advance, or generating a synthesized sound from a plurality of sounds automatically tuned based on an image taken by the imaging means,
  A first complexity parameter that is a degree of change of the entire front image captured by the imaging unit, and a second complexity parameter that is a degree of change specific to the vehicle side in the front image. Calculate,
  Calculating a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle from the front image captured by the imaging unit;
  Changing the degree of synthesis of a plurality of synthesized sounds based on the calculated balance between the first complexity parameter and the second complexity parameter 2;
  Changing the volume of the generated synthetic sound based on the relationship between the calculated third complexity parameter and the speed of the vehicle;
  A sound generation control program for causing the user to execute a task.