JP2019020638A - Sound generation device, sound generation control program - Google Patents

Abstract

To make a sound generation device and a sound generation control program have a relationship between surrounding environment during driving and sound, and a function as a reminder function to the output sound when a driver drives.SOLUTION: A complexity parameter 1 is information for monitoring the degree of change in the entire image captured by an image capturing unit 16 during traveling of a vehicle. A complexity parameter 2 is information for intensively monitoring the degree of change in the right and left ends of the travelling direction of the vehicle. Weights 1 to 4 set in a sound information file 1 to a sound information file 4 correspond to the distance from each sound information file to a control coordinate point, and weighting is executed. A complexity parameter 3 is information for monitoring the presence or absence of an obstacle being present in the vicinity of the vehicle. If the vehicle is within the parking lot premises, the road is narrow, a telegraph pole and a vending machine are installed protruding to the road side, and the road is narrow and the human traffic is high, the risk is high and the amplification factor is increased (increase the output volume).SELECTED DRAWING: Figure 9

Description

  The present invention relates to a sound generation device and a sound generation control program for alerting a driver while a vehicle is traveling.

  Conventionally, an engine simulation sound generating device that generates engine simulation sound has been proposed to enhance the sense of presence in a racing game, a drive simulator, or the like, or to generate an engine sound corresponding to an actual running state in an electric vehicle. Has been.

  Patent Document 1 describes that engine simulation sound is generated based on a map formed by assigning sound pressure waveform data of engine sound within a predetermined time to a rotational speed and an accelerator operation amount.

  In Patent Document 2, the engine operating state is divided into a plurality of ranges using the driving operation amount and the engine rotation speed as parameters, and engine sounds recorded in a substantially central state of each range are recorded for one fuel cycle on the crankshaft. Stored as engine sound digital data in units of a sound pressure waveform with a length corresponding to the rotation time, and repeatedly sounding engine sound data in the driving state range according to the input of the operation amount and engine speed Have been described.

  In the sound generation device for conventional vehicles including Patent Document 1 and Patent Document 2 described above, the sound related to traveling of the vehicle or the like corresponds to the four corners of the two-dimensional parameters (two-dimensional coordinates) of the engine speed and the accelerator operation amount. A sample of the engine sound to be prepared is prepared, and the sound at the coordinate point corresponding to the detected engine speed and accelerator operation amount is complemented and output.

  The complementing means changes the sound reproduction speed according to the engine speed and the accelerator operation amount. Further, a weight corresponding to the engine speed and the accelerator operation amount is set, and the sound volume is changed.

  On the other hand, Japanese Patent Application Laid-Open No. H10-33095 describes that the music piece is automatically operated according to the surrounding environment. That is, in Patent Document 3, music is generated by simultaneously playing a plurality of parts. The sound of each part sounds according to the reaction of the image or the radio wave sensor.

  Further, in Patent Document 4, a risk determination unit determines a driving risk based on a signal from a speed sensor, generates a sound scale in music to be generated, a chord to sound in music, a timing to sound an instrument, a musical instrument It is described to change at least one of the performance speeds.

  For reference, a vehicle music generator that generates music using music data and vehicle state detection results is disclosed (Patent Document 5).

  Further, as a reference, a signal performance music performance device that reflects the monitoring work to the auditory sense by converting the change of the monitoring target into the change of music based on the external signal input from the monitoring target displayed on the monitor. Is disclosed (Patent Document 6).

  As a reference, an acoustic component apparatus that automatically reproduces using external environment or human biological information is disclosed (see Patent Document 7).

JP 2000-1142 A JP 2000-10576 A JP, 2006-183988, A Japanese Patent Laid-Open No. 2006-102863 Japanese Patent No. 4984861 Japanese Patent No. 4458581 Japanese Patent No. 3381074

  For example, a vehicle with high quietness has a structure that blocks engine noise in the passenger compartment as much as possible. In an electric vehicle, engine sound does not exist while the drive source of the vehicle is changed from an engine to a motor or the like. For example, the sound output target is effective for the purpose of calling attention to a person walking around a vehicle or a person traveling on a bicycle. For a driver who is driving a car, there is a possibility that it may be annoying while driving.

  On the other hand, as a part of safe driving of the vehicle, there is a demand for the driver to recognize the surrounding environment.

  However, although there is a sound generation technique using the operation state of the vehicle as a parameter, the relationship between the ambient environment during driving and the sound is not considered.

  In consideration of the above facts, the present invention has a relationship between sound and the surrounding environment during driving, and a sound generation device capable of providing a function as a warning function when the driver drives the output sound. The object is to obtain a sound generation control program.

  The sound generation device of the present invention includes an acquisition unit that acquires environmental parameters including at least one of ambient environment information of the vehicle and physical condition information of an occupant while the vehicle is traveling, and the environment acquired by the acquisition unit Sound generating means for generating a sound corresponding to the parameter; and output means for outputting the sound generated by the sound generating means to the occupant.

  According to the present invention, the acquisition means acquires environmental parameters including at least one of ambient environment information of the vehicle and occupant's physical condition information while the vehicle is traveling. The sound generation unit generates a sound according to the environmental parameter acquired by the acquisition unit, and the output unit outputs the sound generated by the sound generation unit to the occupant.

  Here, the sound generation means generates sound based on the environmental parameters acquired by the acquisition means. In other words, it is not a parameter based on the operation state for traveling of the vehicle.

  That is, in the present invention, for example, a sound for alerting is generated based on the surrounding environment information of the vehicle, so that the surrounding environment of the vehicle can be grasped by the output of the generated sound, and driving is considered. can do.

  In the present invention, the acquisition unit acquires a complexity parameter representing the complexity of the surrounding environment information of the vehicle as the environment parameter.

  The environment parameter can express the complexity representing the complexity of the surrounding environment information of the vehicle (complexity parameter).

  In the present invention, the acquisition unit includes an imaging unit that is attached to the vehicle and captures at least a forward image during traveling, and analyzes the image captured by the imaging unit to acquire ambient environment information of the vehicle. It is characterized by that.

  As an information source for acquiring the surrounding environment, a front image while the vehicle is traveling is captured. In traveling of the vehicle, various surrounding environments can be acquired from the most important front image.

  In the present invention, as the surrounding environment information of the vehicle, a first complexity parameter that is a degree of change of the entire front image captured by the imaging unit, and a change specific to the vehicle side portion of the front image. And a first calculation means for calculating a second complexity parameter that is a degree, and the sound generation means includes a first complexity parameter calculated by the first calculation means and a second complexity parameter. The sound to be generated is changed based on the balance of the complexity parameter 2.

  As the surrounding environment information of the vehicle, a first complexity parameter and a second complexity parameter are calculated.

  The first complexity parameter is the degree of change of the entire front image captured by the imaging unit. Further, the second complexity parameter is a degree of change specialized for the vehicle side portion in the front image.

  The changing means changes the sound generation by the sound generating means based on the balance between the first complexity parameter and the second complexity parameter 2. Since the sound changes depending on the balance of different elements, the notification form can be varied by two-dimensional expansion.

  In the present invention, as a surrounding environment information of the vehicle, a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle is calculated from a front image captured by the imaging unit. The sound generation means changes the sound to be generated based on the relationship between the third complexity parameter calculated by the second calculation means and the vehicle speed. It is characterized by doing.

  A third complexity parameter is calculated as the ambient environment information of the vehicle. The third complexity parameter is a degree of change of an obstacle existing within a predetermined distance from the front image captured by the imaging unit with respect to the vehicle.

  The changing means changes the sound to be generated based on the relationship between the third complexity parameter and the vehicle speed. The sound change includes the volume. For example, it is possible to recognize a situation where there are many obstacles within a predetermined distance and the speed of the vehicle is slow, and this situation can be considered a narrow road with many people, etc. It can be an opportunity to emphasize attention.

  In the present invention, the sound generation means generates a sound by reading a plurality of sounds stored in advance, or generates a sound from a plurality of sounds that are automatically tuned based on an image captured by the imaging means, The sound change based on the environmental parameter adjusts the degree of synthesis of a plurality of sounds.

  Since attention can be recognized through hearing, a change in line of sight while driving can be reduced as compared with visual attention.

  The sound generation device of the present invention is attached to a vehicle and captures at least a forward image while traveling, and generates a synthesized sound by reading a plurality of prestored sounds, or is captured by the imaging unit. Sound generating means for generating a synthesized sound from a plurality of sounds that are automatically tuned based on the obtained image, a first complexity parameter that is a degree of change of the entire front image captured by the imaging means, A first calculation unit that calculates a second complexity parameter that is a degree of change specialized for the vehicle side portion, and a predetermined distance from the front image captured by the imaging unit with respect to the vehicle A second calculation means for calculating a third complexity parameter indicating a degree of change of an obstacle existing within the first complexity parameter, and a first complexity parameter and a second complexity parameter calculated by the first calculation means. Two roses A first changing means for changing the degree of synthesis of a plurality of sounds synthesized by the sound generating means, a third complexity parameter calculated by the second calculating means, and a vehicle speed A second changing means for changing the volume of the synthesized sound generated by the sound generating means, and an output means for outputting the sound generated by the sound generating means to the occupant, have.

  According to the present invention, in the first calculation means, the first complexity parameter, which is the degree of change in the entire front image captured by the imaging means, and the change specific to the vehicle side portion in the front image. A second complexity parameter, which is a degree, is calculated, and the sound generated by the sound generating means is changed based on a balance between the two (first changing means).

  The second computing means computes a third complexity parameter indicating the degree of change of the obstacle existing within a predetermined distance from the vehicle from the front image taken by the imaging means, The sound generated by the sound generating means is changed based on the relationship between the complexity parameter 3 and the speed of the vehicle (second changing means).

  As an example, while the first change is a timbre, synthesis balance, rhythm, etc., the second change means uses the sound output by the first change means as the volume.

  By outputting the generated sound, a change in the surrounding environment is expressed as a change in the sound, and it is possible to attract the interest that there is a change in another place.

  For example, when driving in the countryside where there is nothing around, the sound will be relatively quiet, and when driving in a city with many buildings, multiple sounds will be mixed or the volume will be increased. It can be expressed with sound.

  In addition, since the sound changes according to the surrounding environment, it is possible to avoid monotonous driving that tends to be distracting.

  Furthermore, the surrounding situation can be grasped sensuously by the change of sound, and the line of sight can be concentrated on driving.

  The sound generation control program of the present invention reads a plurality of sounds stored in advance in a computer to generate a synthesized sound, or generates a synthesized sound from a plurality of sounds that are automatically composed based on an image captured by an imaging means. A first complexity parameter that is the degree of change of the entire front image that is generated and imaged by the imaging means, and a second complexity parameter that is a degree of change specific to the vehicle side portion of the front image And a third complexity parameter indicating the degree of change of an obstacle existing within a predetermined distance from the vehicle is calculated from the front image captured by the imaging unit, and the calculated first The degree of synthesis of a plurality of sounds to be synthesized is changed based on the balance between the complexity parameter 1 and the second complexity parameter 2, and the relationship between the calculated third complexity parameter and the vehicle speed is changed. Base Te, change the volume of the generated synthesized speech, characterized in that to perform the.

  As described above, according to the present invention, it is possible to provide a relationship between the surrounding environment during driving and sound, and to provide a function as a warning function when the driver drives the output sound.

It is the schematic of the sound production | generation control apparatus which concerns on this Embodiment. It is a functional block diagram which blocks and shows the sound production | generation process and reproduction | regeneration processing in the sound production | generation control apparatus performed with RAM based on a control program. FIGS. 3A and 3B are images captured by an imaging unit, where FIG. 3A is a front view of an image for obtaining complexity parameter 1, FIG. 3B is a front view of an image for obtaining complexity parameter 2, and FIG. ) Is an array diagram of numbers for specifying the image frames divided in (4). It is a complexity parameter 1-complexity parameter 2 characteristic view for determining the weight of a control coordinate point. It is a characteristic figure which shows the list of the weight setting for every sound information file. It is a speed-complexity parameter 3 characteristic view for deriving the risk during operation. FIG. 7 is a characteristic diagram showing the relationship between the degree of risk derived in FIG. 6 and the amplification factor. It is a flowchart which shows the sound production | generation control routine based on this Embodiment started when IG is ON. FIG. 9 is a flowchart showing a transition state of information (signal) processing in sound generation control executed based on the flowchart of FIG. 8. It is a flowchart which shows the transition state of the information (signal) process at the time of applying the automatic music program concerning the modification of this Embodiment.

  FIG. 1 shows a schematic diagram of a sound generation control device 10 as an attention control device of the present invention.

  As shown in FIG. 1, the sound generation control device 10 includes a microcomputer 12 that performs various controls and various calculations. The microcomputer 12 includes a CPU 12A, a RAM 12B, a ROM 12C, an input / output unit (I / O) 12D, and a bus 12E such as a data bus and a control bus for connecting them.

  A hard disk 14 is connected to the I / O 12D as a nonvolatile memory. Note that another non-volatile memory such as a USB memory or an SD memory may be connected to the I / O 12D via an I / F (not shown).

  The I / O 12D is connected to an imaging unit 16 as an input system device. In the present embodiment, the imaging unit 16 is configured by a camera that images the front of the vehicle. The camera applied as the imaging unit 16 may be dedicated to the sound generation control device 10 of the present embodiment, or may be an existing in-vehicle camera for covering a blind spot area or the like during driving. It suffices that at least image information obtained by imaging the front of the vehicle while traveling is acquired.

  The imaging unit 16 includes not only an image expressed by colors and gradations by a so-called general camera, but also an image expressing a temperature difference by infrared rays and an image expressing a distance to an object by sonar (ultrasound). Means that.

  A speaker 20 is connected to the I / O 12D via a driver 18 as an output system device. In the sound generation control device 10 according to the present embodiment, a sound corresponding to the surrounding environment during travel of the vehicle is selected and output from the speaker 20 (details will be described later). The speaker 20 may be dedicated to the sound generation control device 10 of the present embodiment, or may be a speaker attached to an audio device mounted on the vehicle.

  Here, the ROM 12C stores a control program for executing sound generation processing and sound reproduction processing for sound generation control. The CPU 12A reads the control program stored in the ROM 12C, and executes the control program using, for example, the RAM 12B as a work area. Here, software is synonymous with control program.

  FIG. 2 is a functional block diagram showing in block form the sound generation processing and reproduction processing in the sound generation control device 10 executed by the RAM 12B based on the control program. Note that the block shown in FIG. 2 does not limit the hardware configuration of the sound generation control device 10.

  The imaging unit 16 is connected to the image acquisition unit 22. For example, a signal indicating the running state of the vehicle (for example, an ignition (IG) signal or the like) is input to the image acquisition unit 22, and image information from the imaging unit 16 is recognized when the IG signal is turned on. To get.

  The image acquisition unit 22 is connected to the sound information file reading unit 24 and the image analysis unit 26.

  The image acquisition unit 22 outputs an instruction signal instructing the sound information file reading unit 24 to read the sound information file with the input of the IG signal. In response to the input of the instruction signal, the sound information file reading unit 24 reads the sound information file from the sound information file storage unit 28 and outputs it to the reproduction unit 30.

  In the sound information file storage unit 28, four types of sound information files 1 to 4 (see FIGS. 4 and 9) are stored in advance. In the present embodiment, as the types of the sound information file 1 to the sound information file 4, sounds expressing nature (rivers, winds, insects, etc.), sounds expressing seasonal features (festivals, fireworks, etc.), and musical instruments Sound and so on.

  The sound information file 1 to the sound information file 4 do not necessarily have to be selected from the same sound type, but in the present embodiment, for example, the sound information file 1 includes a clear stream sound, Different sound colors are stored from the same type of sound, such as wind sound in the sound information file 2, sound of insects in the sound information file 3, and thunder in the sound information file 4.

  In the reproducing unit 30, as a default, all kinds of timbres are reproduced with a predetermined period and a predetermined volume, and output from the speaker 20 via the driver 18.

  On the other hand, as shown in FIG. 2, the image analysis unit 26 connected to the image acquisition unit 22 analyzes the captured image information.

  In the image analysis unit 26, as a basic process, the captured image is converted into a gray scale and edge detection is executed.

  The gray scale conversion may be, for example, binarized black and white data or density data (for example, 8-bit gradation data). Note that black and white data has a smaller amount of information than density data, so the accuracy of the degree of change is inferior, but the analysis processing speed can be shortened. In the present embodiment, black and white data is used, but it may be selected depending on the imaging conditions (region, time zone, etc.).

  The image analysis unit 26 is connected to the complexity parameter generation unit 32. The complexity parameter generation unit 32 generates information for extracting the complexity parameter 1, the complexity parameter 2, and the complexity parameter 3 as environment parameters.

(Complexity parameter 1)
The complexity parameter 1 generated by the complexity parameter generation unit 32 is information for monitoring the degree of change of the entire image.

  When generating the complexity parameter 1, the black and white data generated as a basic process is acquired (see FIG. 3A), and the difference (absolute value) of the temporal and backward image information is calculated (difference calculation). The average value when the difference calculation is repeated several tens of times (for example, about 60 times) is calculated according to the travel (movement). With this complexity parameter 1, it is possible to recognize a change in the entire image captured while the vehicle is traveling.

  The complexity parameter 1 is sent to the weight determination unit 34.

(Complexity parameter 2)
The complexity parameter 2 generated by the complexity parameter generation unit 32 is information for mainly monitoring the degree of change in the left and right ends of the traveling direction of the vehicle.

  When generating the complexity parameter 2, monochrome data (absolute value) generated as basic processing is acquired, and the acquired monochrome data image is divided into a plurality of image frames. For example, as shown in FIG. 3C, the image is divided into 9 × 5, and numbers are assigned to the respective image frames from the upper left to the lower right.

  Among the divided image frames, No. 1 in FIG. 21 and no. 26, the average absolute value of the difference from No. 26, and No. 26 25 and No. An average value of absolute values of differences from 30 is extracted. And No. 21 and no. 26, the average absolute value of the difference from No. 26, and No. 26 25 and No. The average value when the average value of the absolute value of the difference from 30 is calculated (average value calculation), and the average value calculation is repeated several tens of times (for example, about 40 times) according to the running (movement) of the vehicle. Calculate the value.

  With this complexity parameter 2, it is possible to recognize changes in the side especially during the running of the vehicle. The complexity parameter 2 is sent to the weight determination unit 34.

(Weighting by complexity parameters 1 and 2)
As shown in FIG. 2, the weight determination unit 34 weights the four types of sound information files (sound information file 1 to sound information file 4) based on the complexity parameter 1 and the complexity parameter 2, respectively. .

  FIG. 4 is a characteristic diagram for specifying the current control coordinate point (x, y) of the image captured by the imaging unit 16 when the complexity parameter 1 is on the horizontal axis and the complexity parameter 2 is on the vertical axis. It is. That is, the control coordinate point is determined by the balance between the complexity parameter 1 and the complexity parameter 2.

  The sound information file 1 to sound information file 4 are located at the four corners of the characteristic diagram of FIG. The weights 1 to 4 set in the sound information file 1 to the sound information file 4 correspond to the distance from each sound information file (sound information file 1 to sound information file 4) to the control coordinate point (x, y). ing.

  The sound information file 1 is arranged at a position where the complexity parameter 1 is minimum and the complexity parameter is minimum (xmin, ymin). As shown in FIG. 5, the sound information file 1 has a characteristic that the weight is stronger as the complexity parameter 1 is smaller, and the weight is stronger as the complexity parameter 2 is smaller.

  The sound information file 2 is arranged at a position where the complexity parameter 1 is minimum and the complexity parameter is maximum (xmin, ymax). As shown in FIG. 5, the sound information file 2 has a characteristic that the weight is stronger as the complexity parameter 1 is smaller, and the weight is stronger as the complexity parameter 2 is larger.

  The sound information file 3 is arranged at a position where the complexity parameter 1 is maximum and the complexity parameter is maximum (xmax, ymax). As shown in FIG. 5, the sound information file 3 has a characteristic that the weight is stronger as the complexity parameter 1 is larger, and the weight is stronger as the complexity parameter 2 is larger.

  The sound information file 4 is arranged at a position where the complexity parameter 1 is maximum and the complexity parameter is minimum (xmax, ymin). As shown in FIG. 5, the sound information file 4 has a characteristic that the weight is stronger as the complexity parameter 1 is larger, and the weight is stronger as the complexity parameter 2 is smaller.

  As shown in FIG. 2, the weight determination unit 34 is connected to the weight adjustment unit 36 and sends the determined weight information to the weight adjustment unit 36. The weight adjustment unit 36 performs weighting on the sound information file 1 to the sound information file 4 reproduced by the reproduction unit 30 based on the weight information determined for the weight. Thus, the sound information file 1 to the sound information file 4 are output with the degree of emphasis (period, volume, etc.) at the time of output being changed based on the complexity parameter 1 and the complexity parameter 2.

  For example, when the values of the complexity parameter 1 and the complexity parameter 2 are small, the sound information file 1 has the strongest weight and is emphasized.

(Complexity parameter 3)
The complexity parameter 3 generated by the complexity parameter generation unit 32 is information for monitoring the presence or absence of an obstacle present in the vicinity of the vehicle.

  When generating the complexity parameter 3, the reference line A (see FIG. 6) of the characteristic curve of the speed-complexity parameter 3 is set.

  The reference line A shown in FIG. 6 is a characteristic curve when it is assumed that the complexity parameter 3 changes linearly with respect to the vehicle speed. The difference between the reference line A and the actual characteristic curve is This is information for monitoring the presence or absence of obstacles in the vicinity.

  For example, when the vehicle is traveling on a road with few obstacles, the actual speed-complexity parameter 3 characteristic curve becomes the characteristic curve B, and there is no element exceeding the reference line A.

  On the other hand, when traveling on a road with many obstacles, the actual speed-complexity parameter 3 characteristic curve becomes the characteristic curve C, and there are elements exceeding the reference line A (see the hatched area D in FIG. 6). ). The situation where the hatched area D occurs is when the vehicle is in a parking lot site, the road is narrow and a telegraph pole or a vending machine protrudes on the road side, and there is a lot of traffic. Can be mentioned.

  The complexity parameter 3 generated by the complexity parameter generation unit 32 is sent to the risk determination unit 38. The risk determination unit 38 determines the current vehicle risk by monitoring whether or not the hatched area D in FIG. 6 has occurred. The situation where the degree of risk during driving is high is not limited to the above.

  As shown in FIG. 2, the risk determination unit 38 is connected to the gain adjustment unit 40. The gain adjustment unit 40 is connected to the reproduction unit 30. That is, when the risk level determination unit 38 sends a numerical value indicating the risk level to the gain adjustment unit 40, the gain adjustment unit 40 displays a risk level-gain (here, volume) characteristic diagram as shown in FIG. Based on the above, the gain adjustment amount is specified. The reproduction unit 30 adjusts the gain (amplification factor) of the synthesized sound of the sound information file 1 to the sound information file 4 based on the specified gain adjustment amount.

  For example, if the vehicle has a narrow road and telegraph poles and vending machines are installed on the road side, or if you are traveling in a busy area, you can increase the volume higher than the default volume. It is designed to call attention. In this embodiment, the gain is set as the volume, but for example, the playback pitch amount (increases the playback pitch when the degree of danger is high), the scale (increases the scale when the degree of danger is high), and the like are adjusted. It may be.

  The operation of the present embodiment will be described below.

  FIG. 8 is a flowchart showing a sound generation control routine according to the present embodiment that is activated when the IG is turned on. FIG. 9 is a flowchart showing a transition state of information (signal) processing in the sound generation control executed based on the flowchart of FIG.

  As shown in FIG. 8, when the IG is turned on, in step 100, the imaging unit 16 is activated to start imaging.

  In the next step 102, the sound information file 1 to the sound information file 4 stored in the sound information file storage unit 28 are read out, and then the process proceeds to step 104 to start the reproduction process. That is, the sound sources of the sound information file 1 to the sound information file 4 read in step 102 are synthesized and repeatedly output from the speaker.

  In the next step 106, the image picked up by the image pickup unit 16 is analyzed. In the present embodiment, the captured image is converted to gray scale (monochrome data), and edge detection is performed (see FIG. 3).

  In the next step 108, the complexity parameter 1 is calculated from the image-analyzed information.

  That is, the complexity parameter 1 calculates the absolute value of the difference between the temporal and backward image information based on the black and white data, and calculates the average value when the difference calculation is repeated according to the running of the vehicle.

  In the next step 110, the complexity parameter 2 is calculated from the image-analyzed information.

  That is, the complexity parameter 2 is the average of the absolute values of the difference between the image frames corresponding to the left and right of the vehicle from among the image frames obtained by dividing the black and white data image into a plurality of image frames (for example, 9 × 5) The value is calculated, and the average value when the average value calculation is repeated is calculated according to the running of the vehicle.

  In the next step 112, the weight is determined based on the complexity parameter 1 obtained in step 108 and the complexity parameter 2 obtained in step 110. The weight is determined according to the distance between the position of the control coordinate point (x, y) and the coordinate point of the sound information file (see FIGS. 4 and 5).

  In the next step 114, weighting processing is executed on each of the sound information files 1 to 4 based on the determined weight, and the process proceeds to step 116.

  In step 116, the complexity parameter 3 is calculated from the image-analyzed information.

  That is, the complexity parameter 3 is information for monitoring the presence or absence of an obstacle present in the vicinity of the vehicle, and the speed-complexity parameter 3 characteristic curve when traveling on a road with many obstacles (see FIG. FFF). In FIG. 5, the information is used to determine whether or not there is an area exceeding the reference line A (hatched area D).

  In the case of the characteristic curve C in FIG. 6, it means that the degree of danger during driving is high. The specific situation is as follows.

  (Situation 1) The vehicle is in the parking lot site.

  (Situation 2) The road is narrow, and telephone poles and vending machines are installed on the road.

  (Situation 3) The road is narrow and there are many people.

  In addition, the situation where the degree of risk during driving is high is not limited to the above-described situations 1 to 3.

  In the next step 118, a gain (amplification factor) is determined based on the size (area, maximum value, etc.) of the hatched region D (see FIG. 6) determined by the complexity parameter 3 calculated in step 116, and then the step The process proceeds to 120 to execute gain adjustment processing. By this gain adjustment process, the higher the degree of danger, the greater the volume that flows from the speaker, or the faster the playback pitch of the sound information file is, the change that alerts the driver.

  In the next step 122, it is determined whether or not IG is turned off. If a negative determination is made, the process returns to step 106 and the above steps are repeated. If the determination in step 122 is affirmative, the process proceeds to step 124 to stop the sound generation control, and the imaging and sound reproduction are stopped, and this routine ends.

(Modification)
In the present embodiment, four types of sound information files 1 to 4 are stored in advance in the sound information file storage unit 28 (see FIG. 2), and the sound information file storage unit 28 stores the sound information. The file 1 to the sound information file 4 are read and reproduced.

  On the other hand, in the modification, the automatic music file 1 to the automatic music file 4 are applied as four types of sound information files.

  The automatic composition file 1 to the automatic composition file 4 are composed each time by executing the automatic composition program based on the image information captured by the imaging unit 16.

  The automatic composition program is stored in the ROM 12C of FIG. 1 and executed by the CPU 12A.

  In the present embodiment, ROM 12C stores “MIDI (Musical Instrument Digital Interface) signal conversion software” and “software for generating music information from MIDI signals”.

  MIDI is an international standard for communicating music information with digital signals. The MIDI signal is information on the MIDI standard and information on a standard MIDI file format (SMF). In “MIDI signal conversion software”, the basic frequency of sound is “note number”, sound pressure level is “velocity level”, tone is “number indicating instrument name”, time change is “note on / note off” Are respectively converted into data representing "."

  More specifically, the automatic music file 1 to the automatic music file 4 are set with a sound source of a musical instrument serving as a reference sound, for example, a sound source such as a piano, a guitar, a violin, and a trumpet.

  Image information picked up by the image pickup unit 16 is color-separated into RGB, and respective brightness (density) information is acquired.

  Based on the brightness information of each color, the three elements of the melody (melody), harmony (harmonic), rhythm (rhythm) song, height (frequency), magnitude (sound pressure level), timbre (frequency component) Each of the three elements of the sound is adjusted and composed.

  Note that the applicants have filed an application for an automatic song that creates sound from imaging information (camera image) of the imaging unit 16 (Japanese Patent Laid-Open No. 2017-102863 as an example).

  FIG. 10 is a flowchart showing a transition state of information (signal) processing in sound generation control according to a modification of the present embodiment.

  In the automatic composition program, when the imaging information imaged by the imaging unit 16 is input, the automatic music composition is executed, and the automatic music composition files 1 to 4 are created. By replacing the automatic music file 1 to the automatic music file 4 with the sound information file 1 to the sound information file 4 shown in FIG. 9, this modified example is shown in FIG. It is processed by the same flow.

  That is, the weights at the time of reproduction of the automatic music file 1 to the automatic music file 4 are adjusted by the complexity parameter 1 and the complexity parameter 2, and the automatic music file 1 to the automatic music file 4 are adjusted by the complexity parameter 3. The gain at the time of reproduction is adjusted, and the same effect as this embodiment can be obtained.

  In the present embodiment (including modifications), the complexity parameter 1 to the complexity parameter 3 are obtained based on the imaging information from the imaging unit 16, but in addition to the imaging unit 16 or the imaging unit Instead of 16, the driver's physical condition (heart rate, blood flow value, eyeball movement, etc.) may be monitored to obtain complexity parameter 1 to complexity parameter 3.

  Further, not only the driver but also the occupant's physical condition (presence of infants, sleeping, etc.) may be monitored to correct the complexity parameter 1 to the complexity parameter 3. For example, in a passenger compartment where an infant is present or a sleeping passenger is present, adjustments such as a moderate change in sound or a slow pitch of sound may be made.

  Furthermore, in the present embodiment (including modifications), the output sound from the speaker is changed based on the complexity parameter 1 to the complexity parameter 3, but the output target is mounted on the vehicle in addition to the sound. It may be related to the control of ambient lighting.

10 sound generation control device 12 microcomputer (sound generation means, first calculation means, second calculation means, first change means, second change means)
12A CPU
12B RAM
12C ROM
12D input / output unit (I / O)
12E bus 14 hard disk 16 imaging unit (acquisition means)
18 Driver 20 Speaker (output means)
DESCRIPTION OF SYMBOLS 22 Image acquisition part 24 Sound information file reading part 26 Image analysis part 28 Sound information file memory | storage part 30 Playback part 32 Complexity parameter production | generation part 34 Weight determination part 36 Weight adjustment part 38 Risk determination part 40 Gain adjustment part

Claims (8)

An acquisition means for acquiring environmental parameters including at least one of the surrounding environment information of the vehicle and the physical condition information of the occupant during traveling of the vehicle;
Sound generation means for generating sound according to the environmental parameter acquired by the acquisition means;
Output means for outputting a sound generated by the sound generation means to an occupant;
A sound generation device having   The sound generation apparatus according to claim 1, wherein the acquisition unit acquires a complexity parameter representing complexity of ambient environment information of the vehicle as the environment parameter. The acquisition means is
An image pickup means that is attached to the vehicle and picks up at least a forward image during traveling,
The sound generation apparatus according to claim 1, wherein an image captured by the imaging unit is analyzed to obtain ambient environment information of the vehicle. As the surrounding environment information of the vehicle, a first complexity parameter that is a degree of change of the entire front image captured by the imaging unit, and a degree of change that is specific to the vehicle side portion of the front image. A first computing means for computing two complexity parameters;
The sound generation means changes a sound to be generated based on a balance between the first complexity parameter and the second complexity parameter 2 calculated by the first calculation means. The sound generator described. A second complexity parameter for calculating a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance with respect to the vehicle from a front image captured by the imaging unit as the ambient environment information of the vehicle; Further having a computing means,
The sound generation means changes the sound to be generated based on the relationship between the third complexity parameter calculated by the second calculation means and the speed of the vehicle. Item 5. The sound generation device according to Item 4. The sound generation means reads a plurality of sounds stored in advance and generates a sound, or generates a sound from a plurality of sounds that are automatically composed based on an image captured by the imaging means,
The sound generation apparatus according to claim 3, wherein the sound change based on the environmental parameter adjusts the synthesis degree of a plurality of sounds. An imaging means attached to the vehicle and capturing at least a forward image during traveling;
Sound generation means for generating a synthesized sound by reading a plurality of sounds stored in advance, or for generating a synthesized sound from a plurality of sounds that are automatically composed based on an image captured by the imaging means;
A first complexity parameter that is a degree of change of the entire front image captured by the imaging means, and a second complexity parameter that is a degree of change specialized for the vehicle side portion of the front image. First computing means for computing;
Second computing means for computing a third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle from a front image captured by the imaging means;
A first degree of synthesis of a plurality of sounds synthesized by the sound generation unit is changed based on a balance between the first complexity parameter and the second complexity parameter 2 calculated by the first calculation unit. Change means,
Second changing means for changing the volume of the synthesized sound generated by the sound generating means based on the relationship between the third complexity parameter calculated by the second calculating means and the speed of the vehicle;
Output means for outputting a sound generated by the sound generation means to an occupant;
A sound generation device having On the computer,
Read out a plurality of sounds stored in advance to generate a synthesized sound, or generate a synthesized sound from a plurality of sounds that are automatically tuned based on an image captured by the imaging means,
A first complexity parameter that is a degree of change of the entire front image captured by the imaging means, and a second complexity parameter that is a degree of change specialized for the vehicle side portion of the front image. Operate,
A third complexity parameter indicating a degree of change of an obstacle existing within a predetermined distance from the vehicle is calculated from a front image captured by the imaging unit,
Based on the balance between the calculated first complexity parameter and second complexity parameter 2, the degree of synthesis of a plurality of sounds to be synthesized is changed,
Changing the volume of the synthesized sound to be generated based on the relationship between the calculated third complexity parameter and the vehicle speed;
A sound generation control program for executing this.