JP4905648B2 - Stimulus presentation device for vehicle and stimulus presentation method - Google Patents

Description

  The present invention relates to a vehicle stimulus presentation device and a stimulus presentation method that give a driver a stimulus by sound or vibration according to a traveling scene of a host vehicle and realize the pleasure of driving.

  2. Description of the Related Art In recent years, in vehicles such as automobiles, an image of the surroundings of a vehicle is captured by an in-vehicle camera, and useful information is acquired from the captured image of the in-vehicle camera and presented to the driver, thereby assisting the driver's driving operation Various systems have been proposed (see, for example, Patent Document 1 and Patent Document 2).

  The device described in Patent Document 1 is a system that detects that the vehicle has deviated from the road and notifies the driver of the fact. The displacement is detected, the driver's consciousness level is evaluated from the standard deviation of the lateral displacement, and the notification pattern is changed based on the lateral displacement amount of the vehicle and the driver's consciousness level.

Moreover, what is described in Patent Document 2 is a system that detects an obstacle behind the vehicle and informs the driver of the obstacle, and the vehicle rear in a situation where there is no obstacle photographed in advance by the in-vehicle camera. Is compared with a captured image of the rear of the vehicle captured by the in-vehicle camera immediately before the vehicle starts to travel backward to determine whether there is an obstacle behind the vehicle. When this is detected, the driver is notified accordingly.
JP-A-5-104976 JP 2000-177513 A

  However, the conventional system that processes the captured image of the in-vehicle camera and presents some information to the driver, including the technique described in the cited document 1 and the technique described in the cited document 2, are all provided by the driver. The purpose is to prevent an inappropriate driving operation, and information that gives the driver a realistic feeling of driving cannot be presented.

  In other words, when the driver wants to enjoy driving the vehicle, for example, when a sound effect or vibration that varies in accordance with the driving scene of the vehicle is given, such as a driving simulator or a racing game, the vehicle It is thought that the realism of driving is enhanced and the driver can feel the joy of driving.

  However, the conventional system that processes the captured image of the in-vehicle camera and presents some information to the driver is not configured to change the content of the information presented to the driver according to the driving scene. It is difficult to present information in a form that enhances the sense of realism of driving a vehicle.

  The present invention was devised in view of the conventional situation as described above, and stimulates the driver in such a way as to enhance the realistic feeling of the vehicle driving based on the image taken by the in-vehicle camera. An object of the present invention is to provide a vehicular stimulus presentation device and a stimulus presentation method that allow a driver to feel the joy of driving.

  The present invention captures a landscape image in front of the vehicle with an in-vehicle camera, processes the landscape image to estimate the current travel scene of the vehicle, and changes according to the estimated current travel scene of the vehicle. By generating the stimulus by the sound and vibration of the mode to be performed, the driver of the own vehicle is given a stimulus that enhances the realism of driving the vehicle, and the enjoyment of driving can be realized.

  According to the present invention, since the stimulation by the sound and vibration that changes in accordance with the current driving scene of the vehicle is generated, the stimulation that enhances the realism of driving the vehicle to the driver of the vehicle. Giving you a sense of driving pleasure.

  Hereinafter, a specific example of a vehicular stimulus presentation apparatus configured by applying the present invention will be described with reference to the drawings.

  The vehicle stimulus presentation device of the present embodiment takes a landscape image in front of the host vehicle, which is a vehicle equipped with the vehicle stimulus presentation device, with an in-vehicle camera, converts the landscape image into sound, and This is presented to the driver of the vehicle as a stimulating sound whose aspect changes in accordance with the current driving scene.

  FIG. 1 is a block diagram showing the configuration of the vehicle stimulus presentation device of the present embodiment. As shown in FIG. 1, this vehicular stimulus presentation apparatus includes an image processing apparatus 10 that converts an image into sound as a main component, and an in-vehicle camera 1 and a speed detection apparatus 2 on the input side of the image processing apparatus 10. Are connected, and the speaker 3 is connected to the output side of the image processing apparatus 10.

  The in-vehicle camera 1 is installed in, for example, a front bumper in front of the host vehicle, and continuously captures landscape images in front of the host vehicle at a predetermined shooting cycle. Then, the in-vehicle camera 1 outputs the image data of the photographed landscape image to the image processing device 10 in units of frames synchronized with the photographing cycle.

  The image processing apparatus 10 includes a contour detection unit 11, an image filter unit 12, a reference image pattern storage unit 13, an image pattern identification unit 14, an image pattern time change processing unit 15, and a presentation related to the present invention. A sound processing unit 16 that processes image data of a landscape image in front of the vehicle input from the in-vehicle camera 1 to estimate a current traveling scene of the vehicle, and generates a stimulating sound whose aspect changes accordingly. Output from the speaker 3.

  In the vehicle stimulus presentation device of the present embodiment, the contour detection unit 11 and the image filter unit 12 of the image processing device 10 correspond to a pattern extraction unit, and a reference image pattern storage unit 13, an image pattern identification unit 14, and an image pattern. The time change processing unit 15 corresponds to a traveling scene estimation unit, and the presentation sound processing unit 16 corresponds to a stimulus control unit. The speaker 3 corresponds to a stimulus generating unit. In other words, in the vehicular stimulus presentation device of the present embodiment, when a landscape image in front of the host vehicle photographed by the in-vehicle camera 1 is input to the image processing device 10, processing by the contour detection unit 11 and the image filter unit 12 is performed. A specific pattern is extracted from the landscape image, and the specific pattern extracted from the landscape image by using the reference image pattern stored in the reference image pattern storage unit 13 by the image pattern identification unit 14 and the image pattern time change processing unit 15. The current driving scene of the vehicle is estimated based on the type of pattern and the temporal change. Then, the presentation sound processing unit 16 generates a stimulating sound whose aspect changes in accordance with the estimated current driving scene of the own vehicle, and this stimulating sound gives the driver of the own vehicle a sense of presence in driving the vehicle. It is output from the speaker 3 as a stimulating stimulus.

  More specifically, landscape image image data input from the in-vehicle camera 1 to the image processing apparatus 10 in units of frames is first converted into digital data and then supplied to the contour detection unit 11. Processing such as known edge detection is performed to detect the contour of a vehicle or road included in the landscape image. Then, the landscape image processed by the contour detection unit 11 is subjected to predetermined filtering processing by the image filter unit 12, thereby removing an excess portion and extracting a specific pattern such as a vehicle or a road.

  For example, when a landscape image in front of the host vehicle as shown in FIG. 2A is photographed by the in-vehicle camera 1, the landscape image is detected by the contour detection unit of the image processing apparatus 10. 11 is processed into an edge image as shown in FIG. 2B, and further processed into a pattern image as shown in FIG. That is, as shown in FIG. 2 (a), a landscape image is captured as a landscape image by the in-vehicle camera 1 while the vehicle is traveling on a relatively wide straight road and there is one preceding vehicle and one oncoming vehicle ahead. Then, after processing by the contour detection unit 11 and the image filter unit 12 of the image processing apparatus 10, a straight line pattern that converges to a certain point in the image from this landscape image becomes a road side portion of a straight road, A road pattern P1 representing a center line, a lane line, and the like is extracted, and two substantially rectangular patterns existing on the road are extracted as a vehicle pattern P2 representing a preceding vehicle and an oncoming vehicle ahead of the host vehicle.

  Further, for example, when a landscape image in front of the host vehicle as shown in FIG. 3A is captured by the in-vehicle camera 1, the landscape image is displayed in FIG. 3B in the contour detection unit 11 of the image processing apparatus 10. The image is processed into an edge image as shown in the drawing, and further processed into a pattern image as shown in FIG. That is, as shown in FIG. 3 (a), when a landscape in which the vehicle is traveling on a road with a relatively narrow right curve and one preceding vehicle exists ahead is photographed as a landscape image by the in-vehicle camera 1, A curve pattern that converges on a certain point in the image from the landscape image through the processing in the contour detection unit 11 and the image filter unit 12 of the image processing apparatus 10 becomes a road side portion, a center line, or a lane on a curved road. A road pattern P1 representing a lane marking or the like is extracted, and one substantially rectangular pattern existing on the road is extracted as a vehicle pattern P2 representing a preceding vehicle ahead of the host vehicle.

  Here, as a specific pattern used for estimating the current traveling scene of the own vehicle, a road pattern P1 representing a straight road or a curved road or a vehicle pattern P2 representing a preceding vehicle or an oncoming vehicle ahead of the own vehicle is used. Although extracted from a landscape image in front of the vehicle, in addition to the road pattern P1 and the vehicle pattern P2, for example, structures and trees beside the road, structures such as traffic lights and signs on the road, and pedestrians on the road A pattern representing a motorcycle or a motorcycle may be extracted from a landscape image in front of the host vehicle as a specific pattern for estimating the current travel scene of the host vehicle.

  When a specific pattern such as a road pattern P1 and a vehicle pattern P2 is extracted from the landscape image in the image filter unit 12, the image pattern identification unit 14 specifies the road pattern P1 and the vehicle pattern P2 extracted from the landscape image. Are compared with the reference image pattern stored in the reference image pattern storage unit 13, and the current traveling scene of the host vehicle is estimated. That is, in the reference image pattern storage unit 13, for example, various road patterns corresponding to roads of various shapes, various vehicle patterns corresponding to vehicles existing in various positional relations with respect to the own vehicle, It is stored in advance as a reference image pattern. Then, by combining various road patterns and various vehicle patterns stored in the reference image pattern storage unit 13, for example, as shown in part in FIG. A reference image pattern representing the entire landscape image in front of the vehicle is obtained. The image pattern recognition unit 14 selects specific patterns such as a road pattern P1 and a vehicle pattern P2 extracted from a landscape image from combinations of various road patterns and various vehicle patterns stored in the reference image pattern storage unit 13. Based on the reference image pattern of the entire landscape image specified, for example, whether the vehicle is traveling on a straight road or a curved road, and to what extent The current driving scene of the host vehicle is estimated, such as whether the vehicle is traveling on a road having a width of, or whether there is a preceding vehicle or an oncoming vehicle ahead.

  In addition to the method of estimating the road width of the road on which the vehicle is running by comparing the road pattern P1 extracted from the landscape image with various road patterns stored in the reference image pattern storage unit 13, For example, it can be estimated by a method of measuring the distance W between the left and right road patterns P1 in front of the specified distance from the own vehicle in the pattern images as shown in FIG. 2 (c) and FIG. 3 (c). Is possible.

  Further, for example, for a traveling scene to which an element of time change is added, such as a scene in which the host vehicle passes an oncoming vehicle or a vehicle overtaking the preceding vehicle, the image pattern time change processing unit 15 extracts a road pattern extracted from the landscape image. A temporal change of a specific pattern such as P1 or vehicle pattern P2 is estimated by being grasped together with the traveling speed of the own vehicle obtained from the speed detection device 2. That is, the image pattern time change processing unit 15 temporarily stores the processing result of the image pattern identification unit 14 over a plurality of frames of the landscape image captured by the in-vehicle camera 1, and performs processing for each frame. By comparing the results, a temporal change of a specific pattern such as the road pattern P1 or the vehicle pattern P2 extracted from the landscape image is recognized. The temporal change of each pattern is, for example, a change in shape such as a road that has changed from a straight road to a curved road in the case of a road pattern, or a change in road width. This is a change in the relative position of the preceding vehicle or the oncoming vehicle, a change in dimensions on the image, or the like. Then, the image pattern time change processing unit 15 makes the specific pattern such as the road pattern P1 and the vehicle pattern P2 extracted from the landscape image as time elapses with respect to the traveling speed of the own vehicle obtained from the speed detection device 2. Based on the identification of how the vehicle is changing, a travel scene to which an element of time change such as a scene in which the host vehicle passes the oncoming vehicle or a scene overtaking the preceding vehicle is added is estimated.

  Here, whether the vehicle ahead of the host vehicle is a preceding vehicle or an oncoming vehicle can be distinguished by, for example, the following method. That is, the image pattern time change processing unit 15 stores in advance a temporal dimensional change on the screen of a stationary object on the road with respect to the traveling speed V of the host vehicle as a function of the traveling speed V: ΔL = f (V). Keep it. Then, if the temporal dimensional change of the vehicle pattern P2 extracted from the landscape image over a plurality of frames is smaller than ΔL, the image pattern time change processing unit 15 represents the preceding vehicle ahead of the host vehicle. If it is determined that the vehicle pattern is a vehicle pattern and the temporal dimensional change of the vehicle pattern P2 extracted from the landscape image over a plurality of frames is larger than ΔL, the vehicle pattern P2 is a vehicle pattern representing an oncoming vehicle in front of the host vehicle. Judge that there is.

  The current running scene of the vehicle is determined based on the type and temporal change of a specific pattern such as the road pattern P1 and the vehicle pattern P2 extracted from the landscape image by the image pattern identification unit 14 and the image pattern time change processing unit 15. Once estimated, the presentation sound processing unit 16 generates a stimulus sound whose aspect changes in accordance with the estimated traveling scene. Then, this stimulation sound is output from the speaker 3 as a stimulus that enhances the realistic feeling of the vehicle driving for the driver of the own vehicle. In addition, the aspect of the stimulation sound that changes in accordance with the current driving scene of the vehicle is, for example, one of the output intervals of each sound (in the case of intermittent stimulation sounds), the frequency of the sound, or the sound pressure. Or a combination of these.

  Here, a specific example of the stimulation sound generated by the presentation sound processing unit 16 and output from the speaker 3 will be described by taking as an example a case where the current traveling scene of the own vehicle is estimated to be a scene that passes by the oncoming vehicle. explain.

  FIG. 5 shows temporal changes in the road pattern P1 and the vehicle pattern P2 in a scene in which the host vehicle passes the oncoming vehicle. In a scene where the own vehicle passes the oncoming vehicle, the vehicle pattern P2 representing the oncoming vehicle in front of the own vehicle has its dimensions over time as shown in FIGS. 5 (a) to 5 (b) and 5 (c). The relative position with respect to the own vehicle, that is, the position in the image, gradually changes from the center to the lower right with time. The image pattern time change processing unit 15 of the image processing apparatus 10 determines the temporal change of the vehicle pattern P2 having such a tendency at an initial stage when the vehicle pattern P2 is extracted from the landscape image. The vehicle pattern representing the oncoming vehicle ahead of the host vehicle is determined by the above-described method, and it is estimated that the current traveling scene of the host vehicle is a scene that is different from the oncoming vehicle. Then, the image pattern time change processing unit 15 tracks the change in the relative positional relationship between the oncoming vehicle and the own vehicle thereafter, while the present driving scene of the own vehicle is determined as the oncoming vehicle to the presentation sound processing unit 16. Information indicating that the scene is a passing scene and information on the relative positional relationship between the oncoming vehicle and the host vehicle, which changes from moment to moment, are output as needed.

  In the presentation sound processing unit 16, when the image pattern time change processing unit 15 estimates that the current traveling scene of the own vehicle is a scene passing the oncoming vehicle and is given information to that effect, for example, as shown in FIG. Then, a continuous stimulation sound whose sound pressure or sound frequency changes in accordance with the change in the relative positional relationship between the oncoming vehicle and the own vehicle is generated, and the generated stimulation sound is output from the speaker 3 as needed. That is, when the current driving scene of the host vehicle is a scene that passes the oncoming vehicle, a high-frequency continuous stimulation sound is generated at a stage before the time T1 when the host vehicle passes the oncoming vehicle. The pressure is output from the speaker 3 so that the pressure gradually increases in inverse proportion to the square of the distance between the own vehicle and the oncoming vehicle, and after the time T1 when the own vehicle passes the oncoming vehicle, the frequency is higher than before the passing frequency. The continuous stimulation sound changed to a low frequency is output from the speaker 3 so that the sound pressure gradually decreases in inverse proportion to the square of the distance between the host vehicle and the oncoming vehicle.

  After the time T1 when the host vehicle passes the oncoming vehicle, the oncoming vehicle no longer appears in the landscape image in front of the host vehicle taken by the in-vehicle camera 1, so the change of the oncoming vehicle after passing the It cannot be followed from the change of the vehicle pattern P2 extracted from the image. Therefore, after the oncoming vehicle moves out of the shooting range of the in-vehicle camera 1 (after passing the own vehicle), the vehicle pattern P2 until immediately before moving out of the shooting range of the in-vehicle camera 1 (just before passing the own vehicle). From the change, the subsequent movement of the oncoming vehicle is estimated. In other words, assuming that the relative speed and relative acceleration of the host vehicle and the oncoming vehicle immediately before passing each other are kept constant thereafter, the amount of movement of the oncoming vehicle after passing from the relative speed and relative speed immediately before passing each other is calculated. Calculate to estimate the relative positional relationship with the vehicle.

  When it is estimated that the vehicle is passing the oncoming vehicle, the change in the mode of the stimulation sound that is output from the speaker 3 above is due to the Doppler effect of the siren sound that is usually heard when passing the ambulance. It is a change in sound similar to what you hear. In this way, by amplifying and giving stimuli that match the driver's experience as stimulus sounds, it gives the driver a sense of realism in driving the vehicle, arouses refreshment, comfort, etc. You can feel fun. Here, the stimulus sound output from the speaker 3 may be a single sound or a chord, but if the sound that simulates the engine sound or exhaust sound of a car is output from the speaker 3 as a stimulus sound, the object is a car. It is easy to understand and is more effective. Also, when the road width changes while the host vehicle passes the oncoming vehicle, or when the road shape changes, such as switching between a straight road and a curved road, a stimulus sound is output from the speaker 3 in accordance with the change. You may make it change the aspect of.

  Next, a specific example of the stimulation sound generated by the presentation sound processing unit 16 and output from the speaker 3 will be described by taking as an example a case where the current traveling scene of the own vehicle is estimated to be a scene overtaking the preceding vehicle. explain.

  FIG. 7 shows a temporal change in the road pattern P1 and the vehicle pattern P2 in a scene where the own vehicle passes the oncoming vehicle. In a scene in which the host vehicle overtakes the preceding vehicle, the vehicle pattern P2 representing the preceding vehicle ahead of the host vehicle has its dimensions over time as shown in FIGS. 7 (a) to 7 (b) and 7 (c). The relative position with respect to the own vehicle, that is, the position in the image, gradually changes from the center to the lower left as time passes. The image pattern time change processing unit 15 of the image processing apparatus 10 determines the temporal change of the vehicle pattern P2 having such a tendency at an initial stage when the vehicle pattern P2 is extracted from the landscape image. The vehicle pattern representing the preceding vehicle ahead of the host vehicle is determined by the above-described method, and it is estimated that the current traveling scene of the host vehicle is a scene overtaking the preceding vehicle. Then, the image pattern time change processing unit 15 tracks the change in the relative positional relationship between the preceding vehicle and the own vehicle thereafter, while the present driving scene of the own vehicle indicates the preceding vehicle to the presentation sound processing unit 16. Information indicating that the scene is an overtaking scene and information on the relative positional relationship between the preceding vehicle and the host vehicle that change from moment to moment are output as needed.

  In the presentation sound processing unit 16, when the image pattern time change processing unit 15 estimates that the current traveling scene of the own vehicle is a scene overtaking the preceding vehicle and is given information to that effect, for example, as shown in FIG. In response to a change in the relative positional relationship between the preceding vehicle and the own vehicle, an intermittent stimulation sound in which the output interval d, sound pressure, and sound frequency of each sound changes is generated, and the generated stimulation sound Is output from the speaker 3 at any time. That is, when the current driving scene of the own vehicle is a scene that overtakes the preceding vehicle, the stimulation sound output from the speaker 3 is an intermittent sound, and at the stage before the time T2 of the moment when the own vehicle overtakes the preceding vehicle. As the vehicle approaches the preceding vehicle, the output interval d of each sound is gradually shortened, and the intermittent sound that changes so that the sound frequency gradually increases and the sound pressure gradually increases is a speaker. After the time T2 of the moment when the vehicle overtakes the preceding vehicle, the sound output interval d is gradually increased as the overtaking preceding vehicle moves away, and the sound frequency is gradually lowered. An intermittent stimulation sound that changes so that the sound pressure gradually decreases is output from the speaker 3.

  In addition, after time T2 of the moment when the own vehicle overtakes the preceding vehicle, the preceding vehicle is not reflected in the landscape image in front of the own vehicle photographed by the in-vehicle camera 1, so the preceding vehicle is out of the shooting range of the in-vehicle camera 1. After moving (after overtaking the preceding vehicle), the subsequent movement of the preceding vehicle from the change in the vehicle pattern P2 until just before moving outside the shooting range of the in-vehicle camera 1 in the same manner as when passing the oncoming vehicle Is estimated. That is, assuming that the relative speed and relative acceleration of the host vehicle and the preceding vehicle immediately before passing are kept constant thereafter, the amount of movement of the preceding vehicle after passing is calculated from the relative speed and relative speed immediately before passing. Calculate to estimate the relative positional relationship with the vehicle.

  When the vehicle is estimated to be a scene overtaking the preceding vehicle, the change in the state of the stimulation sound output from the speaker 3 is a change that increases the degree of excitement of the driver when overtaking the preceding vehicle. . By giving such a stimulating sound in the scene where the vehicle overtakes the preceding vehicle, it gives the driver of the vehicle a sense of realism and arouses a sense of excitement and realizes the enjoyment of driving. it can. Here, the stimulation sound to be output from the speaker 3 may be a single sound or a chord. However, as in the above-described example of a scene passing by the oncoming vehicle, a sound simulating the engine sound or exhaust sound of the automobile is used as the stimulation sound from the speaker 3. If it is made to output, it will be easy to understand that a target object is a motor vehicle, and it is more effective. In addition, when the road width changes while the host vehicle overtakes the preceding vehicle, or when the road shape changes, such as switching between a straight road and a curved road, a stimulus sound is output from the speaker 3 in accordance with the change. You may make it change the aspect of.

  As described above, the case where the current driving scene of the own vehicle is estimated to be a scene that passes the oncoming vehicle and the case where the current driving scene of the own vehicle is estimated to be a scene overtaking the preceding vehicle are taken as examples. Although the change in the mode of the stimulus sound output from the speaker 3 has been specifically described, the vehicle stimulus presentation device according to the present embodiment is not limited to a scene other than the above-mentioned scene passing by an oncoming vehicle or a scene overtaking a preceding vehicle. A stimulus sound whose aspect changes in accordance with various driving scenes of the car is output from the speaker 3.

  For example, even if there is no vehicle such as a preceding vehicle or an oncoming vehicle in front of the host vehicle, a stimulus sound whose aspect changes according to the width of the road on which the host vehicle is currently traveling is output from the speaker 3. As a result, it is possible to increase the sensitivity that the driver of the own vehicle feels from the width of the road on which the driver is currently traveling, and to make the driver feel the joy of driving. In addition, even if there is no preceding vehicle or oncoming vehicle ahead of the vehicle, depending on the road shape such as whether the road on which the vehicle is currently traveling is a straight road or a curved road, If the speaker 3 outputs a stimulating sound whose aspect changes, the sensibility that the driver of the vehicle senses from the road shape of the road on which the vehicle is currently traveling is increased, and the driver feels the joy of driving. be able to.

  Specifically, for example, when the vehicle is traveling on a wide road or traveling on a straight road, a relaxing tempo stimulation sound is output from the speaker 3 to give a feeling of healing and the vehicle is narrow. When traveling on a road or traveling on a curved road, an effect of increasing the sense of tension by outputting a fast-temporal stimulus sound from the speaker 3 can be expected. Also, according to the personal characteristics of the driver of the own vehicle, contrary to the above example, a rapid tempo stimulating sound that increases the arousal level when the vehicle is traveling on a wide road or traveling on a straight road May be output from the speaker 3 so as to relieve a sense of tension when the vehicle is traveling on a narrow road or traveling on a curved road with a relaxing tempo sound.

  In addition, the speaker 3 outputs a stimulating sound whose aspect changes according to the road environment in which the vehicle is currently traveling, such as whether the vehicle is traveling in an urban area or a suburb. By doing so, it is possible to increase the sensitivity that the driver of the own vehicle feels from the surrounding scenery, and to make the driver feel the joy of driving. Specifically, for example, when the vehicle is traveling in an urban area, a high-temporal stimulus sound is output from the speaker 3 to increase the sense of tension, and when the vehicle is traveling in the suburbs, the tempo is relaxed. An effect of giving a feeling of healing by outputting a stimulation sound from the speaker 3 can be expected. Note that the estimation of whether the road environment in which the vehicle is currently traveling is an urban area or a suburb is performed by, for example, the in-vehicle camera 1 using the contour detection unit 11 or the image filter unit 12 of the image processing apparatus 10. This can be realized by extracting a pattern representing a building, a tree, or the like as a specific pattern from the landscape image in front of the vehicle and confirming the frequency with which the pattern representing the building, a tree, or the like is extracted. That is, it is possible to estimate that the vehicle is traveling in an urban area if a pattern representing a building is frequently extracted and that the vehicle is traveling in a suburb if a pattern representing a tree is frequently extracted. Further, if a relatively long straight edge component is extracted from the edge image as shown in FIG. 2B or 3B obtained by the contour detection unit 11 of the image processing apparatus 10, the urban area is extracted. If many fine edge components are extracted during traveling, it can be estimated that the vehicle is traveling in the suburbs.

  In addition, for example, the mode of the stimulation sound output from the speaker 3 according to various traveling scenes such as whether the vehicle is traveling on a congested road or an unoccupied road It is desirable to change

  FIG. 9 is a flowchart showing a flow of a series of operations in the vehicular stimulus presentation apparatus of the present embodiment as described above. As shown in the flowchart of FIG. 9, in the vehicle stimulus presentation device of the present embodiment, when power is turned on by operating the power switch by the driver of the vehicle, first, in step S1, the vehicle-mounted camera 1 A landscape image in front of the host vehicle is captured, and image data of the landscape image captured by the in-vehicle camera 1 is input to the image processing apparatus 10. In step S2, the contour detection unit 11 and the image filter 12 of the image processing apparatus 10 extract a specific pattern such as a road pattern or a vehicle pattern from a landscape image in front of the host vehicle.

  The processes in steps S1 and S2 are repeatedly performed in synchronization with the imaging cycle of the in-vehicle camera 1. In step S3, the image pattern identification unit 14 and the image pattern time change processing unit 15 of the image processing apparatus 10 are based on the types and temporal changes of specific patterns such as the road pattern and the vehicle pattern extracted in step S2. Thus, the current traveling scene of the vehicle is estimated. In the estimation of the travel scene in step S3, for example, for a travel scene in a large classification such as a road width or a road shape of the road on which the vehicle is currently traveling, each of the landscape images photographed by the in-vehicle camera 1 is used. For a driving scene that includes a temporal change element, such as a scene that is judged for each frame and the vehicle passes by an oncoming vehicle or a vehicle that overtakes the preceding vehicle, a landscape image captured by the in-vehicle camera 1 is used. It is judged from the change of a specific pattern over a plurality of frames.

  When the current traveling scene of the own vehicle is estimated in step S3, next, in step S4, the presentation sound processing unit 16 of the image processing apparatus 10 causes the stimulus to change in form according to the current traveling scene of the own vehicle. A sound is generated, and the stimulus sound generated by the presentation sound processing unit 16 of the image processing apparatus 10 is output from the speaker 3 in step S5.

  As described above in detail with reference to specific examples, according to the vehicle stimulus presentation device of the present embodiment, a road pattern, a vehicle pattern, and the like are obtained from a landscape image in front of the host vehicle photographed by the in-vehicle camera 1. Extract a specific pattern, estimate the current driving scene of the vehicle based on the type of these specific patterns and temporal changes, and generate a stimulating sound whose aspect changes according to the estimated current driving scene of the vehicle Since it is made to output from the speaker 3, it can give the driver | operator the irritation | stimulation which raises the realistic feeling of a vehicle driving, and can realize the pleasure of driving | operation. Specifically, for example, in a driving scene that passes by an oncoming vehicle or a driving scene that overtakes a preceding vehicle, the engine sound or exhaust sound of the vehicle that changes in response to the oncoming vehicle or the preceding vehicle approaching or leaving the host vehicle. By giving the driver an auditory stimulus that matches the visual information, such as a stimulating sound that simulates driving, giving the driver a sense of amplification that is greater than that obtained during normal driving, and enhancing the realistic feeling of vehicle driving Can do.

  In addition, in a driving scene such as driving on a wide road or a straight road with little change, it is possible to arouse emotions such as comfort and healing by giving a relaxing sound of tempo, narrow roads and curved roads, In driving scenes that require high arousal conditions such as fast-changing roads, a stimulating sound with a fast tempo stimulating sound is created to create an excited state and make driving fun, and activate a dull head due to lack of sleep, fatigue, etc. It can be changed into a crisp driving.

  The vehicular stimulus presentation device described above exemplifies an application example of the present invention, and the technical scope of the present invention is not limited to the contents disclosed in the description of the above embodiments. Of course, various alternative techniques that can be easily derived from these disclosures are also included. For example, in the above-described vehicular stimulus presentation device, intermittent or continuous stimulation sound is output from the speaker 3 as a stimulus whose aspect changes in accordance with the current running scene of the host vehicle. Instead of outputting, or in combination with the output of the stimulation sound, a vibration whose aspect changes in accordance with the current traveling scene of the host vehicle may be generated as a stimulus to the driver of the host vehicle. In this case, for example, a vibration generator such as a vibration motor is connected to the image processing apparatus 10, and the image processing apparatus 10 has a function of converting an image into vibration so that vibration is generated in accordance with the current traveling scene of the vehicle. It suffices to change the frequency and amplitude of vibration generated by the vessel.

It is a block diagram which shows the structure of the stimulus presentation apparatus for vehicles to which this invention is applied. It is a figure explaining the process which extracts a specific pattern from the landscape image ahead of the own vehicle image | photographed with the vehicle-mounted camera, (a) is an example of the landscape image image | photographed with the vehicle-mounted camera, (b) is a figure of (a). An edge image obtained by processing the landscape image by the contour detection unit, and (c) is a pattern image obtained by processing the edge image of (b) by the image filter unit. It is a figure explaining the process which extracts a specific pattern from the landscape image ahead of the own vehicle image | photographed with the vehicle-mounted camera, (a) is another example of the landscape image image | photographed with the vehicle-mounted camera, (b) is (a ) Is an edge image obtained by processing the landscape image by the contour detection unit, and (c) is a pattern image obtained by processing the edge image of (b) by the image filter unit. It is a figure which shows the example of the reference image pattern showing the whole landscape image ahead of the own vehicle obtained by the combination of the various road patterns memorize | stored in the reference image pattern memory | storage part, various vehicle patterns, etc. as a list. It is a figure which shows the mode of the time change of the road pattern and vehicle pattern in the scene where the own vehicle passes an oncoming vehicle, (a) is a pattern image in the state where the oncoming vehicle is away from the own vehicle, and (b) is opposite A pattern image in the process in which the vehicle approaches the own vehicle, (c) is a pattern image in a state where the oncoming vehicle is approaching the own vehicle. It is a figure which shows the mode of a change of the aspect of the stimulation sound output from a speaker in the scene in which the own vehicle passes an oncoming vehicle. It is a figure which shows the mode of the time change of the road pattern and vehicle pattern in the scene where the own vehicle overtakes the preceding vehicle, (a) is a pattern image in a state where the preceding vehicle is away from the own vehicle, (b) is the preceding A pattern image in the process in which the vehicle approaches the host vehicle, and (c) is a pattern image in a state in which the preceding vehicle approaches the host vehicle. It is a figure which shows the mode of a change of the aspect of the stimulation sound output from a speaker in the scene where the own vehicle overtakes a preceding vehicle. It is a flowchart which shows the flow of a series of operation | movement in the stimulus presentation apparatus for vehicles to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Car-mounted camera 2 Speed detection apparatus 3 Speaker 10 Image processing apparatus 11 Contour detection part 12 Image filter part 13 Reference image pattern memory | storage part 14 Image pattern identification part 15 Image pattern time change process part 16 Presentation sound process part

Claims (12)

An in-vehicle camera that captures landscape images in front of the vehicle,
Pattern extraction means for processing a landscape image taken by the in-vehicle camera and extracting a specific pattern from the landscape image;
A driving scene estimation unit that estimates a current driving scene of the vehicle based on the type and / or temporal change of the specific pattern extracted by the pattern extraction unit;
A stimulus generating means for generating a stimulus by sound and / or vibration;
A stimulus control unit that changes a mode of stimulation by sound and / or vibration generated by the stimulus generation unit in accordance with a current traveling scene of the host vehicle estimated by the traveling scene estimation unit ;
The pattern extraction means, from the image taken by the in-vehicle camera, includes a substantially rectangular pattern representing a vehicle contour and a straight line or curved line pattern representing a road side portion, a center line, or a lane marking line, as the specific pattern. Extract as
The traveling scene estimation means stores in advance a dimensional change on the image of a stationary object on the road with respect to the vehicle speed, and the dimensional change on the image of the rectangular pattern representing the vehicle contour is a stationary object on the road. If the dimensional change is larger than the dimensional change of the stationary object on the road, it is determined that the rectangular pattern is a pattern representing an oncoming vehicle. If it is smaller, it is determined that the rectangular pattern is a pattern representing a preceding vehicle . The stimulus generating means generates intermittent or continuous stimulus sound as the stimulus,
The stimulus control means is configured to adjust at least one of an output interval, a sound frequency, and a sound pressure of a stimulus sound generated by the stimulus generation means in accordance with a current running scene of the host vehicle estimated by the running scene estimation means. The vehicle stimulus presentation device according to claim 1, wherein the device is changed. The travel scene estimation means calculates a relative positional relationship between the oncoming vehicle and the own vehicle when it is estimated that the current travel scene of the own vehicle is a scene passing the oncoming vehicle,
2. The stimulus control means, wherein the stimulus generation means generates a stimulus that changes in accordance with a change in a relative positional relationship between the oncoming vehicle and the host vehicle calculated by the traveling scene estimation means. Or the stimulus presentation apparatus for vehicles of 2 . After the oncoming vehicle has moved out of the shooting range of the in-vehicle camera, the traveling scene estimation means is based on the relative speed and relative acceleration between the oncoming vehicle and the vehicle immediately before moving out of the shooting range of the in-vehicle camera. The vehicle stimulus presentation apparatus according to claim 3 , wherein a relative positional relationship between the oncoming vehicle and the host vehicle is estimated. The stimulation control means has a higher frequency before the own vehicle passes the oncoming vehicle than after the own vehicle passes the oncoming vehicle and is inversely proportional to the square of the distance between the own vehicle and the oncoming vehicle. so that the sound pressure, the vehicular stimulus presentation device according to claim 3 or 4, characterized in that changing the stimulation by sound of the stimulus generating means generates. When the driving scene estimation means estimates that the current driving scene of the own vehicle is a scene overtaking the preceding vehicle, the relative position relationship between the preceding vehicle and the own vehicle is calculated;
2. The stimulus control means, wherein the stimulus generation means generates a stimulus that changes in accordance with a change in a relative positional relationship between the preceding vehicle and the host vehicle calculated by the traveling scene estimation means. Or the stimulus presentation apparatus for vehicles of 2 . After the preceding vehicle has moved out of the shooting range of the in-vehicle camera, the traveling scene estimation means is based on the relative speed and relative acceleration between the preceding vehicle and the host vehicle immediately before moving out of the shooting range of the in-vehicle camera. The vehicle stimulus presentation apparatus according to claim 6 , wherein a relative positional relationship between the preceding vehicle and the host vehicle is estimated. The stimulus control means is configured such that the shorter the distance between the vehicle and the preceding vehicle, the shorter the interval, and the shorter the distance between the vehicle and the preceding vehicle, the intermittent sound with higher sound pressure and frequency. The stimulus presentation apparatus for a vehicle according to claim 6 or 7 , wherein the stimulus by the sound generated by the stimulus generation means is changed. The travel scene estimation means estimates the road width of the road on which the vehicle is currently traveling,
The stimulation control unit, the travel according to the scene of the road on which the vehicle estimated by the estimation means is currently traveling road width, claim 1 stimulation unit is characterized by changing the stimulus generated or 3. The vehicle stimulus presentation device according to 2. The travel scene estimation means estimates the shape of the road on which the vehicle is currently traveling,
The stimulation control unit, the travel according to the shape of the road on which the vehicle that is estimated is currently traveling by scene estimation means, claim 1 stimulation unit is characterized by changing the stimulus generated or 3. The vehicle stimulus presentation device according to 2. The travel scene estimation means estimates whether the road environment in which the vehicle is currently traveling is an urban area or a suburb,
The stimulation control unit, according to the road environment where the vehicle estimated by the running scene estimating means is currently traveling, according to claim 1 or 2, characterized in that varying the stimulus generating means generates stimulation The stimulus presentation device for a vehicle according to the above. A first step of taking a landscape image in front of the vehicle;
A second step of processing the landscape image taken in the first step and extracting a specific pattern from the landscape image;
A third step of estimating a current traveling scene of the vehicle based on the type and / or temporal change of the specific pattern extracted in the second step;
A fourth step of generating a stimulus by sound or vibration,
In the fourth step, the mode of stimulation by the generated sound or vibration is changed in accordance with the current driving scene of the vehicle estimated in the third step ,
In the second step, from the image captured in the first step, a substantially rectangular pattern representing a vehicle contour and a straight line or curved line pattern representing a road side portion, a center line, or a lane line are specified. Extracted as a pattern,
In the third step, a dimensional change on the image of the stationary object on the road with respect to the vehicle speed is stored in advance, and the dimensional change on the image of the rectangular pattern representing the vehicle contour is the stationary object on the road. If the dimensional change is larger than the dimensional change of the stationary object on the road, it is determined that the rectangular pattern is a pattern representing an oncoming vehicle. If it is small, it determines with the said rectangular pattern being a pattern showing a preceding vehicle, The stimulus presentation method characterized by the above-mentioned.