JP2009251761A - Driving support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving support system capable of urging a driver to visually confirm a dead angle range with simple configuration. <P>SOLUTION: The driving support system comprises an upper camera (110), a front camera (120), a control ECU (130) and a display part (140). A driver's head is imaged by the cameras 110, 120. The control ECU (130) generates information on the eye position and face direction of the driver from the picked-up image. The control ECU (130) computes a vehicle outside area visually confirmed by the driver, and when the vehicle outside area includes a predetermined area, controls a display part (140) to inform the driver to the effect. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a driving support system, and more particularly to a driving support system that is mounted on a vehicle and can prompt a driver of the vehicle to visually check a blind spot range.

2. Description of the Related Art Conventionally, there has been known a driving field assist device that detects an eye point of a driver who is driving a vehicle and detects a range that is a blind spot due to the structure of the vehicle from the eye point. As an example of such an apparatus, there is an apparatus disclosed in Patent Document 1, for example. The apparatus disclosed in Patent Literature 1 includes an eye point detection sensor that detects a driver's eye point, a calculation unit that calculates a blind spot range of the vehicle external environment based on the eye point, and an obstacle in the vehicle external environment. And an obstacle detection sensor for detecting. The device issues an alarm to the driver when there is an obstacle in a blind spot for the driver while the vehicle is stopped. Accordingly, the driver can know that an obstacle such as a person or a bicycle exists in the blind spot range of the outside of the vehicle.
Japanese Utility Model Publication No. 1-178700

  However, since the apparatus detects obstacles outside the vehicle by the obstacle sensor, for example, when the vehicle is temporarily stopped at the intersection, a person or a bicycle traveling at the intersection may be blind spot for the driver of the vehicle. Will almost cross the range. That is, as a result, the device always issues a warning to the driver, and the driver feels troublesome. In addition, when stopping at an intersection or the like, it is originally desirable to visually check the surroundings by moving the head of the driver himself. Cannot be judged.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driving support system that can prompt the driver to visually check the blind spot range with a simple configuration.

  In order to achieve the above object, the present invention has the following features. 1st invention is the driving assistance system which alert | reports to the driver | operator of a vehicle. The driving support system includes a first imaging unit, a second imaging unit, a line-of-sight direction calculating unit, a visual field region calculating unit, a blind spot region calculating unit, a currently viewing region calculating unit, a previously viewed region storing unit, a previously viewed region updating unit, And an overall visual field judging means. The first image pickup means and the second image pickup means pick up images of the head of the driver who is seated in the driver's seat of the vehicle from different directions. The line-of-sight direction calculating means detects the feature point of the driver using the images picked up by the respective image pickup means, and calculates the position and line-of-sight direction of the driver's eyes from the feature point. The field-of-view area calculation means is an area outside the vehicle that the driver will be able to see based on the position of the eyes and the direction of the line of sight, assuming that there are no obstacles that block the driver's field of vision. Calculate the field of view. The blind spot area calculating means calculates the area of the vehicle outside that will be a blind spot based on the position of the eyes and the direction of the line of sight and the position of a specific part of the vehicle that can block the view. The visually recognizing area calculation means calculates a visually recognizing area that is an area outside the vehicle that the driver will actually view by removing the blind spot area from the visually recognizing area. The already-viewed area storage means stores an area outside the vehicle that the driver would have already viewed as the already-viewed area. The already-viewed area update unit updates the already-viewed area stored in the already-viewed area storage unit so that the calculated currently-viewed area is included in the already-viewed area every time the currently-viewed area is calculated. The overall visual recognition determination means determines whether or not the driver has visually recognized the entire visual recognition required area, which is a predetermined area in the outside of the vehicle, based on the existing visual recognition area. The notifying means notifies the driver based on the determination result of the overall visual recognition determining means.

  In a second aspect based on the first aspect, the line-of-sight direction calculating means detects the driver's eyes using images captured by the first imaging means and the second imaging means, The eye information is detected as the feature point.

  According to a third aspect, in the first or second aspect, the currently-viewing area calculation unit determines whether the vehicle is stopped based on the vehicle information input to the currently-viewing area calculation unit, When it is determined that the vehicle is at a stop, a visible area, which is the area of the vehicle outside that the driver will actually see, is calculated, and the outside of the vehicle outside that the driver has already seen is calculated. The area is stored in the already-viewed area storage unit as the already-viewed area. Further, as the vehicle travels, the already-viewed area is completely erased from the already-viewed area storage unit.

  According to a fourth aspect, in the third aspect, the already-viewed area stored in the already-viewed area storage unit is sequentially deleted by the already-viewed area update unit after a predetermined time has elapsed.

  In a fifth aspect based on the fourth aspect, the notifying means is an indicator lamp provided in the vehicle, and the overall visual recognition determining means is configured so that the entire visual recognition area, which is a predetermined area in the external environment of the vehicle, is displayed to the driver. When it is determined that the vehicle has been visually recognized, the indicator lamp is turned on, and the indicator lamp is turned off when the vehicle travels.

  In a sixth aspect based on the first aspect, the feature point includes any of eyes, nose, nostril, mouth, eyebrows, and ears.

  According to the first aspect of the present invention, whether or not the driver himself / herself has confirmed the area of the vehicle outside that becomes a blind spot by the vehicle structure (such as a front pillar) for the driver seated in the driver's seat of the vehicle. Can be judged. And when the said driver | operator confirms the area | region of the said vehicle external world used as a blind spot, since it notifies that, the driver | operator can know that it confirmed also to the blind spot area | region. In other words, the driver can know that the blind spot area has not been sufficiently confirmed, and the driver can be prompted to visually confirm the blind spot range with a simple configuration.

  According to the second aspect, the line-of-sight direction calculating unit detects the eyes of the driver using the images captured by the first imaging unit and the second imaging unit, and the information about the eyes is used as the feature point. Detect as. Therefore, the visual field area of the driver can be accurately calculated.

  According to the third aspect of the present invention, if the vehicle stops once when entering an intersection or the like, it is determined whether or not the driver has confirmed a region where it is desirable to visually check the surroundings by moving his head. can do. Further, when the vehicle travels, the already-viewed area is completely erased from the already-viewed area storage unit. That is, when entering the next intersection or the like, if it stops once again, it can be newly determined whether or not an area that is desired to be visually confirmed is confirmed.

  According to the fourth aspect, when the vehicle is stopped, the currently-viewing area is stored in the already-viewed area storage unit at predetermined time intervals, and the currently-viewing area stored in the already-stored area storage unit is The data are sequentially erased after a predetermined time has elapsed. Therefore, it is possible to accurately notify the driver even in a road environment that changes every moment so that a person or a bicycle does not go.

  According to the fifth aspect of the invention, the driver can know that the region that is desired to be visually confirmed can be confirmed. If the indicator lamp does not light up, the driver can know that “there is an area that has not been visually recognized yet”.

  According to the sixth aspect, it is possible to detect feature points such as eyes, nose, nostril, mouth, eyebrows, and ears.

  Hereinafter, a driving support system according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, description will be made assuming that the driving support system is mounted on a vehicle.

  First, an outline of the driving support system of the present embodiment will be briefly described.

  The driver needs to start the vehicle after confirming the front of the vehicle (including right front and left front) after temporarily stopping before the intersection. FIG. 1 is a view when the vehicle is temporarily stopped before entering the intersection. As shown in FIG. 1, when the driver looks at the right front of the vehicle from the position a, a blind spot is generated in the outside of the vehicle due to the front pillar of the vehicle. Therefore, even if there is a motorcycle, a pedestrian, or the like approaching the blind spot from the right side of the vehicle, the driver may not notice the presence of the motorcycle. Therefore, before starting the vehicle, the driver changes the position of the head (for example, the position of the head in front of the position a above) in order to confirm that there are no motorcycles or pedestrians in such a blind spot. It is desirable to visually confirm the front of the vehicle (especially right front and left front of the vehicle). The driving support system of the present embodiment determines whether or not the driver has visually recognized the area to be visually recognized when the driver visually confirms the front of the vehicle while changing the position of the head in this way, The judgment result is presented to the driver.

  FIG. 2 is a block diagram illustrating a configuration of a driving support system according to an embodiment. As shown in FIG. 2, the driving support system system includes an upper camera 110, a front camera 120, a control ECU 130, and a display unit 140. FIG. 3 is a view showing a state of the interior of the vehicle equipped with the driving support system.

  The upper camera 110 is installed at a position where the head of the driver seated in the driver's seat can be imaged from above. As an example of the installation location of the upper camera 110, as shown in FIG. 3, a position where an image can be taken from the upper front of the driver seated in the driver's seat, such as the ceiling of the vehicle, can be cited. Moreover, the upper camera 110 images the driver's head at predetermined time intervals, for example. Then, the image captured by upper camera 110 is output to control ECU 130. Note that an image captured by the upper camera 110 is hereinafter referred to as an upper image.

  The front camera 120 is installed at a position where the head (more specifically, the face) of the driver seated in the driver's seat can be imaged from the front. As an example of the installation location of the front camera 120, as shown in FIG. 3, there is a position where the face of the driver seated in the driver's seat such as on the steering column of the vehicle can be imaged from the front. In addition, the front camera 120 images the driver's face from the front at predetermined time intervals, for example. Then, the image captured by front camera 120 is output to control ECU 130. Note that an image captured by the front camera 120 is hereinafter referred to as a front image. The upper camera 110 and the front camera 120 correspond to an example of the first imaging unit and the second imaging unit of the present invention.

  The control ECU 130 includes an image processing unit 131, a storage unit 132, and a face image pattern storage unit 133. The control ECU 130 is typically a microcomputer including a central processing unit, a readable / writable storage medium, and the like. The control ECU 130 processes the images captured by the cameras 110 and 120 and calculates the position of the driver's eyes in the vehicle room space. Then, based on the information on the position of the eyes, an area of the vehicle external environment that can be visually recognized by the driver is calculated. Further, the display unit 140 is controlled based on the information related to the calculated area. The control ECU 130 may acquire vehicle information of a vehicle on which the driving support system is mounted. The vehicle information is, for example, the traveling speed of the vehicle.

  As will be described in detail later, the image processing unit 131 detects the driver's eyes from the image captured by the upper camera 110 and the image captured by the front camera 120, and calculates the position of the driver's eyes in each image. Further, the image processing unit 131 includes information on the position (x, y, z) where the driver's eyes are present in the vehicle cabin space from the position of the eyes and the installation positions of the cameras 110 and 120, Information on the driver's face orientation angle θ is calculated. Note that the position (x, y, z) where the driver's eyes exist is, as shown in FIG. 3, with an arbitrary point in the passenger compartment space of the vehicle as an origin, and the lateral width direction of the vehicle as an x coordinate, When a coordinate system is used in which the height direction is the y-coordinate and the length direction is the z-coordinate, this is the position where the eyes of the driver are present in the vehicle cabin space.

  Furthermore, the image processing unit 131 determines the vehicle external environment that the driver will actually see based on the calculated information (specifically, the eye position (x, y, z) and the face orientation angle θ). This area is calculated, and it is determined whether or not the area that would have been viewed can cover a predetermined area (specifically, a blind spot area caused by a front pillar or the like). Then, the image processing unit 131 controls the display unit 140 based on the determination result. The image processing unit 131 corresponds to an example of a line-of-sight direction calculating unit, a visual field region calculating unit, a blind spot region calculating unit, a currently-viewing region calculating unit, an already-viewed region updating unit, and an overall visual determining unit.

  The storage unit 132 is a readable / writable storage medium for temporarily storing various types of information calculated by the image processing unit 131. The storage unit 132 corresponds to an example of the already-viewed area storage unit of the present invention.

  The face image pattern storage unit 133 is configured by a readable / writable storage medium, and stores information indicating a person's face image pattern. Specifically, the face image pattern includes various patterns such as a face size, an eye size, and a nose size. The face image pattern includes template images (for example, eye template images) of various human face parts. Note that the face information stored in the face image pattern storage unit 133 may be appropriately updated by, for example, rewriting work of the user of the vehicle.

  The display unit 140 turns on or off a warning lamp provided in the vehicle according to an instruction from the image processing unit 131. The display unit 140 corresponds to an example of the notification unit of the present invention.

  As described above, the driving support system according to the embodiment images the head of the driver with the cameras 110 and 120. Then, the image processing unit 131 generates information on the driver's eye position (x, y, z) and the face orientation angle θ in the vehicle interior space of the vehicle from the captured image. That is, the image processing unit 131 can calculate the direction of the driver's line of sight and the position of the eyes. Further, the image processing unit 131 calculates the area of the vehicle outer environment visually recognized by the driver from the direction of the line of sight and the position of the eyes, and the driver calculates the entire area of the vehicle outer environment (the setting area S described later). When it is confirmed that the user has visually confirmed, the display unit 140 is controlled to notify the driver of the fact.

  Hereinafter, an example of processing performed by each unit of the driving support system according to the embodiment will be described with reference to FIGS.

  4 to 5 are flowcharts illustrating an example of a flow of processing performed by each unit of the driving support system according to the embodiment. 4 to 5 is started when the power of the driving support system is turned on (for example, an ignition switch of a vehicle on which the driving support system is mounted is turned on). In addition, the processing ends when the driving support system is turned off (for example, an ignition switch is turned off). The ignition switch is hereinafter referred to as IG.

  In step S40 of FIG. 4, the image processing unit 131 initializes the value of the variable n to 1 (n = 1).

  In step S <b> 41, the image processing unit 131 acquires images (upper image and front image) output from the cameras 110 and 120. Then, the image processing unit 131 advances the processing to the next step S42. At this time, the image processing unit 131 may temporarily store the acquired images in the storage unit 132.

  In step S42, the image processing unit 131 calculates the position P of the driver's eyes in the passenger compartment space of the vehicle from the upper image and the front image. Hereinafter, a method of calculating the eye position of the driver performed by the image processing unit 131 will be described.

  FIG. 6 is a flowchart illustrating an example of an operation of calculating the driver's eye position performed by the image processing unit 131 in step S42 of FIG. FIG. 7 is an example of a driver's face image captured by the front camera 120. FIG. 8 is an example of a driver's head image captured by the upper camera 110. FIG. 9 is a diagram when the head of the driver is imaged by the upper camera 110 and the front camera 120.

  In step S61 of FIG. 6, the image processing unit 131 includes a driver's face width (hereinafter referred to as face width W) and face length (hereinafter referred to as length L) in the front image acquired in step S41. Calculated). Specifically, the image processing unit 131 performs a Sobel filter process in a predetermined direction in the front image, and uses the luminance difference in the processed Sobel image to extract an edge point from the Sobel image. Do. As described above, the image processing unit 131 can detect the upper end, the lower end, the right end, and the left end of the driver's face in the front image by performing the process of extracting the edge points (see FIG. 7). Then, the image processing unit 131 calculates the face width W and the length L in the Sobel image using the edge points extracted from the Sobel image, and proceeds to the next Step S62. At this time, the image processing unit 131 may temporarily store the calculated face width W and length L information in the storage unit 132.

  In step S <b> 62, the image processing unit 131 refers to the face width W and length L stored in the storage unit 132 and detects the driver's nostril in the front image. As an example of a specific method, the image processing unit 131 refers to the face width W and length L detected in step S61 and the face image pattern stored in advance in the face image pattern storage unit 133. A range where a nostril is expected to be present in the front image, that is, a nostril search range is set. Then, the image processing unit 131 detects pixels having relatively low luminance from the image within the set nostril search range, and calculates the circularity of the low luminance pixel group. Then, the image processing unit 131 detects the point group having the highest circularity (close to a circle) as the driver's nostril, and proceeds to the next step S63.

  In step S63, the image processing unit 131 calculates the position point FP of the driver's eyes in the front image. For example, the image processing unit 131 estimates the position of the eye in the front image on the basis of the position of the driver's nostril detected in step S62. Then, the image processing unit 131 performs image processing for extracting an edge line at the estimated eye position in the front image. Further, the image processing unit 131 uses the eye template image stored in the face image pattern storage unit 133 to perform pattern matching processing on the front image from which the edge line has been extracted, and the position of the right eye in the front image. And the position of the left eye. Next, the image processing unit 131 calculates an intermediate position between the right eye position and the left eye position in the front image as the eye position point FP, and proceeds to the next step S64 (see FIG. 7). At this time, the image processing unit 131 may temporarily store the information on the eye position in the storage unit 132.

  In step S64, the image processing unit 131 performs a process of extracting eye edge points from the upper image. For example, the image processing unit 131 performs Sobel filter processing in a predetermined direction in the upper image. Then, the image processing unit 131 detects edge points within a range in which the driver's eyes are expected to be present in the upper image from the width and length of the head, and extracts edge lines.

  In step S65, the image processing unit 131 calculates the position point TP of the driver's eyes in the upper image. Specifically, the image processing unit 131 uses the eye template image stored in the face image pattern storage unit 133 to perform pattern matching processing on the upper image from which the edge line has been extracted. Specify the position of the right eye and the position of the left eye. Next, the image processing unit 131 calculates an intermediate point between the position of the right eye and the position of the left eye in the upper image as the eye position point TP (see FIG. 8). Then, the image processing unit 131 proceeds to the next step S66. Note that the process of calculating the eye position from the upper image by the image processing unit 131 may be performed by the same process as the process of calculating the eye position in the front image as described above. Further, the image processing unit 131 may estimate the position of the driver's eyes in the upper image from the edge of the head (edge line), and calculate the eye position point TP.

  In step S66, the image processing unit 131 calculates an angle between the camera optical axis and each point. Specifically, as shown in FIG. 9, the image processing unit 131 detects the angle formed by the upper end of the driver's face in the front image and the optical axis of the front camera 120, and the driver's eyes in the front image. An angle formed by a straight line passing through the position point FP and the camera optical axis of the front camera 120 is calculated. Similarly, the image processing unit 131 determines that the driver in the upper image also has an angle formed by the tip of the head and the optical axis of the upper camera 110, a straight line passing through the driver's eye position point TP in the upper image, and the upper camera. An angle formed by 110 optical axes is calculated. Then, the image processing unit 131 advances the processing to the next step S67.

  In step S67, the image processing unit 131 determines the driver's eye position point FP and point TP from the angles calculated in step S66, the installation positions of the cameras 110 and 120 in the vehicle, and the like. This is converted to the eye position P of the driver in the space.

  As described above, the image processing unit 131 can calculate the position P of the driver's eyes in the passenger compartment space by the processes in steps S61 to S67. In other words, the image processing unit 131 can determine where the driver's eyes are present in the passenger compartment space.

  Note that the above-described example is an example, and other examples may be used as long as the method can calculate the position of the eye in the space by two cameras.

  Returning to FIG. 4, after the calculation process of the driver's eye position P in step S <b> 67, in step S <b> 43, the image processing unit 131 calculates the driver's face orientation angle θ. As an example of specific processing, the image processing unit 131 calculates the position of the nose and eyes and the face width W in the steps so far. The image processing unit 131 obtains the center line of the face from the calculated nose and eye positions and the face width W, calculates the left / right ratio of the face from the center line of the face, and converts the left / right ratio into an angle. Then, the driver's face orientation angle θ is calculated. Note that the face orientation angle θ is θ = 0 in a state of facing the front camera 120. Then, the value of the face orientation angle θ increases as the face turns to the right from the front facing the front camera 120. On the other hand, the value of the face angle θ decreases as the face turns to the left from the state where the face faces the front camera 120. That is, the face angle θ becomes a positive value when the driver turns to the right in the traveling direction, and becomes a negative value when the driver turns to the left. Note that the above-described method by which the image processing unit 131 calculates the face orientation angle θ is an example. Therefore, the method of calculating the face orientation angle θ is not limited to the above-described example, and a conventional known method may be used.

  As described above, the image processing unit 131 can calculate the eye position P and the face orientation angle θ at that time by performing the processing from step S41 to step S43 described above. That is, the image processing unit 131 can calculate from which position the driver is looking in which direction (the driver's eye position and line-of-sight direction).

  In step S44, the image processing unit 131 determines whether the vehicle is stopped based on the vehicle information. Specifically, the image processing unit 131 determines whether or not the vehicle is stopped based on the traveling speed of the vehicle. In this step, if the determination result is affirmative (YES), that is, if the image processing unit 131 determines that the vehicle is stopped, the process proceeds to step S45. On the other hand, if the determination result is negative (NO), the process proceeds to step S415.

  In step S45, the image processing unit 131 detects the presence of the vehicle front pillar and the like based on the information generated in steps S43 and S44 (the driver's eye position P and the face angle θ in the vehicle cabin). When not considered, that is, when it is assumed that there are no obstacles that block the driver's field of vision, the visual field region α of the outside of the vehicle that the driver will be able to visually recognize is calculated. FIG. 10 shows the visual field region α when the driver's eye position is P1 and the face orientation angle is θ1. The visual field region α may be calculated in consideration of the visual field angle that can be generally visually recognized by the human eye based on the driver's eye position and face orientation angle.

  In step S46, the image processing unit 131 is prepared in advance with the information generated in steps S43 and S44 (the driver's eye position P (x, y, z) and face orientation angle θ in the vehicle cabin). Based on the positional information of the vehicle structure (for example, the front pillar), a blind spot area β of the outside of the vehicle generated by the structure is calculated. FIG. 11 shows the blind spot region β when the driver's eye position is P1 and the face angle is θ1. Then, the image processing unit 131 advances the processing to step S47.

  In step S47, the image processing unit 131 calculates a visible area γ (n) that is an area outside the vehicle that the driver can actually see by removing the blind spot area β from the visual field area α. FIG. 12 shows the visually recognizing region γ when the eye position of the driver is P1 and the face orientation angle is θ1. Then, the image processing unit 131 advances the processing to step S48 in FIG.

  In step S <b> 48 of FIG. 5, the image processing unit 131 adds the currently-viewed region γ (n) calculated in step S <b> 47 to the already-viewed region δ stored in the storage unit 132. The already-viewed region δ is a region actually viewed by the driver during a period (for example, the past 5 seconds) from a time point before a certain time to the current time point. Note that when adding the currently-viewed region γ (n) to the already-viewed region δ, the value of the variable n (or information regarding the time at which the currently-viewed region γ (n) was generated) is also stored together. Remember it.

  In step S <b> 49, the image processing unit 131 deletes a region that has not been viewed at all for a certain period of time from the already-viewed region δ stored in the storage unit 132. The determination of the area to be deleted is performed based on the value of the variable n stored together with the storage unit 132 when the currently-viewing area γ (n) is added to the already-viewed area δ. As described above, when adding the currently visible region γ (n) to the existing visually recognized region δ, information related to the time when the currently visible region γ (n) was generated is also stored in the storage unit 132 together. In this case, the area to be deleted can be determined based on the information regarding this time.

  In step S410, the image processing unit 131 determines whether or not the already-viewed region δ stored in the storage unit 132 covers the entire setting region S. The setting area S is an area defined in advance as an area to be visually recognized by the driver for safety confirmation before the vehicle starts. In this embodiment, the setting area S is in contact with the front end of the vehicle as shown in FIG. A rectangular area. The setting area S is not limited to a rectangle, but may be another shape. Furthermore, the setting area S may be composed of a plurality of areas separated from each other. If the determination result in step S410 is affirmative (YES), the image processing unit 131 advances the process to the next step S411. On the other hand, when the determination result is negative (NO), the image processing unit 131 proceeds to step S412 and instructs the lamp to be turned off when the lamp of the display unit 140 is turned on.

  In step S411, the image processing unit 131 controls the display unit 140 to turn on the lamp. Here, the case where the lamp is lit is a case where the already-viewed region δ covers the entire setting region S. That is, when the driver visually recognizes an area outside the vehicle that is required to be visually recognized for safety, the lamp is turned on. On the other hand, if there remains an area in the setting area S that the driver has not yet visually recognized, the lamp will not be lit. Thus, the driver can know whether or not all the areas that are required to be visually confirmed for safety have been confirmed. In other words, if the display unit 140 is not turned on, the driver can know that “there is an area that has not been visually recognized yet”. That is, the driving support system according to the present embodiment can further prompt the driver to check the outside of the vehicle.

  In step S413, the control ECU 130 determines whether or not the ignition switch of the vehicle is turned off. If the control ECU 130 determines that the ignition switch has been turned off (YES), the control ECU 130 ends the process of the flowchart. On the other hand, when the control ECU 130 determines that the ignition switch of the vehicle is not turned off (NO), the control ECU 130 increments the variable n (n = n + 1) in step S414, and then returns to step S41 to repeat the process.

  In step S <b> 415, the image processing unit 131 completely erases the already-viewed area δ stored in the storage unit 132 from the storage unit 132. At this time, the value of the variable n may be initialized to 1 as necessary.

  Thus, according to the driving support system according to the present embodiment, the head of the driver is imaged by the upper camera 110 and the front surface 120. Then, the image processing unit 131 generates information regarding the driver's eye position and face orientation from the captured image. Then, the image processing unit 131 calculates the area of the vehicle external environment visually recognized by the driver, and when the area of the vehicle external environment visually recognized by the driver can cover the predetermined area (setting area S), the display unit 140 is displayed. The driver can be notified to that effect.

  Further, the driving support system according to the present embodiment sequentially stores, as the already-viewed area δ, the area outside the vehicle visually recognized by the driver when the vehicle is stopped. Among the already-viewed areas δ, areas that have not been viewed by the driver for a certain time (for example, 5 seconds) are appropriately deleted from the already-viewed areas δ. Accordingly, even when the vehicle starts after stopping for a long time, it is possible to accurately notify the driver whether or not safety has been confirmed immediately before starting.

  Next, an example of a method for storing the already-viewed area δ will be described. In this embodiment, as shown in FIG. 14, a storage area of a k × m two-dimensional array corresponding to the setting area S is prepared in the storage unit 132. The initial value of each element of this two-dimensional array is 0. First, when adding the currently-viewing region γ1 to the already-viewed region δ, as shown in FIG. 15, the value of the element corresponding to the currently-viewing region γ1 is rewritten to 1. Subsequently, when adding the visually-recognized region γ2 to the already-recognized region δ, as shown in FIG. 16, the value of the element corresponding to the visually-recognized region γ2 is rewritten to 2. Subsequently, when adding the visually-recognized region γ3 to the already-recognized region δ, the value of the element corresponding to the visually-recognized region γ3 is rewritten to 3. Similarly, when adding the currently visible region γn to the already-viewed region δ, the value of the element corresponding to the currently visible region γn is rewritten to n. As a result, the value of each element increases each time the driver visually recognizes the area corresponding to the element, but conversely, the value of the element corresponding to the area that the driver has not seen at all for a long time is long. The time will not be updated. Thus, if the difference between the value of each element and the value of the current variable n is calculated, the period during which the driver does not visually recognize the area corresponding to the element can be easily calculated. For example, an area that has not been viewed for more than 5 seconds can be easily excluded from the already-viewed area δ.

  The embodiment of the method for storing the already-viewed area δ is not limited to the above-described method. For example, assuming that the setting area is around the front of the vehicle (for example, left and right azimuth angles with respect to the traveling direction of the vehicle), a one-dimensional array of storage areas may be prepared. .

  The aspect described in the above embodiment is merely a specific example, and does not limit the technical scope of the present invention. Therefore, it is possible to employ any configuration within the range where the effects of the present application are achieved.

  The driving support system according to the present invention is useful as a driving support system mounted on a vehicle or the like that can prompt the driver to visually check the blind spot range with a simple configuration.

The figure for demonstrating the outline | summary of the driving assistance system which concerns on embodiment. The block diagram which shows the structure of the driving assistance system which concerns on embodiment. A diagram showing the interior of a vehicle equipped with a driving support system The flowchart which showed the first half of an example of the flow of the processing which each part of a driving support system performs The flowchart which showed the second half of an example of the flow of processing which each part of a driving support system performs FIG. 4 is a flowchart illustrating an example of an operation for calculating the driver's eye position performed by each unit of the control ECU 130 in step S42. Example of driver's face image captured by front camera 120 Example of driver's head image taken by upper camera 110 The figure when imaging the driver's head with the upper camera 110 and the front camera 120 The figure which showed the visual field area | region (alpha) when the position of a driver | operator's eyes is P1 and a face direction angle is (theta) 1. The figure which showed the blind spot area | region (beta) when the position of a driver | operator's eyes is P1 and a face direction angle is (theta) 1. A diagram showing a currently visible region γ when the driver's eye position is P1 and the face angle is θ1. The figure which showed an example of setting field S Conceptual diagram of storage area of k × m two-dimensional array corresponding to setting area S The figure which showed the element corresponding to the visual recognition area | region γ1 in the storage area of the k × m two-dimensional array The figure which rewritten the element corresponding to field γ2 under visual recognition to the storage area of the 2 × array of k × m

Explanation of symbols

110 Upper camera 120 Front camera 130 Control ECU
131 Image processing unit 132 Storage unit 133 Face image pattern storage unit 140 Display unit

Claims (6)

A driving support system for notifying a driver of a vehicle,
First imaging means and second imaging means for imaging the driver's head seated in the driver's seat of the vehicle from different directions;
A gaze direction calculation that detects a feature point of the driver using images captured by the first imaging unit and the second imaging unit, and calculates a position and a gaze direction of the driver's eyes from the feature point. Means,
Based on the driver's eye position and line-of-sight direction calculated by the line-of-sight direction calculation means, the driver can visually recognize that there is no obstacle that blocks the driver's view. Visual field region calculating means for calculating a visual field region which is the region of the outside of the vehicle,
Based on the position and line-of-sight direction of the driver's eyes calculated by the line-of-sight direction calculating means and the position of the specific part of the vehicle that can obstruct the driver's field of view, A blind spot area calculating means for calculating a blind spot area that is an area of the vehicle external world that would be
By visually removing the blind spot area calculated by the blind spot area calculating means from the visual field area calculated by the visual field area calculating means, the driver is actually viewing the area outside the vehicle. A currently-viewing area calculating means for calculating the area;
An already-viewed area storage means for storing an area of the outside of the vehicle that the driver would have already viewed as an already-viewed area;
The already-viewed area stored in the already-viewed area storage unit so that the calculated currently-viewed area is included in the already-viewed area each time the currently-viewed area is calculated by the currently-viewed area calculating unit. An already-viewed area updating means for updating the area;
Overall viewing for determining whether or not the driver has viewed the entire viewable area that is a predetermined area of the outside of the vehicle based on the already-viewed area stored in the already-viewed area storage unit. A determination means;
A driving support system comprising: notifying means for notifying the driver based on the determination result of the overall visual recognition determining means.   The line-of-sight direction calculating means detects eyes of the driver using images picked up by the first imaging means and the second imaging means, and detects information on the eyes as the feature points. The driving support system according to claim 1, wherein the driving support system is characterized.   The currently-viewing area calculation means determines whether or not the vehicle is stopped based on vehicle information input to the currently-viewing area calculation means, and determines that the vehicle is stopped when the vehicle is stopped. The currently visible region, which is the region of the vehicle external environment that the driver will actually view, is calculated, and the region of the vehicle external environment that the driver has already viewed is calculated as the already viewed region. The driving assistance system according to claim 1 or 2, wherein the already-viewed area is erased from the already-viewed area storing means by storing in the means and the vehicle travels.   The driving assistance system according to claim 3, wherein the already-viewed area stored in the already-viewed-area storage unit is sequentially deleted by the already-viewed area updating unit after a predetermined time has elapsed.   The notification means is an indicator lamp provided in the vehicle, and when the overall visual recognition determination means determines that the driver has visually recognized the entire visual recognition area that is a predetermined area of the outside of the vehicle. The driving support system according to claim 4, wherein the indicator lamp is turned on and the indicator lamp is turned off when the vehicle travels.   The driving support system according to claim 1, wherein the feature points include any of eyes, nose, nostrils, mouth, eyebrows, and ears.

Images